diff options
362 files changed, 18720 insertions, 12017 deletions
diff --git a/.gitignore b/.gitignore index cba7caa074..aa32bd9582 100644 --- a/.gitignore +++ b/.gitignore @@ -182,6 +182,8 @@ a.out.dSYM/ /lib/jinterface/priv/OtpErlang.jar /lib/jinterface/priv/com/ /lib/jinterface/doc/html/java +/lib/jinterface/pom.xml +/lib/jinterface/target # kernel diff --git a/INSTALL-WIN32.md b/INSTALL-WIN32.md index 643cd33c84..6481ca1b06 100644 --- a/INSTALL-WIN32.md +++ b/INSTALL-WIN32.md @@ -182,7 +182,7 @@ Frequently Asked Questions include workarounds for the bugs I've found in different Cygwin releases, please help me to add workarounds for new Cygwin-related bugs as soon as you encounter them. Also please do submit bug reports - to the appropriate Cygwin developers. The Cygwin GCC we used for R13B + to the appropriate Cygwin developers. The Cygwin GCC we used for %OTP-REL% was version 3.4.4. We used VC++ 8.0 (i.e. Visual studio 2005 SP1), Sun's JDK 1.5.0\_17, NSIS 2.37, and Win32 OpenSSL 0.9.8e. Please read the next section for details on what you need. diff --git a/Makefile.in b/Makefile.in index ac2015f8e4..7bdcdcc297 100644 --- a/Makefile.in +++ b/Makefile.in @@ -119,7 +119,7 @@ BINDIR = $(DESTDIR)$(EXTRA_PREFIX)$(bindir) # # Erlang base public files # -ERL_BASE_PUB_FILES=erl erlc epmd run_erl to_erl dialyzer typer escript +ERL_BASE_PUB_FILES=erl erlc epmd run_erl to_erl dialyzer typer escript run_test # ERLANG_INST_LIBDIR is the top directory where the Erlang installation # will be located when running. diff --git a/erts/Makefile.in b/erts/Makefile.in index fabf86db7c..dc8edcd928 100644 --- a/erts/Makefile.in +++ b/erts/Makefile.in @@ -1,19 +1,19 @@ # # %CopyrightBegin% -# -# Copyright Ericsson AB 2006-2009. All Rights Reserved. -# +# +# Copyright Ericsson AB 2006-2010. All Rights Reserved. +# # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in # compliance with the License. You should have received a copy of the # Erlang Public License along with this software. If not, it can be # retrieved online at http://www.erlang.org/. -# +# # Software distributed under the License is distributed on an "AS IS" # basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See # the License for the specific language governing rights and limitations # under the License. -# +# # %CopyrightEnd% # include $(ERL_TOP)/make/target.mk @@ -92,10 +92,12 @@ local_setup: $(ERL_TOP)/bin/escript $(ERL_TOP)/bin/escript.exe \ $(ERL_TOP)/bin/dialyzer $(ERL_TOP)/bin/dialyzer.exe \ $(ERL_TOP)/bin/typer $(ERL_TOP)/bin/typer.exe \ + $(ERL_TOP)/bin/run_test $(ERL_TOP)/bin/run_test.exe \ $(ERL_TOP)/bin/start*.boot $(ERL_TOP)/bin/start*.script @if [ "X$(TARGET)" = "Xwin32" ]; then \ cp $(ERL_TOP)/bin/$(TARGET)/dialyzer.exe $(ERL_TOP)/bin/dialyzer.exe; \ cp $(ERL_TOP)/bin/$(TARGET)/typer.exe $(ERL_TOP)/bin/typer.exe; \ + cp $(ERL_TOP)/bin/$(TARGET)/run_test.exe $(ERL_TOP)/bin/run_test.exe; \ cp $(ERL_TOP)/bin/$(TARGET)/erlc.exe $(ERL_TOP)/bin/erlc.exe; \ cp $(ERL_TOP)/bin/$(TARGET)/erl.exe $(ERL_TOP)/bin/erl.exe; \ cp $(ERL_TOP)/bin/$(TARGET)/werl.exe $(ERL_TOP)/bin/werl.exe; \ @@ -115,6 +117,7 @@ local_setup: $(ERL_TOP)/erts/etc/unix/cerl.src > $(ERL_TOP)/bin/cerl; \ cp $(ERL_TOP)/bin/$(TARGET)/dialyzer $(ERL_TOP)/bin/dialyzer; \ cp $(ERL_TOP)/bin/$(TARGET)/typer $(ERL_TOP)/bin/typer; \ + cp $(ERL_TOP)/bin/$(TARGET)/run_test $(ERL_TOP)/bin/run_test; \ cp $(ERL_TOP)/bin/$(TARGET)/erlc $(ERL_TOP)/bin/erlc; \ cp $(ERL_TOP)/bin/$(TARGET)/escript $(ERL_TOP)/bin/escript; \ chmod 755 $(ERL_TOP)/bin/erl $(ERL_TOP)/bin/erlc \ diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 77a628e82b..54a0a80536 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -2767,8 +2767,10 @@ os_prompt%</pre> Otherwise, some other point in time is chosen. It is also guaranteed that subsequent calls to this BIF returns continuously increasing values. Hence, the return value from - <c>now()</c> can be used to generate unique time-stamps. It - can only be used to check the local time of day if + <c>now()</c> can be used to generate unique time-stamps, + and if it is called in a tight loop on a fast machine + the time of the node can become skewed.</p> + <p>It can only be used to check the local time of day if the time-zone info of the underlying operating system is properly configured.</p> </desc> diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index a4867adf22..6174917807 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -30,6 +30,531 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 5.8</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Driver threads, such as async threads, using <seealso + marker="erl_driver#ErlDrvPDL">port data locks</seealso> + peeked at the port status field without proper locking + when looking up the driver queue.</p> + <p> + Own Id: OTP-8475</p> + </item> + <item> + <p> + The use of <c>mmap()</c> was unnecessarily disabled when + cross compiling.</p> + <p> + The <c>configure</c> arguments <c>--with-ssl</c>, and + <c>--with-odbc</c> refused to accept libraries outside of + <c>$erl_xcomp_sysroot</c> when cross compiling for no + good reason.</p> + <p> + The <c>configure</c> argument <c>--with-odbc</c> didn't + handle the value <c>yes</c> correct.</p> + <p> + The <c>configure</c> arguments <c>--with-odbc</c>, and + <c>--without-odbc</c> have also been added to the + configure help.</p> + <p> + (Thanks to Steve Vinoski for reporting these issues)</p> + <p> + Own Id: OTP-8484</p> + </item> + <item> + <p> + A call to the BIF <c>unregister(RegName)</c> when a port + had the name <c>RegName</c> registered in the runtime + system without SMP support caused a runtime system crash. + (Thanks to Per Hedeland for the bugfix and test case.)</p> + <p> + Own Id: OTP-8487</p> + </item> + <item> + <p> + The runtime system crashed if fewer logical processors + were found than reported by <c>sysconf( + SC_NPROCESSORS_CONF)</c>.</p> + <p> + Own Id: OTP-8549</p> + </item> + <item> + <p> + Fix memory management bug causing crash of non-SMP + emulator with async threads enabled. The bug did first + appear in R13B03.</p> + <p> + Own Id: OTP-8591 Aux Id: seq11554 </p> + </item> + <item> + <p> + Port locks could be prematurely destroyed.</p> + <p> + Own Id: OTP-8612</p> + </item> + <item> + <p>The <c>empd</c> program could loop and consume 100% + CPU time if an unexpected error ocurred in + <c>listen()</c> or <c>accept()</c>. Now <c>epmd</c> will + terminate if a non-recoverable error occurs. (Thanks to + Michael Santos.)</p> + <p> + Own Id: OTP-8618</p> + </item> + <item> + <p> + When kernel poll has been enabled, a livelock could in + rare circumstances occur. Problem reported by Chetan + Ahuja, fix by Mikael Pettersson.</p> + <p> + Own Id: OTP-8632</p> + </item> + <item> + <p> + Windows: Closing port of program that stalled without + reading all data could deadlock scheduler thread.</p> + <p> + Own Id: OTP-8641</p> + </item> + <item> + <p> + On some combination of Montavista Linux on Cavium Octeon + processors, some socket-related system calls returned + other numbers than -1 for errors. This caused a core dump + in inet_drv.c. Now the code works around this problem.</p> + <p> + Own Id: OTP-8654</p> + </item> + <item> + <p> + Missing memory barriers in <c>erts_poll()</c> could cause + the runtime system to hang indefinitely.</p> + <p> + Own Id: OTP-8658</p> + </item> + <item> + <p> + <c>ethr_rwmutex_tryrlock()</c> acquired and refused to + acquire a lock with inverted logic. The lock was however + never acquired in a thread unsafe manner. (Thanks to JR + Zhang for noting this issue)</p> + <p> + Own Id: OTP-8662</p> + </item> + <item> + <p> + Extreme combinations of register/unregister in a highly + parallell SMP application could crash the VM. The error + is corrected.</p> + <p> + Own Id: OTP-8663</p> + </item> + <item> + <p> + On Windows, files are now opened with FILE_SHARE_DELETE + to get closer to Unix semantics.</p> + <p> + Own Id: OTP-8667</p> + </item> + <item> + <p> + <c>erlang:system_info(multi_scheduling)</c> sometimes + erroneously returned <c>enabled</c> when it should have + returned <c>blocked</c>.</p> + <p> + Own Id: OTP-8675</p> + </item> + <item> + <p> + Fix bug causing <c>erlang:decode_packet</c> and + <c>enif_make_string</c> to generate faulty strings with + negative character values for ascii values larger than + 127. (Thanks to Paul Guyot)</p> + <p> + Own Id: OTP-8685</p> + </item> + <item> + <p> + <c>open_port/2</c> with the <c>spawn</c> and + <c>spawn_executable</c> options can include an + <c>{env,Env}</c> option. In some cases unsetting + variables would not work on Unix (typically if more + variables were unset than were actually present in the + environment).</p> + <p> + Own Id: OTP-8701</p> + </item> + <item> + <p> + A user defined CPU topology set via a call to <seealso + marker="erlang#system_flag_cpu_topology">erlang:system_flag(cpu_topology, + CPUTopology)</seealso> was not properly verified, and + could in worst case cause an emulator crash. The emulator + crash could only occur when a user defined CPU topology + already existed and was redefined.</p> + <p> + Own Id: OTP-8710</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The grammar for match specifications in ERTS users guide + only described the tracing dialect of match + specifications. An informal grammar for the ETS dialect + is added.</p> + <p> + Own Id: OTP-8086 Aux Id: seq11333 </p> + </item> + <item> + <p> + The module binary from EEP31 (and EEP9) is implemented.</p> + <p> + Own Id: OTP-8217</p> + </item> + <item> + <p> + New NIF API function <c>enif_make_new_binary</c></p> + <p> + Own Id: OTP-8474</p> + </item> + <item> + <p> + The guard BIF <c>is_boolean/1</c> (introduced in R10B) + has now been included in the lists of BIFs allowed in + guards in the Reference Manual.</p> + <p> + Own Id: OTP-8477</p> + </item> + <item> + <p> + Added function <c>zip:foldl/3</c> to iterate over zip + archives.</p> + <p> + Added functions to create and extract escripts. See + <c>escript:create/2</c> and <c>escript:extract/2</c>.</p> + <p> + The undocumented function <c>escript:foldl/3</c> has been + removed. The same functionality can be achieved with the + more flexible functions <c>escript:extract/2</c> and + <c>zip:foldl/3</c>.</p> + <p> + Record fields has been annotated with type info. Source + files as been adapted to fit within 80 chars and trailing + whitespace has been removed.</p> + <p> + Own Id: OTP-8521</p> + </item> + <item> + <p>A regular expression with many levels of parenthesis + could cause a buffer overflow. That has been corrected. + (Thanks to Michael Santos.)</p> + <p> + Own Id: OTP-8539</p> + </item> + <item> + <p> + <c>erlang:decode_packet(httph_bin,..)</c> could return + corrupt header strings or even crash the VM. This has + been fixed. It only happened on 32-bit VM if the header + name was unknown and between 16 and 20 characters long. + Sockets with simular <c>packet</c> option did not suffer + from this bug.</p> + <p> + Own Id: OTP-8548</p> + </item> + <item> + <p>New NIF features:</p> <list><item> Send messages from + a NIF, or from thread created by NIF, to any local + process (<c>enif_send</c>) </item><item> Store terms + between NIF calls (<c>enif_alloc_env</c>, + <c>enif_make_copy</c>) </item><item> Create binary terms + with user defined memory management + (<c>enif_make_resource_binary</c>) </item></list> <p>And + some incompatible changes made to the API. For more + information see the warning text in <seealso + marker="erl_nif">erl_nif(3)</seealso>.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8555</p> + </item> + <item> + <p>If the '<c>fop</c>' program (needed for building PDF + files) cannot not be found, it is now possible to build + the HTML and man pages anyway (there will also be dummy + PDF files with no real content created). (Thanks to + Tuncer Ayaz.)</p> + <p> + Own Id: OTP-8559</p> + </item> + <item> + <p>When defining macros the closing right parenthesis + before the dot is now mandatory.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8562</p> + </item> + <item> + <p>Local and imported functions now override the + auto-imported BIFs when the names clash. The pre R14 + behaviour was that auto-imported BIFs would override + local functions. To avoid that old programs change + behaviour, the following will generate an error:</p> + <list><item><p>Doing a call without explicit module name + to a local function having a name clashing with the name + of an auto-imported BIF that was present (and + auto-imported) before OTP R14A</p></item> + <item><p>Explicitly importing a function having a name + clashing with the name of an autoimported BIF that was + present (and autoimported) before OTP R14A</p></item> + <item><p>Using any form of the old compiler directive + <c>nowarn_bif_clash</c></p></item> </list> <p>If the BIF + was added or auto-imported in OTP R14A or later, + overriding it with an import or a local function will + only result in a warning,</p> <p>To resolve clashes, you + can either use the explicit module name <c>erlang</c> to + call the BIF, or you can remove the auto-import of that + specific BIF by using the new compiler directive + <c>-compile({no_auto_import,[F/A]}).</c>, which makes all + calls to the local or imported function without explicit + module name pass without warnings or errors.</p> <p>The + change makes it possible to add auto-imported BIFs + without breaking or silently changing old code in the + future. However some current code ingeniously utilizing + the old behaviour or the <c>nowarn_bif_clash</c> compiler + directive, might need changing to be accepted by the + compiler.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8579</p> + </item> + <item> + <p> + A bug in re that could cause certain regular expression + matches never to terminate is corrected. (Thanks to + Michael Santos and Gordon Guthrie.)</p> + <p> + Own Id: OTP-8589</p> + </item> + <item> + <p>The erlang:open_port spawn and spawn_executable + directives can include an <c>{env,Env}</c> directive to + set up environment variables for the spawned process. A + bug prevented applications from using <c>{env,Env}</c> to + set an environment variable whose value ended with a + '<c>=</c>' (equal sign) character; the trailing equal + sign was mistaken as an indication that an environment + variable was to be cleared from the environment of the + spawned process. (Thanks to Steve Vinoski.)</p> + <p> + Own Id: OTP-8614</p> + </item> + <item> + <p><c>receive</c> statements that can only read out a + newly created reference are now specially optimized so + that it will execute in constant time regardless of the + number of messages in the receive queue for the process. + That optimization will benefit calls to + <c>gen_server:call()</c>. (See <c>gen:do_call/4</c> for + an example of a receive statement that will be + optimized.)</p> + <p> + Own Id: OTP-8623</p> + </item> + <item> + <p> + The functions file:advise/4 and file:datasync/1 have been + added. (Thanks to Filipe David Manana.)</p> + <p> + Own Id: OTP-8637</p> + </item> + <item> + <p> + New NIF API functions: <c>enif_make_atom_len</c>, + <c>enif_make_existing_atom_len</c>, + <c>enif_make_string_len</c>, <c>enif_get_atom_length</c>, + <c>enif_get_list_length</c>, <c>enif_is_list</c>, + <c>enif_is_tuple</c> (by Tuncer Ayaz)</p> + <p> + Own Id: OTP-8640</p> + </item> + <item> + <p> + Support for using gcc's built-in functions for atomic + memory access has been added. This functionallity will be + used if available and no other native atomic + implementation in ERTS is available.</p> + <p> + Own Id: OTP-8659</p> + </item> + <item> + <p> + The number of spinlocks used when implementing atomic + fall-backs when no native atomic implementation is + available has been increased from 16 to 1024.</p> + <p> + Own Id: OTP-8660</p> + </item> + <item> + <p> + Writer preferred pthread read/write locks has been + enabled on Linux.</p> + <p> + Own Id: OTP-8661</p> + </item> + <item> + <p> + The runtime system will by default bind schedulers to + logical processors using the <c>default_bind</c> bind + type if the amount of schedulers are at least equal to + the amount of logical processors configured, binding of + schedulers is supported, and a CPU topology is available + at startup. </p> + <p> + <em>NOTE:</em> If the Erlang runtime system is the only + operating system process that binds threads to logical + processors, this improves the performance of the runtime + system. However, if other operating system processes (as + for example another Erlang runtime system) also bind + threads to logical processors, there might be a + performance penalty instead. If this is the case you, are + are advised to unbind the schedulers using the <seealso + marker="erl#+sbt">+sbtu</seealso> command line argument, + or by invoking <seealso + marker="erlang#system_flag_scheduler_bind_type">erlang:system_flag(scheduler_bind_type, + unbound)</seealso>.</p> + <p> + Own Id: OTP-8666</p> + </item> + <item> + <p> + The recently added BIFs erlang:min/2, erlang:max/2 and + erlang:port_command/3 are now auto-imported (as they were + originally intended to be). Due to the recent compiler + change (OTP-8579), the only impact on old code defining + it's own min/2, max/2 or port_command/3 functions will be + a warning, the local functions will still be used. The + warning can be removed by using + -compile({no_auto_import,[min/2,max/2,port_command/3]}). + in the source file.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8669 Aux Id: OTP-8579 </p> + </item> + <item> + <p> + There is a new option 'exclusive' to file:open/2 that + uses the OS O_EXCL flag where supported to open the file + in exclusive mode.</p> + <p> + Own Id: OTP-8670</p> + </item> + <item> + <p> + Now, binary_to_term/2 is auto-imported. This will cause a + compile warning if and only if a module has got a local + function with that name.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8671</p> + </item> + <item> + <p> + Alignment of scheduler data and run queues were adjusted.</p> + <p> + Own Id: OTP-8673</p> + </item> + <item> + <p>Call time breakpoint tracing</p> <list><item>Introduce + a <c>call_time</c> option to + <c>erlang:trace_pattern/3</c>.This option enables call + time breakpoint tracing on code that is executed by + processes with call tracing enabled. Call time tracing + stores the number of calls and the time spent of each + function with this trace pattern enabled. The information + can be retrieved with <c>erlang:trace_info/2</c></item> + <item>Add a scheduler array for BpData. To solve the + issue of multiple schedulers constantly updating the head + pointer to the bp data wheel, each scheduler now has its + own entrypoint to the wheel. This head pointer can be + updated without a locking being taken.</item></list> + <p>Teach call count tracing to use atomics</p> + <list><item>Call count previously used a global lock for + accessing and writing its counter in the breakpoint. This + is now changed to atomics instead.</item> <item>The + change will let call count tracing and cprof to scale + better when increasing the number of + schedulers.</item></list> + <p> + Own Id: OTP-8677</p> + </item> + <item> + <p><c>eprof</c> has been reimplemented with support in + the Erlang virtual machine and is now both faster (i.e. + slows down the code being measured less) and scales much + better. In measurements we saw speed-ups compared to the + old eprof ranging from 6 times (for sequential code that + only uses one scheduler/core) up to 84 times (for + parallel code that uses 8 cores).</p> + <p>Note: The API for the <c>eprof</c> has been cleaned up + and extended. See the documentation.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8706</p> + </item> + </list> + </section> + +</section> + +<section><title>Erts 5.7.5.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Driver threads, such as async threads, using <seealso + marker="erl_driver#ErlDrvPDL">port data locks</seealso> + peeked at the port status field without proper locking + when looking up the driver queue.</p> + <p> + Own Id: OTP-8475</p> + </item> + <item> + <p> + A call to the BIF <c>unregister(RegName)</c> when a port + had the name <c>RegName</c> registered in the runtime + system without SMP support caused a runtime system crash. + (Thanks to Per Hedeland for the bugfix and test case.)</p> + <p> + Own Id: OTP-8487</p> + </item> + <item> + <p> + Fix memory management bug causing crash of non-SMP + emulator with async threads enabled. The bug did first + appear in R13B03.</p> + <p> + Own Id: OTP-8591 Aux Id: seq11554 </p> + </item> + </list> + </section> + +</section> + <section><title>Erts 5.7.5</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/erts/emulator/beam/beam_bif_load.c b/erts/emulator/beam/beam_bif_load.c index 596ad9a010..4fc271d41c 100644 --- a/erts/emulator/beam/beam_bif_load.c +++ b/erts/emulator/beam/beam_bif_load.c @@ -402,7 +402,7 @@ check_process_code(Process* rp, Module* modp) BeamInstr* end; Eterm* sp; #ifndef HYBRID /* FIND ME! */ - ErlFunThing* funp; + struct erl_off_heap_header* oh; int done_gc = 0; #endif @@ -470,27 +470,30 @@ check_process_code(Process* rp, Module* modp) #ifndef HYBRID /* FIND ME! */ rescan: - for (funp = MSO(rp).funs; funp; funp = funp->next) { - BeamInstr* fun_code; - - fun_code = funp->fe->address; - - if (INSIDE((BeamInstr *) funp->fe->address)) { - if (done_gc) { - return am_true; - } else { - /* - * Try to get rid of this fun by garbage collecting. - * Clear both fvalue and ftrace to make sure they - * don't hold any funs. - */ - rp->freason = EXC_NULL; - rp->fvalue = NIL; - rp->ftrace = NIL; - done_gc = 1; - FLAGS(rp) |= F_NEED_FULLSWEEP; - (void) erts_garbage_collect(rp, 0, rp->arg_reg, rp->arity); - goto rescan; + for (oh = MSO(rp).first; oh; oh = oh->next) { + if (thing_subtag(oh->thing_word) == FUN_SUBTAG) { + ErlFunThing* funp = (ErlFunThing*) oh; + BeamInstr* fun_code; + + fun_code = funp->fe->address; + + if (INSIDE((BeamInstr *) funp->fe->address)) { + if (done_gc) { + return am_true; + } else { + /* + * Try to get rid of this fun by garbage collecting. + * Clear both fvalue and ftrace to make sure they + * don't hold any funs. + */ + rp->freason = EXC_NULL; + rp->fvalue = NIL; + rp->ftrace = NIL; + done_gc = 1; + FLAGS(rp) |= F_NEED_FULLSWEEP; + (void) erts_garbage_collect(rp, 0, rp->arg_reg, rp->arity); + goto rescan; + } } } } diff --git a/erts/emulator/beam/beam_bp.h b/erts/emulator/beam/beam_bp.h index b5d5b3c203..ebc171078d 100644 --- a/erts/emulator/beam/beam_bp.h +++ b/erts/emulator/beam/beam_bp.h @@ -123,7 +123,7 @@ typedef struct { Uint ms; Uint s; Uint us; - Uint *pc; + BeamInstr *pc; } process_breakpoint_time_t; /* used within psd */ extern erts_smp_spinlock_t erts_bp_lock; diff --git a/erts/emulator/beam/beam_emu.c b/erts/emulator/beam/beam_emu.c index c0680086aa..260f349563 100644 --- a/erts/emulator/beam/beam_emu.c +++ b/erts/emulator/beam/beam_emu.c @@ -3385,8 +3385,8 @@ apply_bif_or_nif_epilogue: HTOP += PROC_BIN_SIZE; pb->thing_word = HEADER_PROC_BIN; pb->size = num_bytes; - pb->next = MSO(c_p).mso; - MSO(c_p).mso = pb; + pb->next = MSO(c_p).first; + MSO(c_p).first = (struct erl_off_heap_header*) pb; pb->val = bptr; pb->bytes = (byte*) bptr->orig_bytes; pb->flags = 0; @@ -3486,8 +3486,8 @@ apply_bif_or_nif_epilogue: HTOP += PROC_BIN_SIZE; pb->thing_word = HEADER_PROC_BIN; pb->size = tmp_arg1; - pb->next = MSO(c_p).mso; - MSO(c_p).mso = pb; + pb->next = MSO(c_p).first; + MSO(c_p).first = (struct erl_off_heap_header*) pb; pb->val = bptr; pb->bytes = (byte*) bptr->orig_bytes; pb->flags = 0; @@ -4463,7 +4463,7 @@ apply_bif_or_nif_epilogue: E -= 2; E[0] = make_cp(I); E[1] = make_cp(c_p->cp); /* original return address */ - c_p->cp = (BeamInstr *) make_cp(beam_return_time_trace); + c_p->cp = beam_return_time_trace; } } @@ -4493,20 +4493,20 @@ apply_bif_or_nif_epilogue: BeamInstr real_I; Uint32 flags; Eterm tracer_pid; - BeamInstr *cpp; + Uint* cpp; int return_to_trace = 0, need = 0; flags = 0; SWAPOUT; reg[0] = r(0); if (*(c_p->cp) == (BeamInstr) OpCode(return_trace)) { - cpp = (BeamInstr*)&E[2]; + cpp = &E[2]; } else if (*(c_p->cp) == (BeamInstr) OpCode(i_return_to_trace)) { return_to_trace = !0; - cpp = (BeamInstr*)&E[0]; + cpp = &E[0]; } else if (*(c_p->cp) == (BeamInstr) OpCode(i_return_time_trace)) { return_to_trace = !0; - cpp = (BeamInstr*)&E[0]; + cpp = &E[0]; } else { cpp = NULL; } @@ -6323,8 +6323,8 @@ new_fun(Process* p, Eterm* reg, ErlFunEntry* fe, int num_free) erts_refc_inc(&fe->refc, 2); funp->thing_word = HEADER_FUN; #ifndef HYBRID /* FIND ME! */ - funp->next = MSO(p).funs; - MSO(p).funs = funp; + funp->next = MSO(p).first; + MSO(p).first = (struct erl_off_heap_header*) funp; #endif funp->fe = fe; funp->num_free = num_free; diff --git a/erts/emulator/beam/bif.c b/erts/emulator/beam/bif.c index 506bf383ca..6e9755ad48 100644 --- a/erts/emulator/beam/bif.c +++ b/erts/emulator/beam/bif.c @@ -3611,11 +3611,11 @@ BIF_RETTYPE list_to_pid_1(BIF_ALIST_1) etp = (ExternalThing *) HAlloc(BIF_P, EXTERNAL_THING_HEAD_SIZE + 1); etp->header = make_external_pid_header(1); - etp->next = MSO(BIF_P).externals; + etp->next = MSO(BIF_P).first; etp->node = enp; etp->data.ui[0] = make_pid_data(c, b); - MSO(BIF_P).externals = etp; + MSO(BIF_P).first = (struct erl_off_heap_header*) etp; erts_deref_dist_entry(dep); BIF_RET(make_external_pid(etp)); } diff --git a/erts/emulator/beam/big.c b/erts/emulator/beam/big.c index 90d3a0304a..2d250f32cf 100644 --- a/erts/emulator/beam/big.c +++ b/erts/emulator/beam/big.c @@ -1509,14 +1509,14 @@ Eterm erts_uint64_to_big(Uint64 x, Eterm **hpp) *hp = make_pos_bignum_header(2); BIG_DIGIT(hp, 0) = (Uint) (x & ((Uint) 0xffffffff)); BIG_DIGIT(hp, 1) = (Uint) ((x >> 32) & ((Uint) 0xffffffff)); - *hpp += 2; + *hpp += 3; } else #endif { *hp = make_pos_bignum_header(1); BIG_DIGIT(hp, 0) = (Uint) x; - *hpp += 1; + *hpp += 2; } return make_big(hp); } @@ -1539,7 +1539,7 @@ Eterm erts_sint64_to_big(Sint64 x, Eterm **hpp) *hp = make_pos_bignum_header(2); BIG_DIGIT(hp, 0) = (Uint) (x & ((Uint) 0xffffffff)); BIG_DIGIT(hp, 1) = (Uint) ((x >> 32) & ((Uint) 0xffffffff)); - *hpp += 2; + *hpp += 3; } else #endif @@ -1549,7 +1549,7 @@ Eterm erts_sint64_to_big(Sint64 x, Eterm **hpp) else *hp = make_pos_bignum_header(1); BIG_DIGIT(hp, 0) = (Uint) x; - *hpp += 1; + *hpp += 2; } return make_big(hp); } diff --git a/erts/emulator/beam/binary.c b/erts/emulator/beam/binary.c index c68392fad4..3fd714f9c2 100644 --- a/erts/emulator/beam/binary.c +++ b/erts/emulator/beam/binary.c @@ -88,8 +88,8 @@ new_binary(Process *p, byte *buf, int len) pb = (ProcBin *) HAlloc(p, PROC_BIN_SIZE); pb->thing_word = HEADER_PROC_BIN; pb->size = len; - pb->next = MSO(p).mso; - MSO(p).mso = pb; + pb->next = MSO(p).first; + MSO(p).first = (struct erl_off_heap_header*)pb; pb->val = bptr; pb->bytes = (byte*) bptr->orig_bytes; pb->flags = 0; @@ -127,8 +127,8 @@ Eterm erts_new_mso_binary(Process *p, byte *buf, int len) pb = (ProcBin *) HAlloc(p, PROC_BIN_SIZE); pb->thing_word = HEADER_PROC_BIN; pb->size = len; - pb->next = MSO(p).mso; - MSO(p).mso = pb; + pb->next = MSO(p).first; + MSO(p).first = (struct erl_off_heap_header*)pb; pb->val = bptr; pb->bytes = (byte*) bptr->orig_bytes; pb->flags = 0; @@ -487,16 +487,6 @@ BIF_RETTYPE split_binary_2(BIF_ALIST_2) BIF_ERROR(BIF_P, BADARG); } -void -erts_cleanup_mso(ProcBin* pb) -{ - while (pb != NULL) { - ProcBin* next = pb->next; - if (erts_refc_dectest(&pb->val->refc, 0) == 0) - erts_bin_free(pb->val); - pb = next; - } -} /* * Local functions. diff --git a/erts/emulator/beam/break.c b/erts/emulator/beam/break.c index 857cb177c8..f339e19761 100644 --- a/erts/emulator/beam/break.c +++ b/erts/emulator/beam/break.c @@ -611,29 +611,29 @@ static void bin_check(void) { Process *rp; - ProcBin *bp; - int i, printed; + struct erl_off_heap_header* hdr; + int i, printed = 0; for (i=0; i < erts_max_processes; i++) { if ((rp = process_tab[i]) == NULL) continue; - if (!(bp = rp->off_heap.mso)) - continue; - printed = 0; - while (bp) { - if (printed == 0) { - erts_printf("Process %T holding binary data \n", rp->id); - printed = 1; + for (hdr = rp->off_heap.first; hdr; hdr = hdr->next) { + if (hdr->thing_word == HEADER_PROC_BIN) { + ProcBin *bp = (ProcBin*) hdr; + if (!printed) { + erts_printf("Process %T holding binary data \n", rp->id); + printed = 1; + } + erts_printf("0x%08lx orig_size: %ld, norefs = %ld\n", + (unsigned long)bp->val, + (long)bp->val->orig_size, + erts_smp_atomic_read(&bp->val->refc)); } - erts_printf("0x%08lx orig_size: %ld, norefs = %ld\n", - (unsigned long)bp->val, - (long)bp->val->orig_size, - erts_smp_atomic_read(&bp->val->refc)); - - bp = bp->next; } - if (printed == 1) + if (printed) { erts_printf("--------------------------------------\n"); + printed = 0; + } } /* db_bin_check() has to be rewritten for the AVL trees... */ /*db_bin_check();*/ diff --git a/erts/emulator/beam/copy.c b/erts/emulator/beam/copy.c index 521a1b1788..57be0169ba 100644 --- a/erts/emulator/beam/copy.c +++ b/erts/emulator/beam/copy.c @@ -314,9 +314,9 @@ copy_struct(Eterm obj, Uint sz, Eterm** hpp, ErlOffHeap* off_heap) *argp = make_binary(hbot); pb = (ProcBin*) hbot; erts_refc_inc(&pb->val->refc, 2); - pb->next = off_heap->mso; + pb->next = off_heap->first; pb->flags = 0; - off_heap->mso = pb; + off_heap->first = (struct erl_off_heap_header*) pb; off_heap->overhead += pb->size / sizeof(Eterm); } break; @@ -363,9 +363,9 @@ copy_struct(Eterm obj, Uint sz, Eterm** hpp, ErlOffHeap* off_heap) to->val = from->val; erts_refc_inc(&to->val->refc, 2); to->bytes = from->bytes + offset; - to->next = off_heap->mso; + to->next = off_heap->first; to->flags = 0; - off_heap->mso = to; + off_heap->first = (struct erl_off_heap_header*) to; off_heap->overhead += to->size / sizeof(Eterm); } *argp = make_binary(hbot); @@ -396,8 +396,8 @@ copy_struct(Eterm obj, Uint sz, Eterm** hpp, ErlOffHeap* off_heap) } #ifndef HYBRID /* FIND ME! */ funp = (ErlFunThing *) tp; - funp->next = off_heap->funs; - off_heap->funs = funp; + funp->next = off_heap->first; + off_heap->first = (struct erl_off_heap_header*) funp; erts_refc_inc(&funp->fe->refc, 2); #endif *argp = make_fun(tp); @@ -416,8 +416,8 @@ copy_struct(Eterm obj, Uint sz, Eterm** hpp, ErlOffHeap* off_heap) *htop++ = *objp++; } - etp->next = off_heap->externals; - off_heap->externals = etp; + etp->next = off_heap->first; + off_heap->first = (struct erl_off_heap_header*)etp; erts_refc_inc(&etp->node->refc, 2); *argp = make_external(tp); @@ -650,8 +650,8 @@ Eterm copy_struct_lazy(Process *from, Eterm orig, Uint offs) *hp++ = *objp++; } erts_refc_inc(&pb->val->refc, 2); - pb->next = erts_global_offheap.mso; - erts_global_offheap.mso = pb; + pb->next = erts_global_offheap.first; + erts_global_offheap.first = pb; erts_global_offheap.overhead += pb->size / sizeof(Eterm); continue; } @@ -672,9 +672,9 @@ Eterm copy_struct_lazy(Process *from, Eterm orig, Uint offs) while (i--) { *hp++ = *objp++; } -#ifndef HYBRID // FIND ME! - funp->next = erts_global_offheap.funs; - erts_global_offheap.funs = funp; +#ifndef HYBRID /* FIND ME! */ + funp->next = erts_global_offheap.first; + erts_global_offheap.first = funp; erts_refc_inc(&funp->fe->refc, 2); #endif for (i = k; i < j; i++) { @@ -718,8 +718,8 @@ Eterm copy_struct_lazy(Process *from, Eterm orig, Uint offs) *hp++ = *objp++; } - etp->next = erts_global_offheap.externals; - erts_global_offheap.externals = etp; + etp->next = erts_global_offheap.first; + erts_global_offheap.first = etp; erts_refc_inc(&etp->node->refc, 2); continue; } @@ -775,8 +775,8 @@ Eterm copy_struct_lazy(Process *from, Eterm orig, Uint offs) to_bin->size = real_size; to_bin->val = from_bin->val; to_bin->bytes = from_bin->bytes + sub_offset; - to_bin->next = erts_global_offheap.mso; - erts_global_offheap.mso = to_bin; + to_bin->next = erts_global_offheap.first; + erts_global_offheap.first = to_bin; erts_global_offheap.overhead += to_bin->size / sizeof(Eterm); res_binary=make_binary(to_bin); hp += PROC_BIN_SIZE; @@ -910,57 +910,43 @@ copy_shallow(Eterm* ptr, Uint sz, Eterm** hpp, ErlOffHeap* off_heap) break; case REFC_BINARY_SUBTAG: { - ProcBin* pb = (ProcBin *) (hp-1); - int tari = thing_arityval(val); - - sz -= tari; - while (tari--) { - *hp++ = *tp++; - } + ProcBin* pb = (ProcBin *) (tp-1); erts_refc_inc(&pb->val->refc, 2); - pb->next = off_heap->mso; - off_heap->mso = pb; off_heap->overhead += pb->size / sizeof(Eterm); } - break; + goto off_heap_common; + case FUN_SUBTAG: { -#ifndef HYBRID /* FIND ME! */ - ErlFunThing* funp = (ErlFunThing *) (hp-1); -#endif - int tari = thing_arityval(val); - - sz -= tari; - while (tari--) { - *hp++ = *tp++; - } -#ifndef HYBRID /* FIND ME! */ - funp->next = off_heap->funs; - off_heap->funs = funp; + ErlFunThing* funp = (ErlFunThing *) (tp-1); erts_refc_inc(&funp->fe->refc, 2); -#endif } - break; + goto off_heap_common; + case EXTERNAL_PID_SUBTAG: case EXTERNAL_PORT_SUBTAG: case EXTERNAL_REF_SUBTAG: { - ExternalThing* etp = (ExternalThing *) (hp-1); + ExternalThing* etp = (ExternalThing *) (tp-1); + erts_refc_inc(&etp->node->refc, 2); + } + off_heap_common: + { + struct erl_off_heap_header* ohh = (struct erl_off_heap_header*)(hp-1); int tari = thing_arityval(val); - + sz -= tari; while (tari--) { *hp++ = *tp++; } - etp->next = off_heap->externals; - off_heap->externals = etp; - erts_refc_inc(&etp->node->refc, 2); + ohh->next = off_heap->first; + off_heap->first = ohh; } break; default: { int tari = header_arity(val); - + sz -= tari; while (tari--) { *hp++ = *tp++; @@ -1029,12 +1015,6 @@ void move_multi_frags(Eterm** hpp, ErlOffHeap* off_heap, ErlHeapFragment* first, static void move_one_frag(Eterm** hpp, Eterm* src, Uint src_sz, ErlOffHeap* off_heap) { - union { - Uint *up; - ProcBin *pbp; - ErlFunThing *efp; - ExternalThing *etp; - } ohe; Eterm* ptr = src; Eterm* end = ptr + src_sz; Eterm dummy_ref; @@ -1046,23 +1026,17 @@ move_one_frag(Eterm** hpp, Eterm* src, Uint src_sz, ErlOffHeap* off_heap) val = *ptr; ASSERT(val != ERTS_HOLE_MARKER); if (is_header(val)) { + struct erl_off_heap_header* hdr = (struct erl_off_heap_header*)hp; ASSERT(ptr + header_arity(val) < end); - ohe.up = hp; MOVE_BOXED(ptr, val, hp, &dummy_ref); switch (val & _HEADER_SUBTAG_MASK) { case REFC_BINARY_SUBTAG: - ohe.pbp->next = off_heap->mso; - off_heap->mso = ohe.pbp; - break; case FUN_SUBTAG: - ohe.efp->next = off_heap->funs; - off_heap->funs = ohe.efp; - break; case EXTERNAL_PID_SUBTAG: case EXTERNAL_PORT_SUBTAG: case EXTERNAL_REF_SUBTAG: - ohe.etp->next = off_heap->externals; - off_heap->externals = ohe.etp; + hdr->next = off_heap->first; + off_heap->first = hdr; break; } } diff --git a/erts/emulator/beam/dist.c b/erts/emulator/beam/dist.c index d465017949..16b6aeac3f 100644 --- a/erts/emulator/beam/dist.c +++ b/erts/emulator/beam/dist.c @@ -922,12 +922,8 @@ int erts_net_message(Port *prt, UseTmpHeapNoproc(DIST_CTL_DEFAULT_SIZE); /* Thanks to Luke Gorrie */ - off_heap.mso = NULL; -#ifndef HYBRID /* FIND ME! */ - off_heap.funs = NULL; -#endif + off_heap.first = NULL; off_heap.overhead = 0; - off_heap.externals = NULL; ERTS_SMP_CHK_NO_PROC_LOCKS; @@ -1434,16 +1430,8 @@ int erts_net_message(Port *prt, } } - if (off_heap.mso) { - erts_cleanup_mso(off_heap.mso); - } - if (off_heap.externals) { - erts_cleanup_externals(off_heap.externals); - } + erts_cleanup_offheap(&off_heap); #ifndef HYBRID /* FIND ME! */ - if (off_heap.funs) { - erts_cleanup_funs(off_heap.funs); - } if (ctl != ctl_default) { erts_free(ERTS_ALC_T_DCTRL_BUF, (void *) ctl); } @@ -1453,16 +1441,8 @@ int erts_net_message(Port *prt, return 0; data_error: - if (off_heap.mso) { - erts_cleanup_mso(off_heap.mso); - } - if (off_heap.externals) { - erts_cleanup_externals(off_heap.externals); - } + erts_cleanup_offheap(&off_heap); #ifndef HYBRID /* FIND ME! */ - if (off_heap.funs) { - erts_cleanup_funs(off_heap.funs); - } if (ctl != ctl_default) { erts_free(ERTS_ALC_T_DCTRL_BUF, (void *) ctl); } diff --git a/erts/emulator/beam/erl_bif_binary.c b/erts/emulator/beam/erl_bif_binary.c index 82f1e06e8e..1d8fd11b7b 100644 --- a/erts/emulator/beam/erl_bif_binary.c +++ b/erts/emulator/beam/erl_bif_binary.c @@ -2511,8 +2511,8 @@ BIF_RETTYPE binary_copy_trap(BIF_ALIST_2) pb = (ProcBin *) HAlloc(BIF_P, PROC_BIN_SIZE); pb->thing_word = HEADER_PROC_BIN; pb->size = target_size; - pb->next = MSO(BIF_P).mso; - MSO(BIF_P).mso = pb; + pb->next = MSO(BIF_P).first; + MSO(BIF_P).first = (struct erl_off_heap_header*) pb; pb->val = save; pb->bytes = t; pb->flags = 0; diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index 48cda52612..ca1ab44012 100644 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -122,22 +122,26 @@ static char erts_system_version[] = ("Erlang " ERLANG_OTP_RELEASE #endif static Eterm -bld_bin_list(Uint **hpp, Uint *szp, ProcBin* pb) +bld_bin_list(Uint **hpp, Uint *szp, ErlOffHeap* oh) { + struct erl_off_heap_header* ohh; Eterm res = NIL; Eterm tuple; - for (; pb; pb = pb->next) { - Eterm val = erts_bld_uword(hpp, szp, (UWord) pb->val); - Eterm orig_size = erts_bld_uint(hpp, szp, pb->val->orig_size); - - if (szp) - *szp += 4+2; - if (hpp) { - Uint refc = (Uint) erts_smp_atomic_read(&pb->val->refc); - tuple = TUPLE3(*hpp, val, orig_size, make_small(refc)); - res = CONS(*hpp + 4, tuple, res); - *hpp += 4+2; + for (ohh = oh->first; ohh; ohh = ohh->next) { + if (ohh->thing_word == HEADER_PROC_BIN) { + ProcBin* pb = (ProcBin*) ohh; + Eterm val = erts_bld_uword(hpp, szp, (UWord) pb->val); + Eterm orig_size = erts_bld_uint(hpp, szp, pb->val->orig_size); + + if (szp) + *szp += 4+2; + if (hpp) { + Uint refc = (Uint) erts_smp_atomic_read(&pb->val->refc); + tuple = TUPLE3(*hpp, val, orig_size, make_small(refc)); + res = CONS(*hpp + 4, tuple, res); + *hpp += 4+2; + } } } return res; @@ -176,10 +180,10 @@ static void do_make_one_mon_element(ErtsMonitor *mon, void * vpmlc) Eterm tup; Eterm r = (IS_CONST(mon->ref) ? mon->ref - : STORE_NC(&(pmlc->hp), &MSO(pmlc->p).externals, mon->ref)); + : STORE_NC(&(pmlc->hp), &MSO(pmlc->p), mon->ref)); Eterm p = (IS_CONST(mon->pid) ? mon->pid - : STORE_NC(&(pmlc->hp), &MSO(pmlc->p).externals, mon->pid)); + : STORE_NC(&(pmlc->hp), &MSO(pmlc->p), mon->pid)); tup = TUPLE5(pmlc->hp, pmlc->tag, make_small(mon->type), r, p, mon->name); pmlc->hp += 6; pmlc->res = CONS(pmlc->hp, tup, pmlc->res); @@ -240,7 +244,7 @@ static void do_make_one_lnk_element(ErtsLink *lnk, void * vpllc) Eterm old_res, targets = NIL; Eterm p = (IS_CONST(lnk->pid) ? lnk->pid - : STORE_NC(&(pllc->hp), &MSO(pllc->p).externals, lnk->pid)); + : STORE_NC(&(pllc->hp), &MSO(pllc->p), lnk->pid)); if (lnk->type == LINK_NODE) { targets = make_small(ERTS_LINK_REFC(lnk)); } else if (ERTS_LINK_ROOT(lnk) != NULL) { @@ -1225,7 +1229,7 @@ process_info_aux(Process *BIF_P, hp = HAlloc(BIF_P, 3 + mic.sz); res = NIL; for (i = 0; i < mic.mi_i; i++) { - item = STORE_NC(&hp, &MSO(BIF_P).externals, mic.mi[i].entity); + item = STORE_NC(&hp, &MSO(BIF_P), mic.mi[i].entity); res = CONS(hp, item, res); hp += 2; } @@ -1258,9 +1262,7 @@ process_info_aux(Process *BIF_P, else { /* Monitor by pid. Build {process, Pid} and cons it. */ Eterm t; - Eterm pid = STORE_NC(&hp, - &MSO(BIF_P).externals, - mic.mi[i].entity); + Eterm pid = STORE_NC(&hp, &MSO(BIF_P), mic.mi[i].entity); t = TUPLE2(hp, am_process, pid); hp += 3; res = CONS(hp, t, res); @@ -1282,7 +1284,7 @@ process_info_aux(Process *BIF_P, res = NIL; for (i = 0; i < mic.mi_i; ++i) { - item = STORE_NC(&hp, &MSO(BIF_P).externals, mic.mi[i].entity); + item = STORE_NC(&hp, &MSO(BIF_P), mic.mi[i].entity); res = CONS(hp, item, res); hp += 2; } @@ -1491,7 +1493,7 @@ process_info_aux(Process *BIF_P, case am_group_leader: { int sz = NC_HEAP_SIZE(rp->group_leader); hp = HAlloc(BIF_P, 3 + sz); - res = STORE_NC(&hp, &MSO(BIF_P).externals, rp->group_leader); + res = STORE_NC(&hp, &MSO(BIF_P), rp->group_leader); break; } @@ -1516,9 +1518,9 @@ process_info_aux(Process *BIF_P, case am_binary: { Uint sz = 3; - (void) bld_bin_list(NULL, &sz, MSO(rp).mso); + (void) bld_bin_list(NULL, &sz, &MSO(rp)); hp = HAlloc(BIF_P, sz); - res = bld_bin_list(&hp, NULL, MSO(rp).mso); + res = bld_bin_list(&hp, NULL, &MSO(rp)); break; } @@ -2678,7 +2680,7 @@ BIF_RETTYPE port_info_2(BIF_ALIST_2) hp = HAlloc(BIF_P, 3 + mic.sz); res = NIL; for (i = 0; i < mic.mi_i; i++) { - item = STORE_NC(&hp, &MSO(BIF_P).externals, mic.mi[i].entity); + item = STORE_NC(&hp, &MSO(BIF_P), mic.mi[i].entity); res = CONS(hp, item, res); hp += 2; } @@ -2698,7 +2700,7 @@ BIF_RETTYPE port_info_2(BIF_ALIST_2) res = NIL; for (i = 0; i < mic.mi_i; i++) { Eterm t; - item = STORE_NC(&hp, &MSO(BIF_P).externals, mic.mi[i].entity); + item = STORE_NC(&hp, &MSO(BIF_P), mic.mi[i].entity); t = TUPLE2(hp, am_process, item); hp += 3; res = CONS(hp, t, res); diff --git a/erts/emulator/beam/erl_bif_trace.c b/erts/emulator/beam/erl_bif_trace.c index 443cac9033..0509e51a6f 100644 --- a/erts/emulator/beam/erl_bif_trace.c +++ b/erts/emulator/beam/erl_bif_trace.c @@ -2161,7 +2161,7 @@ trace_delivered_1(BIF_ALIST_1) #ifdef ERTS_SMP bp = new_message_buffer(REF_THING_SIZE + 4); hp = &bp->mem[0]; - msg_ref = STORE_NC(&hp, &bp->off_heap.externals, ref); + msg_ref = STORE_NC(&hp, &bp->off_heap, ref); #else hp = HAlloc(BIF_P, 4); msg_ref = ref; diff --git a/erts/emulator/beam/erl_bits.c b/erts/emulator/beam/erl_bits.c index defe18c92b..96faa673f6 100644 --- a/erts/emulator/beam/erl_bits.c +++ b/erts/emulator/beam/erl_bits.c @@ -1335,8 +1335,8 @@ erts_bs_append(Process* c_p, Eterm* reg, Uint live, Eterm build_size_term, hp += PROC_BIN_SIZE; pb->thing_word = HEADER_PROC_BIN; pb->size = used_size_in_bytes; - pb->next = MSO(c_p).mso; - MSO(c_p).mso = pb; + pb->next = MSO(c_p).first; + MSO(c_p).first = (struct erl_off_heap_header*)pb; pb->val = bptr; pb->bytes = (byte*) bptr->orig_bytes; pb->flags = PB_IS_WRITABLE | PB_ACTIVE_WRITER; @@ -1506,8 +1506,8 @@ erts_bs_init_writable(Process* p, Eterm sz) hp += PROC_BIN_SIZE; pb->thing_word = HEADER_PROC_BIN; pb->size = 0; - pb->next = MSO(p).mso; - MSO(p).mso = pb; + pb->next = MSO(p).first; + MSO(p).first = (struct erl_off_heap_header*) pb; pb->val = bptr; pb->bytes = (byte*) bptr->orig_bytes; pb->flags = PB_IS_WRITABLE | PB_ACTIVE_WRITER; diff --git a/erts/emulator/beam/erl_db_hash.c b/erts/emulator/beam/erl_db_hash.c index 124129a371..96008ccef0 100644 --- a/erts/emulator/beam/erl_db_hash.c +++ b/erts/emulator/beam/erl_db_hash.c @@ -2751,11 +2751,7 @@ static void db_finalize_dbterm_hash(DbUpdateHandle* handle) newDbTerm = &newp->dbterm; newDbTerm->size = handle->new_size; - newDbTerm->off_heap.mso = NULL; - newDbTerm->off_heap.externals = NULL; - #ifndef HYBRID /* FIND ME! */ - newDbTerm->off_heap.funs = NULL; - #endif + newDbTerm->off_heap.first = NULL; newDbTerm->off_heap.overhead = 0; /* make a flat copy */ diff --git a/erts/emulator/beam/erl_db_tree.c b/erts/emulator/beam/erl_db_tree.c index b6b3cabafe..da2696163a 100644 --- a/erts/emulator/beam/erl_db_tree.c +++ b/erts/emulator/beam/erl_db_tree.c @@ -1081,11 +1081,12 @@ static int db_select_continue_tree(Process *p, static int db_select_tree(Process *p, DbTable *tbl, Eterm pattern, int reverse, Eterm *ret) { + /* Strategy: Traverse backwards to build resulting list from tail to head */ DbTableTree *tb = &tbl->tree; DbTreeStack* stack; struct select_context sc; struct mp_info mpi; - Eterm lastkey = NIL; + Eterm lastkey = THE_NON_VALUE; Eterm key; Eterm continuation; unsigned sz; @@ -1293,7 +1294,7 @@ static int db_select_count_tree(Process *p, DbTable *tbl, DbTreeStack* stack; struct select_count_context sc; struct mp_info mpi; - Eterm lastkey = NIL; + Eterm lastkey = THE_NON_VALUE; Eterm key; Eterm continuation; unsigned sz; @@ -1395,7 +1396,7 @@ static int db_select_chunk_tree(Process *p, DbTable *tbl, DbTreeStack* stack; struct select_context sc; struct mp_info mpi; - Eterm lastkey = NIL; + Eterm lastkey = THE_NON_VALUE; Eterm key; Eterm continuation; unsigned sz; @@ -1636,7 +1637,7 @@ static int db_select_delete_tree(Process *p, DbTable *tbl, DbTableTree *tb = &tbl->tree; struct select_delete_context sc; struct mp_info mpi; - Eterm lastkey = NIL; + Eterm lastkey = THE_NON_VALUE; Eterm key; Eterm continuation; unsigned sz; @@ -2588,11 +2589,7 @@ static void db_finalize_dbterm_tree(DbUpdateHandle* handle) newDbTerm = &newp->dbterm; newDbTerm->size = handle->new_size; - newDbTerm->off_heap.mso = NULL; - newDbTerm->off_heap.externals = NULL; - #ifndef HYBRID /* FIND ME! */ - newDbTerm->off_heap.funs = NULL; - #endif + newDbTerm->off_heap.first = NULL; newDbTerm->off_heap.overhead = 0; /* make a flat copy */ @@ -2628,7 +2625,7 @@ static void traverse_backwards(DbTableTree *tb, { TreeDbTerm *this, *next; - if (lastkey == NIL) { + if (lastkey == THE_NON_VALUE) { stack->pos = stack->slot = 0; if (( this = tb->root ) == NULL) { return; @@ -2666,7 +2663,7 @@ static void traverse_forward(DbTableTree *tb, { TreeDbTerm *this, *next; - if (lastkey == NIL) { + if (lastkey == THE_NON_VALUE) { stack->pos = stack->slot = 0; if (( this = tb->root ) == NULL) { return; diff --git a/erts/emulator/beam/erl_db_util.c b/erts/emulator/beam/erl_db_util.c index 93a47eb76f..48e0080525 100644 --- a/erts/emulator/beam/erl_db_util.c +++ b/erts/emulator/beam/erl_db_util.c @@ -363,12 +363,7 @@ static ErtsMatchPseudoProcess *match_pseudo_process; static ERTS_INLINE void cleanup_match_pseudo_process(ErtsMatchPseudoProcess *mpsp, int keep_heap) { - if (mpsp->process.mbuf - || mpsp->process.off_heap.mso -#ifndef HYBRID /* FIND ME! */ - || mpsp->process.off_heap.funs -#endif - || mpsp->process.off_heap.externals) { + if (mpsp->process.mbuf || mpsp->process.off_heap.first) { erts_cleanup_empty_process(&mpsp->process); } #ifdef DEBUG @@ -2495,11 +2490,7 @@ void* db_get_term(DbTableCommon *tb, DbTerm* old, Uint offset, Eterm obj) p = (DbTerm*) ((void *)(((char *) structp) + offset)); } p->size = size; - p->off_heap.mso = NULL; - p->off_heap.externals = NULL; -#ifndef HYBRID /* FIND ME! */ - p->off_heap.funs = NULL; -#endif + p->off_heap.first = NULL; p->off_heap.overhead = 0; top = DBTERM_BUF(p); diff --git a/erts/emulator/beam/erl_fun.c b/erts/emulator/beam/erl_fun.c index 15d9538301..1f74393a96 100644 --- a/erts/emulator/beam/erl_fun.c +++ b/erts/emulator/beam/erl_fun.c @@ -192,20 +192,6 @@ erts_erase_fun_entry(ErlFunEntry* fe) erts_fun_write_unlock(); } -#ifndef HYBRID /* FIND ME! */ -void -erts_cleanup_funs(ErlFunThing* funp) -{ - while (funp) { - ErlFunEntry* fe = funp->fe; - if (erts_refc_dectest(&fe->refc, 0) == 0) { - erts_erase_fun_entry(fe); - } - funp = funp->next; - } -} -#endif - void erts_cleanup_funs_on_purge(BeamInstr* start, BeamInstr* end) { diff --git a/erts/emulator/beam/erl_fun.h b/erts/emulator/beam/erl_fun.h index 944d4b3df5..2f165afa06 100644 --- a/erts/emulator/beam/erl_fun.h +++ b/erts/emulator/beam/erl_fun.h @@ -53,10 +53,10 @@ typedef struct erl_fun_entry { typedef struct erl_fun_thing { Eterm thing_word; /* Subtag FUN_SUBTAG. */ + ErlFunEntry* fe; /* Pointer to fun entry. */ #ifndef HYBRID /* FIND ME! */ - struct erl_fun_thing* next; /* Next fun in mso list. */ + struct erl_off_heap_header* next; #endif - ErlFunEntry* fe; /* Pointer to fun entry. */ #ifdef HIPE UWord* native_address; /* Native code for the fun. */ #endif diff --git a/erts/emulator/beam/erl_gc.c b/erts/emulator/beam/erl_gc.c index a19e090f1e..adc50675bf 100644 --- a/erts/emulator/beam/erl_gc.c +++ b/erts/emulator/beam/erl_gc.c @@ -110,9 +110,7 @@ static Uint adjust_after_fullsweep(Process *p, int size_before, int need, Eterm *objv, int nobj); static void shrink_new_heap(Process *p, Uint new_sz, Eterm *objv, int nobj); static void grow_new_heap(Process *p, Uint new_sz, Eterm* objv, int nobj); -static void sweep_proc_bins(Process *p, int fullsweep); -static void sweep_proc_funs(Process *p, int fullsweep); -static void sweep_proc_externals(Process *p, int fullsweep); +static void sweep_off_heap(Process *p, int fullsweep); static void offset_heap(Eterm* hp, Uint sz, Sint offs, char* area, Uint area_size); static void offset_heap_ptr(Eterm* hp, Uint sz, Sint offs, char* area, Uint area_size); static void offset_rootset(Process *p, Sint offs, char* area, Uint area_size, @@ -145,6 +143,16 @@ erts_init_gc(void) { int i = 0; + ASSERT(offsetof(ProcBin,thing_word) == offsetof(struct erl_off_heap_header,thing_word)); + ASSERT(offsetof(ProcBin,thing_word) == offsetof(ErlFunThing,thing_word)); + ASSERT(offsetof(ProcBin,thing_word) == offsetof(ExternalThing,header)); + ASSERT(offsetof(ProcBin,size) == offsetof(struct erl_off_heap_header,size)); + ASSERT(offsetof(ProcBin,size) == offsetof(ErlSubBin,size)); + ASSERT(offsetof(ProcBin,size) == offsetof(ErlHeapBin,size)); + ASSERT(offsetof(ProcBin,next) == offsetof(struct erl_off_heap_header,next)); + ASSERT(offsetof(ProcBin,next) == offsetof(ErlFunThing,next)); + ASSERT(offsetof(ProcBin,next) == offsetof(ExternalThing,next)); + erts_smp_spinlock_init(&info_lck, "gc_info"); garbage_cols = 0; reclaimed = 0; @@ -286,25 +294,14 @@ erts_offset_heap_ptr(Eterm* hp, Uint sz, Sint offs, offset_heap_ptr(hp, sz, offs, (char *) low, ((char *)high)-((char *)low)); } + #define ptr_within(ptr, low, high) ((ptr) < (high) && (ptr) >= (low)) void erts_offset_off_heap(ErlOffHeap *ohp, Sint offs, Eterm* low, Eterm* high) { - if (ohp->mso && ptr_within((Eterm *)ohp->mso, low, high)) { - Eterm** uptr = (Eterm**) (void *) &ohp->mso; - *uptr += offs; - } - -#ifndef HYBRID /* FIND ME! */ - if (ohp->funs && ptr_within((Eterm *)ohp->funs, low, high)) { - Eterm** uptr = (Eterm**) (void *) &ohp->funs; - *uptr += offs; - } -#endif - - if (ohp->externals && ptr_within((Eterm *)ohp->externals, low, high)) { - Eterm** uptr = (Eterm**) (void *) &ohp->externals; + if (ohp->first && ptr_within((Eterm *)ohp->first, low, high)) { + Eterm** uptr = (Eterm**) (void *) &ohp->first; *uptr += offs; } } @@ -504,14 +501,8 @@ erts_garbage_collect_hibernate(Process* p) cleanup_rootset(&rootset); - if (MSO(p).mso) { - sweep_proc_bins(p, 1); - } - if (MSO(p).funs) { - sweep_proc_funs(p, 1); - } - if (MSO(p).externals) { - sweep_proc_externals(p, 1); + if (MSO(p).first) { + sweep_off_heap(p, 1); } /* @@ -1041,15 +1032,8 @@ do_minor(Process *p, int new_sz, Eterm* objv, int nobj) OLD_HTOP(p) = old_htop; HIGH_WATER(p) = (HEAP_START(p) != HIGH_WATER(p)) ? n_heap : n_htop; - if (MSO(p).mso) { - sweep_proc_bins(p, 0); - } - - if (MSO(p).funs) { - sweep_proc_funs(p, 0); - } - if (MSO(p).externals) { - sweep_proc_externals(p, 0); + if (MSO(p).first) { + sweep_off_heap(p, 0); } #ifdef HARDDEBUG @@ -1271,17 +1255,11 @@ major_collection(Process* p, int need, Eterm* objv, int nobj, Uint *recl) } } - if (MSO(p).mso) { - sweep_proc_bins(p, 1); - } - if (MSO(p).funs) { - sweep_proc_funs(p, 1); - } - if (MSO(p).externals) { - sweep_proc_externals(p, 1); + if (MSO(p).first) { + sweep_off_heap(p, 1); } - if (OLD_HEAP(p) != NULL) { + if (OLD_HEAP(p) != NULL) { ERTS_HEAP_FREE(ERTS_ALC_T_OLD_HEAP, OLD_HEAP(p), (OLD_HEND(p) - OLD_HEAP(p)) * sizeof(Eterm)); @@ -1305,6 +1283,7 @@ major_collection(Process* p, int need, Eterm* objv, int nobj, Uint *recl) HIGH_WATER(p) = HEAP_TOP(p); ErtsGcQuickSanityCheck(p); + /* * Copy newly received message onto the end of the new heap. */ @@ -2044,108 +2023,24 @@ next_vheap_size(Process* p, Uint vheap, Uint vheap_sz) { return vheap_sz < p->min_vheap_size ? p->min_vheap_size : vheap_sz; } -static void -sweep_proc_externals(Process *p, int fullsweep) -{ - ExternalThing** prev; - ExternalThing* ptr; - char* oh = 0; - Uint oh_size = 0; - - if (fullsweep == 0) { - oh = (char *) OLD_HEAP(p); - oh_size = (char *) OLD_HEND(p) - oh; - } - - prev = &MSO(p).externals; - ptr = MSO(p).externals; - - while (ptr) { - Eterm* ppt = (Eterm *) ptr; - - if (IS_MOVED_BOXED(*ppt)) { /* Object is alive */ - ExternalThing* ro = external_thing_ptr(*ppt); - - *prev = ro; /* Patch to moved pos */ - prev = &ro->next; - ptr = ro->next; - } else if (in_area(ppt, oh, oh_size)) { - /* - * Object resides on old heap, and we just did a - * generational collection - keep object in list. - */ - prev = &ptr->next; - ptr = ptr->next; - } else { /* Object has not been moved - deref it */ - erts_deref_node_entry(ptr->node); - *prev = ptr = ptr->next; - } - } - ASSERT(*prev == NULL); -} - -static void -sweep_proc_funs(Process *p, int fullsweep) -{ - ErlFunThing** prev; - ErlFunThing* ptr; - char* oh = 0; - Uint oh_size = 0; - - if (fullsweep == 0) { - oh = (char *) OLD_HEAP(p); - oh_size = (char *) OLD_HEND(p) - oh; - } - - prev = &MSO(p).funs; - ptr = MSO(p).funs; - - while (ptr) { - Eterm* ppt = (Eterm *) ptr; - - if (IS_MOVED_BOXED(*ppt)) { /* Object is alive */ - ErlFunThing* ro = (ErlFunThing *) fun_val(*ppt); - - *prev = ro; /* Patch to moved pos */ - prev = &ro->next; - ptr = ro->next; - } else if (in_area(ppt, oh, oh_size)) { - /* - * Object resides on old heap, and we just did a - * generational collection - keep object in list. - */ - prev = &ptr->next; - ptr = ptr->next; - } else { /* Object has not been moved - deref it */ - ErlFunEntry* fe = ptr->fe; - - *prev = ptr = ptr->next; - if (erts_refc_dectest(&fe->refc, 0) == 0) { - erts_erase_fun_entry(fe); - } - } - } - ASSERT(*prev == NULL); -} - struct shrink_cand_data { - ProcBin* new_candidates; - ProcBin* new_candidates_end; - ProcBin* old_candidates; + struct erl_off_heap_header* new_candidates; + struct erl_off_heap_header* new_candidates_end; + struct erl_off_heap_header* old_candidates; Uint no_of_candidates; Uint no_of_active; }; static ERTS_INLINE void link_live_proc_bin(struct shrink_cand_data *shrink, - ProcBin ***prevppp, - ProcBin **pbpp, + struct erl_off_heap_header*** prevppp, + struct erl_off_heap_header** currpp, int new_heap) { - ProcBin *pbp = *pbpp; - - *pbpp = pbp->next; + ProcBin *pbp = (ProcBin*) *currpp; + ASSERT(**prevppp == *currpp); + *currpp = pbp->next; if (pbp->flags & (PB_ACTIVE_WRITER|PB_IS_WRITABLE)) { ASSERT(((pbp->flags & (PB_ACTIVE_WRITER|PB_IS_WRITABLE)) == (PB_ACTIVE_WRITER|PB_IS_WRITABLE)) @@ -2162,15 +2057,16 @@ link_live_proc_bin(struct shrink_cand_data *shrink, /* Our allocators are 8 byte aligned, i.e., shrinking with less than 8 bytes will have no real effect */ if (unused >= 8) { /* A shrink candidate; save in candidate list */ + **prevppp = pbp->next; if (new_heap) { if (!shrink->new_candidates) - shrink->new_candidates_end = pbp; + shrink->new_candidates_end = (struct erl_off_heap_header*)pbp; pbp->next = shrink->new_candidates; - shrink->new_candidates = pbp; + shrink->new_candidates = (struct erl_off_heap_header*)pbp; } else { pbp->next = shrink->old_candidates; - shrink->old_candidates = pbp; + shrink->old_candidates = (struct erl_off_heap_header*)pbp; } shrink->no_of_candidates++; return; @@ -2178,67 +2074,101 @@ link_live_proc_bin(struct shrink_cand_data *shrink, } } - /* Not a shrink candidate; keep in original mso list */ - **prevppp = pbp; + /* Not a shrink candidate; keep in original mso list */ *prevppp = &pbp->next; - } static void -sweep_proc_bins(Process *p, int fullsweep) +sweep_off_heap(Process *p, int fullsweep) { struct shrink_cand_data shrink = {0}; - ProcBin** prev; - ProcBin* ptr; - Binary* bptr; - char* oh = NULL; - Uint oh_size = 0; + struct erl_off_heap_header* ptr; + struct erl_off_heap_header** prev; + char* oheap = NULL; + Uint oheap_sz = 0; Uint bin_vheap = 0; +#ifdef DEBUG + int seen_mature = 0; +#endif if (fullsweep == 0) { - oh = (char *) OLD_HEAP(p); - oh_size = (char *) OLD_HEND(p) - oh; + oheap = (char *) OLD_HEAP(p); + oheap_sz = (char *) OLD_HEND(p) - oheap; } BIN_OLD_VHEAP(p) = 0; - prev = &MSO(p).mso; - ptr = MSO(p).mso; + prev = &MSO(p).first; + ptr = MSO(p).first; - /* - * Note: In R7 we no longer force a fullsweep when we find binaries - * on the old heap. The reason is that with the introduction of the - * bit syntax we can expect binaries to be used a lot more. Note that - * in earlier releases a brand new binary (or any other term) could - * be put on the old heap during a gen-gc fullsweep, but this is - * no longer the case in R7. + /* Firts part of the list will reside on the (old) new-heap. + * Keep if moved, otherwise deref. */ while (ptr) { - Eterm* ppt = (Eterm *) ptr; - - if (IS_MOVED_BOXED(*ppt)) { /* Object is alive */ - bin_vheap += ptr->size / sizeof(Eterm); - ptr = (ProcBin*) binary_val(*ppt); - link_live_proc_bin(&shrink, - &prev, - &ptr, - !in_area(ptr, oh, oh_size)); - } else if (in_area(ppt, oh, oh_size)) { - /* - * Object resides on old heap, and we just did a - * generational collection - keep object in list. - */ - BIN_OLD_VHEAP(p) += ptr->size / sizeof(Eterm); /* for binary gc (words)*/ - link_live_proc_bin(&shrink, &prev, &ptr, 0); - } else { /* Object has not been moved - deref it */ - - *prev = ptr->next; - bptr = ptr->val; - if (erts_refc_dectest(&bptr->refc, 0) == 0) - erts_bin_free(bptr); - ptr = *prev; - } + if (IS_MOVED_BOXED(ptr->thing_word)) { + ASSERT(!in_area(ptr, oheap, oheap_sz)); + *prev = ptr = (struct erl_off_heap_header*) boxed_val(ptr->thing_word); + ASSERT(!IS_MOVED_BOXED(ptr->thing_word)); + if (ptr->thing_word == HEADER_PROC_BIN) { + int to_new_heap = !in_area(ptr, oheap, oheap_sz); + ASSERT(to_new_heap == !seen_mature || (!to_new_heap && (seen_mature=1))); + if (to_new_heap) { + bin_vheap += ptr->size / sizeof(Eterm); + } else { + BIN_OLD_VHEAP(p) += ptr->size / sizeof(Eterm); /* for binary gc (words)*/ + } + link_live_proc_bin(&shrink, &prev, &ptr, to_new_heap); + } + else { + prev = &ptr->next; + ptr = ptr->next; + } + } + else if (!in_area(ptr, oheap, oheap_sz)) { + /* garbage */ + switch (thing_subtag(ptr->thing_word)) { + case REFC_BINARY_SUBTAG: + { + Binary* bptr = ((ProcBin*)ptr)->val; + if (erts_refc_dectest(&bptr->refc, 0) == 0) { + erts_bin_free(bptr); + } + break; + } + case FUN_SUBTAG: + { + ErlFunEntry* fe = ((ErlFunThing*)ptr)->fe; + if (erts_refc_dectest(&fe->refc, 0) == 0) { + erts_erase_fun_entry(fe); + } + break; + } + default: + ASSERT(is_external_header(ptr->thing_word)); + erts_deref_node_entry(((ExternalThing*)ptr)->node); + } + *prev = ptr = ptr->next; + } + else break; /* and let old-heap loop continue */ + } + + /* The rest of the list resides on old-heap, and we just did a + * generational collection - keep objects in list. + */ + while (ptr) { + ASSERT(in_area(ptr, oheap, oheap_sz)); + ASSERT(!IS_MOVED_BOXED(ptr->thing_word)); + if (ptr->thing_word == HEADER_PROC_BIN) { + BIN_OLD_VHEAP(p) += ptr->size / sizeof(Eterm); /* for binary gc (words)*/ + link_live_proc_bin(&shrink, &prev, &ptr, 0); + } + else { + ASSERT(is_fun_header(ptr->thing_word) || + is_external_header(ptr->thing_word)); + prev = &ptr->next; + ptr = ptr->next; + } } if (BIN_OLD_VHEAP(p) >= BIN_OLD_VHEAP_SZ(p)) { @@ -2254,7 +2184,8 @@ sweep_proc_bins(Process *p, int fullsweep) */ if (shrink.no_of_candidates) { - ProcBin *candlist[] = {shrink.new_candidates, shrink.old_candidates}; + ProcBin *candlist[] = { (ProcBin*)shrink.new_candidates, + (ProcBin*)shrink.old_candidates }; Uint leave_unused = 0; int i; @@ -2266,21 +2197,21 @@ sweep_proc_bins(Process *p, int fullsweep) } for (i = 0; i < sizeof(candlist)/sizeof(candlist[0]); i++) { - - for (ptr = candlist[i]; ptr; ptr = ptr->next) { - Uint new_size = ptr->size; + ProcBin* pb; + for (pb = candlist[i]; pb; pb = (ProcBin*)pb->next) { + Uint new_size = pb->size; if (leave_unused) { new_size += (new_size * 100) / leave_unused; /* Our allocators are 8 byte aligned, i.e., shrinking with less than 8 bytes will have no real effect */ - if (new_size + 8 >= ptr->val->orig_size) + if (new_size + 8 >= pb->val->orig_size) continue; } - ptr->val = erts_bin_realloc(ptr->val, new_size); - ptr->val->orig_size = new_size; - ptr->bytes = (byte *) ptr->val->orig_bytes; + pb->val = erts_bin_realloc(pb->val, new_size); + pb->val->orig_size = new_size; + pb->bytes = (byte *) pb->val->orig_bytes; } } @@ -2289,21 +2220,20 @@ sweep_proc_bins(Process *p, int fullsweep) * We now potentially have the mso list divided into three lists: * - shrink candidates on new heap (inactive writable with unused data) * - shrink candidates on old heap (inactive writable with unused data) - * - other binaries (read only + active writable ...) + * - other binaries (read only + active writable ...) + funs and externals * * Put them back together: new candidates -> other -> old candidates * This order will ensure that the list only refers from new * generation to old and never from old to new *which is important*. */ if (shrink.new_candidates) { - if (prev == &MSO(p).mso) /* empty other binaries list */ + if (prev == &MSO(p).first) /* empty other binaries list */ prev = &shrink.new_candidates_end->next; else - shrink.new_candidates_end->next = MSO(p).mso; - MSO(p).mso = shrink.new_candidates; + shrink.new_candidates_end->next = MSO(p).first; + MSO(p).first = shrink.new_candidates; } } - *prev = shrink.old_candidates; } @@ -2334,15 +2264,17 @@ offset_heap(Eterm* hp, Uint sz, Sint offs, char* area, Uint area_size) tari = thing_arityval(val); switch (thing_subtag(val)) { case REFC_BINARY_SUBTAG: + case FUN_SUBTAG: + case EXTERNAL_PID_SUBTAG: + case EXTERNAL_PORT_SUBTAG: + case EXTERNAL_REF_SUBTAG: { - ProcBin* pb = (ProcBin*) hp; - Eterm** uptr = (Eterm **) (void *) &pb->next; + struct erl_off_heap_header* oh = (struct erl_off_heap_header*) hp; - if (*uptr && in_area((Eterm *)pb->next, area, area_size)) { + if (in_area(oh->next, area, area_size)) { + Eterm** uptr = (Eterm **) (void *) &oh->next; *uptr += offs; /* Patch the mso chain */ } - sz -= tari; - hp += tari + 1; } break; case BIN_MATCHSTATE_SUBTAG: @@ -2353,40 +2285,11 @@ offset_heap(Eterm* hp, Uint sz, Sint offs, char* area, Uint area_size) mb->orig = offset_ptr(mb->orig, offs); mb->base = binary_bytes(mb->orig); } - sz -= tari; - hp += tari + 1; } break; - case FUN_SUBTAG: - { - ErlFunThing* funp = (ErlFunThing *) hp; - Eterm** uptr = (Eterm **) (void *) &funp->next; - - if (*uptr && in_area((Eterm *)funp->next, area, area_size)) { - *uptr += offs; - } - sz -= tari; - hp += tari + 1; - } - break; - case EXTERNAL_PID_SUBTAG: - case EXTERNAL_PORT_SUBTAG: - case EXTERNAL_REF_SUBTAG: - { - ExternalThing* etp = (ExternalThing *) hp; - Eterm** uptr = (Eterm **) (void *) &etp->next; - - if (*uptr && in_area((Eterm *)etp->next, area, area_size)) { - *uptr += offs; - } - sz -= tari; - hp += tari + 1; - } - break; - default: - sz -= tari; - hp += tari + 1; } + sz -= tari; + hp += tari + 1; break; } default: @@ -2423,18 +2326,8 @@ offset_heap_ptr(Eterm* hp, Uint sz, Sint offs, char* area, Uint area_size) static void offset_off_heap(Process* p, Sint offs, char* area, Uint area_size) { - if (MSO(p).mso && in_area((Eterm *)MSO(p).mso, area, area_size)) { - Eterm** uptr = (Eterm**) (void *) &MSO(p).mso; - *uptr += offs; - } - - if (MSO(p).funs && in_area((Eterm *)MSO(p).funs, area, area_size)) { - Eterm** uptr = (Eterm**) (void *) &MSO(p).funs; - *uptr += offs; - } - - if (MSO(p).externals && in_area((Eterm *)MSO(p).externals, area, area_size)) { - Eterm** uptr = (Eterm**) (void *) &MSO(p).externals; + if (MSO(p).first && in_area((Eterm *)MSO(p).first, area, area_size)) { + Eterm** uptr = (Eterm**) (void *) &MSO(p).first; *uptr += offs; } } @@ -2555,8 +2448,8 @@ do { \ __FILE__, __LINE__, #EXP); \ } while (0) -#ifdef ERTS_OFFHEAP_DEBUG_CHK_CIRCULAR_EXTERNAL_LIST -# define ERTS_EXTERNAL_VISITED_BIT ((Eterm) 1 << 31) +#ifdef ERTS_OFFHEAP_DEBUG_CHK_CIRCULAR_LIST +# define ERTS_OFFHEAP_VISITED_BIT ((Eterm) 1 << 31) #endif @@ -2566,62 +2459,45 @@ erts_check_off_heap2(Process *p, Eterm *htop) Eterm *oheap = (Eterm *) OLD_HEAP(p); Eterm *ohtop = (Eterm *) OLD_HTOP(p); int old; - ProcBin *pb; - ErlFunThing *eft; - ExternalThing *et; + union erl_off_heap_ptr u; old = 0; - for (pb = MSO(p).mso; pb; pb = pb->next) { - Eterm *ptr = (Eterm *) pb; - long refc = erts_refc_read(&pb->val->refc, 1); + for (u.hdr = MSO(p).first; u.hdr; u.hdr = u.hdr->next) { + long refc; + switch (thing_subtag(u.hdr->thing_word)) { + case REFC_BINARY_SUBTAG: + refc = erts_refc_read(&u.pb->val->refc, 1); + break; + case FUN_SUBTAG: + refc = erts_refc_read(&u.fun->fe->refc, 1); + break; + case EXTERNAL_PID_SUBTAG: + case EXTERNAL_PORT_SUBTAG: + case EXTERNAL_REF_SUBTAG: + refc = erts_refc_read(&u.ext->node->refc, 1); + break; + default: + ASSERT(!!"erts_check_off_heap2: Invalid thing_word"); + } ERTS_CHK_OFFHEAP_ASSERT(refc >= 1); +#ifdef ERTS_OFFHEAP_DEBUG_CHK_CIRCULAR_LIST + ERTS_CHK_OFFHEAP_ASSERT(!(u.hdr->thing_word & ERTS_EXTERNAL_VISITED_BIT)); + u.hdr->thing_word |= ERTS_OFFHEAP_VISITED_BIT; +#endif if (old) { - ERTS_CHK_OFFHEAP_ASSERT(oheap <= ptr && ptr < ohtop); + ERTS_CHK_OFFHEAP_ASSERT(oheap <= u.ep && u.ep < ohtop); } - else if (oheap <= ptr && ptr < ohtop) + else if (oheap <= u.ep && u.ep < ohtop) old = 1; else { - ERTS_CHK_OFFHEAP_ASSERT(within2(ptr, p, htop)); + ERTS_CHK_OFFHEAP_ASSERT(within2(u.ep, p, htop)); } } - old = 0; - for (eft = MSO(p).funs; eft; eft = eft->next) { - Eterm *ptr = (Eterm *) eft; - long refc = erts_refc_read(&eft->fe->refc, 1); - ERTS_CHK_OFFHEAP_ASSERT(refc >= 1); - if (old) - ERTS_CHK_OFFHEAP_ASSERT(oheap <= ptr && ptr < ohtop); - else if (oheap <= ptr && ptr < ohtop) - old = 1; - else - ERTS_CHK_OFFHEAP_ASSERT(within2(ptr, p, htop)); - } - - old = 0; - for (et = MSO(p).externals; et; et = et->next) { - Eterm *ptr = (Eterm *) et; - long refc = erts_refc_read(&et->node->refc, 1); - ERTS_CHK_OFFHEAP_ASSERT(refc >= 1); -#ifdef ERTS_OFFHEAP_DEBUG_CHK_CIRCULAR_EXTERNAL_LIST - ERTS_CHK_OFFHEAP_ASSERT(!(et->header & ERTS_EXTERNAL_VISITED_BIT)); -#endif - if (old) - ERTS_CHK_OFFHEAP_ASSERT(oheap <= ptr && ptr < ohtop); - else if (oheap <= ptr && ptr < ohtop) - old = 1; - else - ERTS_CHK_OFFHEAP_ASSERT(within2(ptr, p, htop)); -#ifdef ERTS_OFFHEAP_DEBUG_CHK_CIRCULAR_EXTERNAL_LIST - et->header |= ERTS_EXTERNAL_VISITED_BIT; -#endif - } - #ifdef ERTS_OFFHEAP_DEBUG_CHK_CIRCULAR_EXTERNAL_LIST - for (et = MSO(p).externals; et; et = et->next) - et->header &= ~ERTS_EXTERNAL_VISITED_BIT; + for (u.hdr = MSO(p).first; u.hdr; u.hdr = u.hdr->next) + u.hdr->thing_word &= ~ERTS_OFFHEAP_VISITED_BIT; #endif - } void diff --git a/erts/emulator/beam/erl_message.c b/erts/emulator/beam/erl_message.c index b63f3df7df..1f61dca230 100644 --- a/erts/emulator/beam/erl_message.c +++ b/erts/emulator/beam/erl_message.c @@ -30,6 +30,7 @@ #include "erl_message.h" #include "erl_process.h" #include "erl_nmgc.h" +#include "erl_binary.h" ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(message, ErlMessage, @@ -164,16 +165,25 @@ erts_resize_message_buffer(ErlHeapFragment *bp, Uint size, void erts_cleanup_offheap(ErlOffHeap *offheap) { - if (offheap->mso) { - erts_cleanup_mso(offheap->mso); - } -#ifndef HYBRID /* FIND ME! */ - if (offheap->funs) { - erts_cleanup_funs(offheap->funs); - } -#endif - if (offheap->externals) { - erts_cleanup_externals(offheap->externals); + union erl_off_heap_ptr u; + + for (u.hdr = offheap->first; u.hdr; u.hdr = u.hdr->next) { + switch (thing_subtag(u.hdr->thing_word)) { + case REFC_BINARY_SUBTAG: + if (erts_refc_dectest(&u.pb->val->refc, 0) == 0) { + erts_bin_free(u.pb->val); + } + break; + case FUN_SUBTAG: + if (erts_refc_dectest(&u.fun->fe->refc, 0) == 0) { + erts_erase_fun_entry(u.fun->fe); + } + break; + default: + ASSERT(is_external_header(u.hdr->thing_word)); + erts_deref_node_entry(u.ext->node); + break; + } } } @@ -201,41 +211,17 @@ link_mbuf_to_proc(Process *proc, ErlHeapFragment *bp) MBUF_SIZE(proc) += bp->used_size; FLAGS(proc) |= F_FORCE_GC; - /* Move any binaries into the process */ - if (bp->off_heap.mso != NULL) { - ProcBin** next_p = &bp->off_heap.mso; + /* Move any off_heap's into the process */ + if (bp->off_heap.first != NULL) { + struct erl_off_heap_header** next_p = &bp->off_heap.first; while (*next_p != NULL) { next_p = &((*next_p)->next); } - *next_p = MSO(proc).mso; - MSO(proc).mso = bp->off_heap.mso; - bp->off_heap.mso = NULL; + *next_p = MSO(proc).first; + MSO(proc).first = bp->off_heap.first; + bp->off_heap.first = NULL; MSO(proc).overhead += bp->off_heap.overhead; } - - /* Move any funs into the process */ -#ifndef HYBRID - if (bp->off_heap.funs != NULL) { - ErlFunThing** next_p = &bp->off_heap.funs; - while (*next_p != NULL) { - next_p = &((*next_p)->next); - } - *next_p = MSO(proc).funs; - MSO(proc).funs = bp->off_heap.funs; - bp->off_heap.funs = NULL; - } -#endif - - /* Move any external things into the process */ - if (bp->off_heap.externals != NULL) { - ExternalThing** next_p = &bp->off_heap.externals; - while (*next_p != NULL) { - next_p = &((*next_p)->next); - } - *next_p = MSO(proc).externals; - MSO(proc).externals = bp->off_heap.externals; - bp->off_heap.externals = NULL; - } } } @@ -506,19 +492,7 @@ erts_link_mbuf_to_proc(struct process *proc, ErlHeapFragment *bp) void erts_move_msg_mbuf_to_heap(Eterm** hpp, ErlOffHeap* off_heap, ErlMessage *msg) { - /* Unions for typecasts avoids warnings about type-punned pointers and aliasing */ - union { - Uint** upp; - ProcBin **pbpp; - ErlFunThing **efpp; - ExternalThing **etpp; - } oh_list_pp, oh_el_next_pp; - union { - Uint *up; - ProcBin *pbp; - ErlFunThing *efp; - ExternalThing *etp; - } oh_el_p; + struct erl_off_heap_header* oh; Eterm term, token, *fhp, *hp; Sint offs; Uint sz; @@ -571,9 +545,7 @@ erts_move_msg_mbuf_to_heap(Eterm** hpp, ErlOffHeap* off_heap, ErlMessage *msg) hp = *hpp; offs = hp - fhp; - oh_list_pp.upp = NULL; - oh_el_next_pp.upp = NULL; /* Shut up compiler warning */ - oh_el_p.up = NULL; /* Shut up compiler warning */ + oh = NULL; while (sz--) { Uint cpy_sz; Eterm val = *fhp++; @@ -593,25 +565,11 @@ erts_move_msg_mbuf_to_heap(Eterm** hpp, ErlOffHeap* off_heap, ErlMessage *msg) case ARITYVAL_SUBTAG: break; case REFC_BINARY_SUBTAG: - oh_list_pp.pbpp = &off_heap->mso; - oh_el_p.up = (hp-1); - oh_el_next_pp.pbpp = &(oh_el_p.pbp)->next; - cpy_sz = thing_arityval(val); - goto cpy_words; case FUN_SUBTAG: -#ifndef HYBRID - oh_list_pp.efpp = &off_heap->funs; - oh_el_p.up = (hp-1); - oh_el_next_pp.efpp = &(oh_el_p.efp)->next; -#endif - cpy_sz = thing_arityval(val); - goto cpy_words; case EXTERNAL_PID_SUBTAG: case EXTERNAL_PORT_SUBTAG: case EXTERNAL_REF_SUBTAG: - oh_list_pp.etpp = &off_heap->externals; - oh_el_p.up = (hp-1); - oh_el_next_pp.etpp = &(oh_el_p.etp)->next; + oh = (struct erl_off_heap_header*) (hp-1); cpy_sz = thing_arityval(val); goto cpy_words; default: @@ -641,44 +599,13 @@ erts_move_msg_mbuf_to_heap(Eterm** hpp, ErlOffHeap* off_heap, ErlMessage *msg) case 1: *hp++ = *fhp++; default: break; } - if (oh_list_pp.upp) { -#ifdef HARD_DEBUG - Uint *dbg_old_oh_list_p = *oh_list_pp.upp; -#endif + if (oh) { /* Add to offheap list */ - *oh_el_next_pp.upp = *oh_list_pp.upp; - *oh_list_pp.upp = oh_el_p.up; - ASSERT(*hpp <= oh_el_p.up); - ASSERT(hp > oh_el_p.up); -#ifdef HARD_DEBUG - switch (val & _HEADER_SUBTAG_MASK) { - case REFC_BINARY_SUBTAG: - ASSERT(off_heap->mso == *oh_list_pp.pbpp); - ASSERT(off_heap->mso->next - == (ProcBin *) dbg_old_oh_list_p); - break; -#ifndef HYBRID - case FUN_SUBTAG: - ASSERT(off_heap->funs == *oh_list_pp.efpp); - ASSERT(off_heap->funs->next - == (ErlFunThing *) dbg_old_oh_list_p); - break; -#endif - case EXTERNAL_PID_SUBTAG: - case EXTERNAL_PORT_SUBTAG: - case EXTERNAL_REF_SUBTAG: - ASSERT(off_heap->externals - == *oh_list_pp.etpp); - ASSERT(off_heap->externals->next - == (ExternalThing *) dbg_old_oh_list_p); - break; - default: - ASSERT(0); - } -#endif - oh_list_pp.upp = NULL; - - + oh->next = off_heap->first; + off_heap->first = oh; + ASSERT(*hpp <= (Eterm*)oh); + ASSERT(hp > (Eterm*)oh); + oh = NULL; } break; } @@ -765,11 +692,7 @@ copy_done: #endif - bp->off_heap.mso = NULL; -#ifndef HYBRID - bp->off_heap.funs = NULL; -#endif - bp->off_heap.externals = NULL; + bp->off_heap.first = NULL; free_message_buffer(bp); msg->data.heap_frag = NULL; diff --git a/erts/emulator/beam/erl_message.h b/erts/emulator/beam/erl_message.h index f478572ac2..55ea92860a 100644 --- a/erts/emulator/beam/erl_message.h +++ b/erts/emulator/beam/erl_message.h @@ -28,13 +28,18 @@ struct external_thing_; * but is stored outside of any heap. */ -typedef struct erl_off_heap { - struct proc_bin* mso; /* List of associated binaries. */ -#ifndef HYBRID /* FIND ME! */ - struct erl_fun_thing* funs; /* List of funs. */ +struct erl_off_heap_header { + Eterm thing_word; + Uint size; +#if HALFWORD_HEAP + void* dummy_ptr_padding__; #endif - struct external_thing_* externals; /* List of external things. */ - int overhead; /* Administrative overhead (used to force GC). */ + struct erl_off_heap_header* next; +}; + +typedef struct erl_off_heap { + struct erl_off_heap_header* first; + int overhead; /* Administrative overhead (used to force GC). */ } ErlOffHeap; #include "external.h" @@ -201,9 +206,7 @@ do { \ (HEAP_FRAG_P)->next = NULL; \ (HEAP_FRAG_P)->alloc_size = (DATA_WORDS); \ (HEAP_FRAG_P)->used_size = (DATA_WORDS); \ - (HEAP_FRAG_P)->off_heap.mso = NULL; \ - (HEAP_FRAG_P)->off_heap.funs = NULL; \ - (HEAP_FRAG_P)->off_heap.externals = NULL; \ + (HEAP_FRAG_P)->off_heap.first = NULL; \ (HEAP_FRAG_P)->off_heap.overhead = 0; \ } while (0) diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c index 3d63fa1caf..e95d9c4f75 100644 --- a/erts/emulator/beam/erl_nif.c +++ b/erts/emulator/beam/erl_nif.c @@ -258,9 +258,7 @@ void enif_free_env(ErlNifEnv* env) static ERTS_INLINE void clear_offheap(ErlOffHeap* oh) { - oh->mso = NULL; - oh->externals = NULL; - oh->funs = NULL; + oh->first = NULL; oh->overhead = 0; } @@ -364,7 +362,7 @@ ERL_NIF_TERM enif_make_copy(ErlNifEnv* dst_env, ERL_NIF_TERM src_term) #ifdef DEBUG static int is_offheap(const ErlOffHeap* oh) { - return oh->mso != NULL || oh->funs != NULL || oh->externals != NULL; + return oh->first != NULL; } #endif @@ -627,8 +625,8 @@ Eterm enif_make_binary(ErlNifEnv* env, ErlNifBinary* bin) pb = (ProcBin *) alloc_heap(env, PROC_BIN_SIZE); pb->thing_word = HEADER_PROC_BIN; pb->size = bptr->orig_size; - pb->next = MSO(env->proc).mso; - MSO(env->proc).mso = pb; + pb->next = MSO(env->proc).first; + MSO(env->proc).first = (struct erl_off_heap_header*) pb; pb->val = bptr; pb->bytes = (byte*) bptr->orig_bytes; pb->flags = 0; diff --git a/erts/emulator/beam/erl_node_tables.c b/erts/emulator/beam/erl_node_tables.c index 5865d33138..7482da251e 100644 --- a/erts/emulator/beam/erl_node_tables.c +++ b/erts/emulator/beam/erl_node_tables.c @@ -1087,30 +1087,24 @@ insert_offheap2(ErlOffHeap *oh, void *arg) static void insert_offheap(ErlOffHeap *oh, int type, Eterm id) { - if(oh->externals) { - ExternalThing *etp = oh->externals; - while (etp) { - insert_node(etp->node, type, id); - etp = etp->next; - } - } + union erl_off_heap_ptr u; + struct insert_offheap2_arg a; + a.type = BIN_REF; - if(oh->mso) { - ProcBin *pb; - struct insert_offheap2_arg a; - a.type = BIN_REF; - for(pb = oh->mso; pb; pb = pb->next) { - if(IsMatchProgBinary(pb->val)) { + for (u.hdr = oh->first; u.hdr; u.hdr = u.hdr->next) { + switch (thing_subtag(u.hdr->thing_word)) { + case REFC_BINARY_SUBTAG: + if(IsMatchProgBinary(u.pb->val)) { InsertedBin *ib; int insert_bin = 1; for (ib = inserted_bins; ib; ib = ib->next) - if(ib->bin_val == pb->val) { + if(ib->bin_val == u.pb->val) { insert_bin = 0; break; } if (insert_bin) { #if HALFWORD_HEAP - UWord val = (UWord) pb->val; + UWord val = (UWord) u.pb->val; DeclareTmpHeapNoproc(id_heap,BIG_UINT_HEAP_SIZE*2); /* extra place allocated */ #else DeclareTmpHeapNoproc(id_heap,BIG_UINT_HEAP_SIZE); @@ -1124,13 +1118,13 @@ insert_offheap(ErlOffHeap *oh, int type, Eterm id) a.id = erts_bld_uword(&hp, NULL, (UWord) val); #else UseTmpHeapNoproc(BIG_UINT_HEAP_SIZE); - a.id = erts_bld_uint(&hp, NULL, (Uint) pb->val); + a.id = erts_bld_uint(&hp, NULL, (Uint) u.pb->val); #endif - erts_match_prog_foreach_offheap(pb->val, + erts_match_prog_foreach_offheap(u.pb->val, insert_offheap2, (void *) &a); nib = erts_alloc(ERTS_ALC_T_NC_TMP, sizeof(InsertedBin)); - nib->bin_val = pb->val; + nib->bin_val = u.pb->val; nib->next = inserted_bins; inserted_bins = nib; #if HALFWORD_HEAP @@ -1139,15 +1133,16 @@ insert_offheap(ErlOffHeap *oh, int type, Eterm id) UnUseTmpHeapNoproc(BIG_UINT_HEAP_SIZE); #endif } - } + } + break; + case FUN_SUBTAG: + break; /* No need to */ + default: + ASSERT(is_external_header(u.hdr->thing_word)); + insert_node(u.ext->node, type, id); + break; } } - -#if 0 - if(oh->funs) { - /* No need to */ - } -#endif } static void doit_insert_monitor(ErtsMonitor *monitor, void *p) @@ -1289,6 +1284,7 @@ setup_reference_table(void) for (i = 0; i < erts_max_processes; i++) if (process_tab[i]) { ErlMessage *msg; + /* Insert Heap */ insert_offheap(&(process_tab[i]->off_heap), HEAP_REF, @@ -1375,21 +1371,22 @@ setup_reference_table(void) { /* Add binaries stored elsewhere ... */ ErlOffHeap oh; - ProcBin pb[2] = {{0},{0}}; - ProcBin *mso = NULL; + ProcBin pb[2]; int i = 0; Binary *default_match_spec; Binary *default_meta_match_spec; - /* Only the ProcBin members val and next will be inspected + oh.first = NULL; + /* Only the ProcBin members thing_word, val and next will be inspected (by insert_offheap()) */ #undef ADD_BINARY -#define ADD_BINARY(Bin) \ - if ((Bin)) { \ - pb[i].val = (Bin); \ - pb[i].next = mso; \ - mso = &pb[i]; \ - i++; \ +#define ADD_BINARY(Bin) \ + if ((Bin)) { \ + pb[i].thing_word = REFC_BINARY_SUBTAG; \ + pb[i].val = (Bin); \ + pb[i].next = oh.first; \ + oh.first = (struct erl_off_heap_header*) &pb[i]; \ + i++; \ } erts_get_default_trace_pattern(NULL, @@ -1401,11 +1398,6 @@ setup_reference_table(void) ADD_BINARY(default_match_spec); ADD_BINARY(default_meta_match_spec); - oh.mso = mso; - oh.externals = NULL; -#ifndef HYBRID /* FIND ME! */ - oh.funs = NULL; -#endif insert_offheap(&oh, BIN_REF, AM_match_spec); #undef ADD_BINARY } diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index bae17d1569..8c88353593 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -3787,11 +3787,11 @@ verify_topology(erts_cpu_topology_t *cpudata, int size) /* Verify logical ids */ logical = erts_alloc(ERTS_ALC_T_TMP, sizeof(int)*size); - for (i = 0; i < user_cpudata_size; i++) - logical[i] = user_cpudata[i].logical; + for (i = 0; i < size; i++) + logical[i] = cpudata[i].logical; - qsort(logical, user_cpudata_size, sizeof(int), int_cmp); - for (i = 0; i < user_cpudata_size-1; i++) { + qsort(logical, size, sizeof(int), int_cmp); + for (i = 0; i < size-1; i++) { if (logical[i] == logical[i+1]) { erts_free(ERTS_ALC_T_TMP, logical); return ERTS_INIT_CPU_TOPOLOGY_NOT_UNIQUE_LIDS; @@ -3804,13 +3804,13 @@ verify_topology(erts_cpu_topology_t *cpudata, int size) /* Verify unique entities */ - for (i = 1; i < user_cpudata_size; i++) { - if (user_cpudata[i-1].processor == user_cpudata[i].processor - && user_cpudata[i-1].node == user_cpudata[i].node - && (user_cpudata[i-1].processor_node - == user_cpudata[i].processor_node) - && user_cpudata[i-1].core == user_cpudata[i].core - && user_cpudata[i-1].thread == user_cpudata[i].thread) { + for (i = 1; i < size; i++) { + if (cpudata[i-1].processor == cpudata[i].processor + && cpudata[i-1].node == cpudata[i].node + && (cpudata[i-1].processor_node + == cpudata[i].processor_node) + && cpudata[i-1].core == cpudata[i].core + && cpudata[i-1].thread == cpudata[i].thread) { return ERTS_INIT_CPU_TOPOLOGY_NOT_UNIQUE_ENTITIES; } } @@ -6761,11 +6761,7 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). /* * Must initialize binary lists here before copying binaries to process. */ - p->off_heap.mso = NULL; -#ifndef HYBRID /* FIND ME! */ - p->off_heap.funs = NULL; -#endif - p->off_heap.externals = NULL; + p->off_heap.first = NULL; p->off_heap.overhead = 0; heap_need += @@ -6855,7 +6851,7 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). p->group_leader = IS_CONST(parent->group_leader) ? parent->group_leader - : STORE_NC(&p->htop, &p->off_heap.externals, parent->group_leader); + : STORE_NC(&p->htop, &p->off_heap, parent->group_leader); } erts_get_default_tracing(&p->trace_flags, &p->tracer_proc); @@ -7056,11 +7052,7 @@ void erts_init_empty_process(Process *p) memset(&(p->u.tm), 0, sizeof(ErlTimer)); #endif p->next = NULL; - p->off_heap.mso = NULL; -#ifndef HYBRID /* FIND ME! */ - p->off_heap.funs = NULL; -#endif - p->off_heap.externals = NULL; + p->off_heap.first = NULL; p->off_heap.overhead = 0; p->reg = NULL; p->heap_sz = 0; @@ -7207,11 +7199,7 @@ erts_debug_verify_clean_empty_process(Process* p) /* Thing that erts_cleanup_empty_process() cleans up */ - ASSERT(p->off_heap.mso == NULL); -#ifndef HYBRID /* FIND ME! */ - ASSERT(p->off_heap.funs == NULL); -#endif - ASSERT(p->off_heap.externals == NULL); + ASSERT(p->off_heap.first == NULL); ASSERT(p->off_heap.overhead == 0); ASSERT(p->mbuf == NULL); @@ -7225,11 +7213,7 @@ erts_cleanup_empty_process(Process* p) /* We only check fields that are known to be used... */ erts_cleanup_offheap(&p->off_heap); - p->off_heap.mso = NULL; -#ifndef HYBRID /* FIND ME! */ - p->off_heap.funs = NULL; -#endif - p->off_heap.externals = NULL; + p->off_heap.first = NULL; p->off_heap.overhead = 0; if (p->mbuf != NULL) { @@ -7266,7 +7250,7 @@ delete_process(Process* p) * The mso list should not be used anymore, but if it is, make sure that * we'll notice. */ - p->off_heap.mso = (void *) 0x8DEFFACD; + p->off_heap.first = (void *) 0x8DEFFACD; if (p->arg_reg != p->def_arg_reg) { erts_free(ERTS_ALC_T_ARG_REG, p->arg_reg); diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index 8f9f7f004e..6ce74a917f 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -828,7 +828,7 @@ extern struct erts_system_profile_flags_t erts_system_profile_flags; #define F_INSLPQUEUE (1 << 1) /* Set if in timer queue */ #define F_TIMO (1 << 2) /* Set if timeout */ #define F_HEAP_GROW (1 << 3) -#define F_NEED_FULLSWEEP (1 << 4) /* If process has old binaries & funs. */ +#define F_NEED_FULLSWEEP (1 << 4) #define F_USING_DB (1 << 5) /* If have created tables */ #define F_DISTRIBUTION (1 << 6) /* Process used in distribution */ #define F_USING_DDLL (1 << 7) /* Process has used the DDLL interface */ diff --git a/erts/emulator/beam/erl_term.h b/erts/emulator/beam/erl_term.h index 3a8c30fe6a..b8e4473141 100644 --- a/erts/emulator/beam/erl_term.h +++ b/erts/emulator/beam/erl_term.h @@ -821,10 +821,10 @@ _ET_DECLARE_CHECKED(struct erl_node_*,internal_ref_node,Eterm) * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ * |A A A A A A A A A A A A A A A A A A A A A A A A A A|t t t t|0 0| Thing * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * |N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N| Next - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ * |E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E E| ErlNode * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + * |N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N N| Next + * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ * |X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X| Data 0 * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ * . . . @@ -835,7 +835,7 @@ _ET_DECLARE_CHECKED(struct erl_node_*,internal_ref_node,Eterm) * t : External pid thing tag (1100) * t : External port thing tag (1101) * t : External ref thing tag (1110) - * N : Next (external thing) pointer + * N : Next (off_heap) pointer * E : ErlNode pointer * X : Type specific data * @@ -852,13 +852,16 @@ _ET_DECLARE_CHECKED(struct erl_node_*,internal_ref_node,Eterm) /* XXX:PaN - this structure is not perfect for halfword heap, it takes a lot of memory due to padding, and the array will not begin at the end of the structure, as otherwise expected. Be sure to access data.ui32 array and not try - to do pointer manipulation on an Eterm * to reach the actual data... */ + to do pointer manipulation on an Eterm * to reach the actual data... + XXX:Sverk - Problem made worse by "one off-heap list" when 'next' pointer + must align with 'next' in ProcBin, erl_fun_thing and erl_off_heap_header. +*/ typedef struct external_thing_ { /* ----+ */ Eterm header; /* | */ - struct external_thing_ *next; /* > External thing head */ - struct erl_node_ *node; /* | */ + struct erl_node_* node; /* > External thing head */ + struct erl_off_heap_header* next; /* | */ /* ----+ */ union { Uint32 ui32[1]; diff --git a/erts/emulator/beam/external.c b/erts/emulator/beam/external.c index f41b61d73d..310a83200b 100644 --- a/erts/emulator/beam/external.c +++ b/erts/emulator/beam/external.c @@ -65,11 +65,9 @@ # endif #endif -/* - * For backward compatibility reasons, only encode integers that - * fit in 28 bits (signed) using INTEGER_EXT. +/* Does Sint fit in Sint32? */ -#define IS_SSMALL28(x) (((Uint) (((x) >> (28-1)) + 1)) < 2) +#define IS_SSMALL32(x) (((Uint) (((x) >> (32-1)) + 1)) < 2) /* * Valid creations for nodes are 1, 2, or 3. 0 can also be sent @@ -1470,11 +1468,11 @@ dec_pid(ErtsDistExternal *edep, Eterm** hpp, byte* ep, ErlOffHeap* off_heap, Ete *hpp += EXTERNAL_THING_HEAD_SIZE + 1; etp->header = make_external_pid_header(1); - etp->next = off_heap->externals; + etp->next = off_heap->first; etp->node = node; etp->data.ui[0] = data; - off_heap->externals = etp; + off_heap->first = (struct erl_off_heap_header*) etp; *objp = make_external_pid(etp); } return ep; @@ -1571,13 +1569,15 @@ enc_term(ErtsAtomCacheMap *acmp, Eterm obj, byte* ep, Uint32 dflags) case SMALL_DEF: { + /* From R14B we no longer restrict INTEGER_EXT to 28 bits, + * as done earlier for backward compatibility reasons. */ Sint val = signed_val(obj); if ((Uint)val < 256) { *ep++ = SMALL_INTEGER_EXT; put_int8(val, ep); ep++; - } else if (sizeof(Sint) == 4 || IS_SSMALL28(val)) { + } else if (sizeof(Sint) == 4 || IS_SSMALL32(val)) { *ep++ = INTEGER_EXT; put_int32(val, ep); ep += 4; @@ -1599,18 +1599,32 @@ enc_term(ErtsAtomCacheMap *acmp, Eterm obj, byte* ep, Uint32 dflags) break; case BIG_DEF: - if ((n = big_bytes(obj)) < 256) { - *ep++ = SMALL_BIG_EXT; - put_int8(n, ep); - ep += 1; - } - else { - *ep++ = LARGE_BIG_EXT; - put_int32(n, ep); - ep += 4; + { + int sign = big_sign(obj); + n = big_bytes(obj); + if (sizeof(Sint)==4 && n<=4) { + Uint dig = big_digit(obj,0); + Sint val = sign ? -dig : dig; + if ((val<0) == sign) { + *ep++ = INTEGER_EXT; + put_int32(val, ep); + ep += 4; + break; + } + } + if (n < 256) { + *ep++ = SMALL_BIG_EXT; + put_int8(n, ep); + ep += 1; + } + else { + *ep++ = LARGE_BIG_EXT; + put_int32(n, ep); + ep += 4; + } + *ep++ = sign; + ep = big_to_bytes(obj, ep); } - *ep++ = big_sign(obj); - ep = big_to_bytes(obj, ep); break; case PID_DEF: @@ -1896,69 +1910,30 @@ is_external_string(Eterm list, int* p_is_string) return len; } -/* Assumes that the ones to undo are preluding the lists. */ +/* Assumes that the ones to undo are preluding the list. */ static void undo_offheap_in_area(ErlOffHeap* off_heap, Eterm* start, Eterm* end) { const Uint area_sz = (end - start) * sizeof(Eterm); - struct proc_bin* mso; - struct proc_bin** mso_nextp = NULL; -#ifndef HYBRID /* FIND ME! */ - struct erl_fun_thing* funs; - struct erl_fun_thing** funs_nextp = NULL; -#endif - struct external_thing_* ext; - struct external_thing_** ext_nextp = NULL; - - for (mso = off_heap->mso; ; mso=mso->next) { - if (!in_area(mso, start, area_sz)) { - if (mso_nextp != NULL) { - *mso_nextp = NULL; - erts_cleanup_mso(off_heap->mso); - off_heap->mso = mso; + struct erl_off_heap_header* hdr; + struct erl_off_heap_header** hdr_nextp = NULL; + + for (hdr = off_heap->first; ; hdr=hdr->next) { + if (!in_area(hdr, start, area_sz)) { + if (hdr_nextp != NULL) { + *hdr_nextp = NULL; + erts_cleanup_offheap(off_heap); + off_heap->first = hdr; } break; } - mso_nextp = &mso->next; + hdr_nextp = &hdr->next; } -#ifndef HYBRID /* FIND ME! */ - for (funs = off_heap->funs; ; funs=funs->next) { - if (!in_area(funs, start, area_sz)) { - if (funs_nextp != NULL) { - *funs_nextp = NULL; - erts_cleanup_funs(off_heap->funs); - off_heap->funs = funs; - } - break; - } - funs_nextp = &funs->next; - } -#endif - for (ext = off_heap->externals; ; ext=ext->next) { - if (!in_area(ext, start, area_sz)) { - if (ext_nextp != NULL) { - *ext_nextp = NULL; - erts_cleanup_externals(off_heap->externals); - off_heap->externals = ext; - } - break; - } - ext_nextp = &ext->next; - } - - /* Assert that the ones to undo were indeed preluding the lists. */ + /* Assert that the ones to undo were indeed preluding the list. */ #ifdef DEBUG - for (mso = off_heap->mso; mso != NULL; mso=mso->next) { - ASSERT(!in_area(mso, start, area_sz)); - } -# ifndef HYBRID /* FIND ME! */ - for (funs = off_heap->funs; funs != NULL; funs=funs->next) { - ASSERT(!in_area(funs, start, area_sz)); - } -# endif - for (ext = off_heap->externals; ext != NULL; ext=ext->next) { - ASSERT(!in_area(ext, start, area_sz)); + for (hdr = off_heap->first; hdr != NULL; hdr = hdr->next) { + ASSERT(!in_area(hdr, start, area_sz)); } #endif /* DEBUG */ } @@ -2202,11 +2177,11 @@ dec_term_atom_common: hp += EXTERNAL_THING_HEAD_SIZE + 1; etp->header = make_external_port_header(1); - etp->next = off_heap->externals; + etp->next = off_heap->first; etp->node = node; etp->data.ui[0] = num; - off_heap->externals = etp; + off_heap->first = (struct erl_off_heap_header*)etp; *objp = make_external_port(etp); } @@ -2284,10 +2259,10 @@ dec_term_atom_common: #else etp->header = make_external_ref_header(ref_words); #endif - etp->next = off_heap->externals; + etp->next = off_heap->first; etp->node = node; - off_heap->externals = etp; + off_heap->first = (struct erl_off_heap_header*)etp; *objp = make_external_ref(etp); ref_num = &(etp->data.ui32[0]); } @@ -2330,8 +2305,8 @@ dec_term_atom_common: hp += PROC_BIN_SIZE; pb->thing_word = HEADER_PROC_BIN; pb->size = n; - pb->next = off_heap->mso; - off_heap->mso = pb; + pb->next = off_heap->first; + off_heap->first = (struct erl_off_heap_header*)pb; pb->val = dbin; pb->bytes = (byte*) dbin->orig_bytes; pb->flags = 0; @@ -2367,8 +2342,8 @@ dec_term_atom_common: pb = (ProcBin *) hp; pb->thing_word = HEADER_PROC_BIN; pb->size = n; - pb->next = off_heap->mso; - off_heap->mso = pb; + pb->next = off_heap->first; + off_heap->first = (struct erl_off_heap_header*)pb; pb->val = dbin; pb->bytes = (byte*) dbin->orig_bytes; pb->flags = 0; @@ -2488,8 +2463,8 @@ dec_term_atom_common: * It is safe to link the fun into the fun list only when * no more validity tests can fail. */ - funp->next = off_heap->funs; - off_heap->funs = funp; + funp->next = off_heap->first; + off_heap->first = (struct erl_off_heap_header*)funp; #endif funp->fe = erts_put_fun_entry2(module, old_uniq, old_index, @@ -2566,8 +2541,8 @@ dec_term_atom_common: * It is safe to link the fun into the fun list only when * no more validity tests can fail. */ - funp->next = off_heap->funs; - off_heap->funs = funp; + funp->next = off_heap->first; + off_heap->first = (struct erl_off_heap_header*)funp; #endif old_uniq = unsigned_val(temp); @@ -2687,7 +2662,7 @@ encode_size_struct2(ErtsAtomCacheMap *acmp, Eterm obj, unsigned dflags) if ((Uint)val < 256) result += 1 + 1; /* SMALL_INTEGER_EXT */ - else if (sizeof(Sint) == 4 || IS_SSMALL28(val)) + else if (sizeof(Sint) == 4 || IS_SSMALL32(val)) result += 1 + 4; /* INTEGER_EXT */ else { DeclareTmpHeapNoproc(tmp_big,2); @@ -2699,7 +2674,10 @@ encode_size_struct2(ErtsAtomCacheMap *acmp, Eterm obj, unsigned dflags) } break; case BIG_DEF: - if ((i = big_bytes(obj)) < 256) + i = big_bytes(obj); + if (sizeof(Sint)==4 && i <= 4 && (big_digit(obj,0)-big_sign(obj)) < (1<<31)) + result += 1 + 4; /* INTEGER_EXT */ + else if (i < 256) result += 1 + 1 + 1 + i; /* tag,size,sign,digits */ else result += 1 + 4 + 1 + i; /* tag,size,sign,digits */ diff --git a/erts/emulator/beam/global.h b/erts/emulator/beam/global.h index b4a7a22082..064dc69da8 100644 --- a/erts/emulator/beam/global.h +++ b/erts/emulator/beam/global.h @@ -464,7 +464,10 @@ typedef union { typedef struct proc_bin { Eterm thing_word; /* Subtag REFC_BINARY_SUBTAG. */ Uint size; /* Binary size in bytes. */ - struct proc_bin *next; /* Pointer to next ProcBin. */ +#if HALFWORD_HEAP + void* dummy_ptr_padding__; +#endif + struct erl_off_heap_header *next; Binary *val; /* Pointer to Binary structure. */ byte *bytes; /* Pointer to the actual data bytes. */ Uint flags; /* Flag word. */ @@ -494,8 +497,8 @@ erts_mk_magic_binary_term(Eterm **hpp, ErlOffHeap *ohp, Binary *mbp) pb->thing_word = HEADER_PROC_BIN; pb->size = 0; - pb->next = ohp->mso; - ohp->mso = pb; + pb->next = ohp->first; + ohp->first = (struct erl_off_heap_header*) pb; pb->val = mbp; pb->bytes = (byte *) mbp->orig_bytes; pb->flags = 0; @@ -512,6 +515,15 @@ erts_mk_magic_binary_term(Eterm **hpp, ErlOffHeap *ohp, Binary *mbp) && (thing_subtag(*binary_val((T))) == REFC_BINARY_SUBTAG) \ && (((ProcBin *) binary_val((T)))->val->flags & BIN_FLAG_MAGIC)) + +union erl_off_heap_ptr { + struct erl_off_heap_header* hdr; + ProcBin *pb; + struct erl_fun_thing* fun; + struct external_thing_* ext; + Eterm* ep; +}; + /* arrays that get malloced at startup */ extern Port* erts_port; extern erts_smp_atomic_t erts_ports_alive; @@ -823,7 +835,6 @@ Eterm erts_new_heap_binary(Process *p, byte *buf, int len, byte** datap); Eterm erts_new_mso_binary(Process*, byte*, int); Eterm new_binary(Process*, byte*, int); Eterm erts_realloc_binary(Eterm bin, size_t size); -void erts_cleanup_mso(ProcBin* pb); /* erl_bif_info.c */ @@ -1499,7 +1510,6 @@ void p_slpq(_VOID_); void erts_silence_warn_unused_result(long unused); void erts_cleanup_offheap(ErlOffHeap *offheap); -void erts_cleanup_externals(ExternalThing *); Uint erts_fit_in_bits(Uint); int list_length(Eterm); @@ -1532,7 +1542,7 @@ erts_bld_atom_2uint_3tup_list(Uint **hpp, Uint *szp, Sint length, Eterm atoms[], Uint uints1[], Uint uints2[]); Eterm store_external_or_ref_in_proc_(Process *, Eterm); -Eterm store_external_or_ref_(Uint **, ExternalThing **, Eterm); +Eterm store_external_or_ref_(Uint **, ErlOffHeap*, Eterm); #define NC_HEAP_SIZE(NC) \ (ASSERT_EXPR(is_node_container((NC))), \ @@ -1957,5 +1967,5 @@ erts_alloc_message_heap(Uint size, # define UnUseTmpHeap(Size,Proc) /* Nothing */ # define UseTmpHeapNoproc(Size) /* Nothing */ # define UnUseTmpHeapNoproc(Size) /* Nothing */ -#endif -#endif +#endif /* HEAP_ON_C_STACK */ +#endif /* !__GLOBAL_H__ */ diff --git a/erts/emulator/beam/io.c b/erts/emulator/beam/io.c index 68625801cf..b840f65cdd 100644 --- a/erts/emulator/beam/io.c +++ b/erts/emulator/beam/io.c @@ -1575,8 +1575,8 @@ static void deliver_read_message(Port* prt, Eterm to, pb = (ProcBin *) hp; pb->thing_word = HEADER_PROC_BIN; pb->size = len; - pb->next = ohp->mso; - ohp->mso = pb; + pb->next = ohp->first; + ohp->first = (struct erl_off_heap_header*)pb; pb->val = bptr; pb->bytes = (byte*) bptr->orig_bytes; pb->flags = 0; @@ -1725,8 +1725,8 @@ deliver_vec_message(Port* prt, /* Port */ } pb->thing_word = HEADER_PROC_BIN; pb->size = iov->iov_len; - pb->next = ohp->mso; - ohp->mso = pb; + pb->next = ohp->first; + ohp->first = (struct erl_off_heap_header*)pb; pb->val = ErlDrvBinary2Binary(b); pb->bytes = base; pb->flags = 0; @@ -2259,8 +2259,8 @@ erts_port_control(Process* p, Port* prt, Uint command, Eterm iolist) ProcBin* pb = (ProcBin *) HAlloc(p, PROC_BIN_SIZE); pb->thing_word = HEADER_PROC_BIN; pb->size = dbin->orig_size; - pb->next = MSO(p).mso; - MSO(p).mso = pb; + pb->next = MSO(p).first; + MSO(p).first = (struct erl_off_heap_header*)pb; pb->val = ErlDrvBinary2Binary(dbin); pb->bytes = (byte*) dbin->orig_bytes; pb->flags = 0; @@ -3033,8 +3033,8 @@ driver_deliver_term(ErlDrvPort port, driver_binary_inc_refc(b); /* caller will free binary */ pb->thing_word = HEADER_PROC_BIN; pb->size = size; - pb->next = ohp->mso; - ohp->mso = pb; + pb->next = ohp->first; + ohp->first = (struct erl_off_heap_header*)pb; pb->val = ErlDrvBinary2Binary(b); pb->bytes = ((byte*) b->orig_bytes) + offset; pb->flags = 0; @@ -3072,8 +3072,8 @@ driver_deliver_term(ErlDrvPort port, hp += PROC_BIN_SIZE; pbp->thing_word = HEADER_PROC_BIN; pbp->size = size; - pbp->next = ohp->mso; - ohp->mso = pbp; + pbp->next = ohp->first; + ohp->first = (struct erl_off_heap_header*)pbp; pbp->val = bp; pbp->bytes = (byte*) bp->orig_bytes; pbp->flags = 0; diff --git a/erts/emulator/beam/packet_parser.c b/erts/emulator/beam/packet_parser.c index 8c8029d450..5bcd567b5f 100644 --- a/erts/emulator/beam/packet_parser.c +++ b/erts/emulator/beam/packet_parser.c @@ -679,7 +679,7 @@ int packet_parse_http(const char* buf, int len, int* statep, while (n && SP(ptr)) { ptr++; n--; } - if (ptr==p0) return -1; + if (ptr==p0 && n>0) return -1; /* NOTE: the syntax allows empty reason phrases */ (*statep) = !0; diff --git a/erts/emulator/beam/utils.c b/erts/emulator/beam/utils.c index da6f9ed12f..25472ef47a 100644 --- a/erts/emulator/beam/utils.c +++ b/erts/emulator/beam/utils.c @@ -162,13 +162,8 @@ erts_heap_alloc(Process* p, Uint need) bp->alloc_size = n; bp->used_size = n; MBUF_SIZE(p) += n; - bp->off_heap.mso = NULL; -#ifndef HYBRID /* FIND ME! */ - bp->off_heap.funs = NULL; -#endif - bp->off_heap.externals = NULL; + bp->off_heap.first = NULL; bp->off_heap.overhead = 0; - return bp->mem; } @@ -409,7 +404,7 @@ erts_bld_uint64(Uint **hpp, Uint *szp, Uint64 ui64) } else { if (szp) - *szp = ERTS_UINT64_HEAP_SIZE(ui64); + *szp += ERTS_UINT64_HEAP_SIZE(ui64); if (hpp) res = erts_uint64_to_big(ui64, hpp); } @@ -426,7 +421,7 @@ erts_bld_sint64(Uint **hpp, Uint *szp, Sint64 si64) } else { if (szp) - *szp = ERTS_SINT64_HEAP_SIZE(si64); + *szp += ERTS_SINT64_HEAP_SIZE(si64); if (hpp) res = erts_sint64_to_big(si64, hpp); } @@ -2729,21 +2724,8 @@ not_equal: } -void -erts_cleanup_externals(ExternalThing *etp) -{ - ExternalThing *tetp; - - tetp = etp; - - while(tetp) { - erts_deref_node_entry(tetp->node); - tetp = tetp->next; - } -} - Eterm -store_external_or_ref_(Uint **hpp, ExternalThing **etpp, Eterm ns) +store_external_or_ref_(Uint **hpp, ErlOffHeap* oh, Eterm ns) { Uint i; Uint size; @@ -2762,8 +2744,8 @@ store_external_or_ref_(Uint **hpp, ExternalThing **etpp, Eterm ns) erts_refc_inc(&((ExternalThing *) to_hp)->node->refc, 2); - ((ExternalThing *) to_hp)->next = *etpp; - *etpp = (ExternalThing *) to_hp; + ((struct erl_off_heap_header*) to_hp)->next = oh->first; + oh->first = (struct erl_off_heap_header*) to_hp; return make_external(to_hp); } @@ -2792,7 +2774,7 @@ store_external_or_ref_in_proc_(Process *proc, Eterm ns) sz = NC_HEAP_SIZE(ns); ASSERT(sz > 0); hp = HAlloc(proc, sz); - return store_external_or_ref_(&hp, &MSO(proc).externals, ns); + return store_external_or_ref_(&hp, &MSO(proc), ns); } void bin_write(int to, void *to_arg, byte* buf, int sz) diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 87691fc1bc..0ea54930ba 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -9333,11 +9333,13 @@ static int packet_inet_input(udp_descriptor* udesc, HANDLE event) if (err != ERRNO_BLOCK) { if (!desc->active) { #ifdef HAVE_SCTP - if (short_recv) + if (short_recv) { async_error_am(desc, am_short_recv); - else -#else + } else { async_error(desc, err); + } +#else + async_error(desc, err); #endif driver_cancel_timer(desc->port); sock_select(desc,FD_READ,0); diff --git a/erts/emulator/hipe/hipe_mkliterals.c b/erts/emulator/hipe/hipe_mkliterals.c index a77aec7919..900dfc5a8a 100644 --- a/erts/emulator/hipe/hipe_mkliterals.c +++ b/erts/emulator/hipe/hipe_mkliterals.c @@ -261,7 +261,7 @@ static const struct literal { /* Field offsets in a process struct */ { "P_HP", offsetof(struct process, htop) }, { "P_HP_LIMIT", offsetof(struct process, stop) }, - { "P_OFF_HEAP_MSO", offsetof(struct process, off_heap.mso) }, + { "P_OFF_HEAP_FIRST", offsetof(struct process, off_heap.first) }, { "P_MBUF", offsetof(struct process, mbuf) }, { "P_ID", offsetof(struct process, id) }, { "P_FLAGS", offsetof(struct process, flags) }, @@ -456,7 +456,7 @@ static const struct rts_param { } rts_params[] = { { 1, "P_OFF_HEAP_FUNS", #if !defined(HYBRID) - 1, offsetof(struct process, off_heap.funs) + 1, offsetof(struct process, off_heap.first) #endif }, diff --git a/erts/emulator/sys/unix/sys.c b/erts/emulator/sys/unix/sys.c index 151fa06e8e..737ffd9f94 100644 --- a/erts/emulator/sys/unix/sys.c +++ b/erts/emulator/sys/unix/sys.c @@ -1328,10 +1328,18 @@ static char **build_unix_environment(char *block) } } - for (j = 0; j < i; j++) { + for (j = 0; j < i; ) { size_t last = strlen(cpp[j])-1; if (cpp[j][last] == '=' && strchr(cpp[j], '=') == cpp[j]+last) { cpp[j] = cpp[--len]; + if (len < i) { + i--; + } else { + j++; + } + } + else { + j++; } } diff --git a/erts/emulator/sys/win32/sys_env.c b/erts/emulator/sys/win32/sys_env.c index ac4be3f316..02c8433a10 100644 --- a/erts/emulator/sys/win32/sys_env.c +++ b/erts/emulator/sys/win32/sys_env.c @@ -1,19 +1,19 @@ /* * %CopyrightBegin% - * - * Copyright Ericsson AB 2002-2009. All Rights Reserved. - * + * + * Copyright Ericsson AB 2002-2010. All Rights Reserved. + * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in * compliance with the License. You should have received a copy of the * Erlang Public License along with this software. If not, it can be * retrieved online at http://www.erlang.org/. - * + * * Software distributed under the License is distributed on an "AS IS" * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See * the License for the specific language governing rights and limitations * under the License. - * + * * %CopyrightEnd% */ @@ -145,15 +145,17 @@ merge_environment(char *old, char *add) for(j = 0; a_arg[j] != NULL; ++j){ char **tmp; char *current = a_arg[j]; + char *eq_p = strchr(current,'='); + int unset = (eq_p!=NULL && eq_p[1]=='\0'); if ((tmp = find_arg(c_arg, current)) != NULL) { - if (current[strlen(current)-1] != '=') { + if (!unset) { *tmp = current; } else { *tmp = c_arg[--i]; c_arg[i] = NULL; } - } else if (current[strlen(current)-1] != '=') { + } else if (!unset) { c_arg[i++] = current; c_arg[i] = NULL; } diff --git a/erts/emulator/test/decode_packet_SUITE.erl b/erts/emulator/test/decode_packet_SUITE.erl index 6cde286871..d9e961be2f 100644 --- a/erts/emulator/test/decode_packet_SUITE.erl +++ b/erts/emulator/test/decode_packet_SUITE.erl @@ -304,6 +304,10 @@ http(Config) when is_list(Config) -> {ok, {http_request, 'GET', ResB, {1,1}}, Rest} = decode_pkt(http_bin,Bin) end, lists:foreach(UriF, http_uri_variants()), + + %% Response with empty phrase + ?line {ok,{http_response,{1,1},200,[]},<<>>} = decode_pkt(http, <<"HTTP/1.1 200\r\n">>, []), + ?line {ok,{http_response,{1,1},200,<<>>},<<>>} = decode_pkt(http_bin, <<"HTTP/1.1 200\r\n">>, []), ok. http_with_bin(http) -> diff --git a/erts/emulator/test/hash_SUITE.erl b/erts/emulator/test/hash_SUITE.erl index 85bdb8bff8..f5d1871bfb 100644 --- a/erts/emulator/test/hash_SUITE.erl +++ b/erts/emulator/test/hash_SUITE.erl @@ -480,14 +480,14 @@ otp_5292_test() -> S2 = md5([md5(hash_int(S, E, PH)) || {Start, N, Sz} <- d(), {S, E} <- int(Start, N, Sz)]), ?line Comment = case S1 of - <<43,186,76,102,87,4,110,245,203,177,206,6,130,69,43,99>> -> + <<4,248,208,156,200,131,7,1,173,13,239,173,112,81,16,174>> -> ?line big = erlang:system_info(endian), "Big endian machine"; - <<21,206,139,15,149,28,167,81,98,225,132,254,49,125,174,195>> -> + <<180,28,33,231,239,184,71,125,76,47,227,241,78,184,176,233>> -> ?line little = erlang:system_info(endian), "Little endian machine" end, - ?line <<140,37,79,80,26,242,130,22,20,229,123,240,223,244,43,99>> = S2, + ?line <<124,81,198,121,174,233,19,137,10,83,33,80,226,111,238,99>> = S2, ?line 2 = erlang:hash(1, (1 bsl 27) -1), ?line {'EXIT', _} = (catch erlang:hash(1, (1 bsl 27))), {comment, Comment}. @@ -507,7 +507,7 @@ hash_int(Start, End, F) -> {Start, End, md5(HL)}. md5(T) -> - erlang:md5(term_to_binary(T)). + erlang:md5(term_to_binary(T)). bit_level_binaries() -> ?line [3511317,7022633,14044578,28087749,56173436,112344123,90467083|_] = diff --git a/erts/emulator/test/nif_SUITE.erl b/erts/emulator/test/nif_SUITE.erl index 888bf582d8..f45cfa3e4a 100644 --- a/erts/emulator/test/nif_SUITE.erl +++ b/erts/emulator/test/nif_SUITE.erl @@ -30,7 +30,7 @@ fin_per_testcase/2, basic/1, reload/1, upgrade/1, heap_frag/1, types/1, many_args/1, binaries/1, get_string/1, get_atom/1, api_macros/1, from_array/1, iolist_as_binary/1, resource/1, resource_binary/1, resource_takeover/1, - threading/1, send/1, send2/1, send_threaded/1, neg/1, is_checks/1, + threading/1, send/1, send2/1, send3/1, send_threaded/1, neg/1, is_checks/1, get_length/1, make_atom/1, make_string/1]). -export([many_args_100/100]). @@ -51,7 +51,7 @@ all(suite) -> [basic, reload, upgrade, heap_frag, types, many_args, binaries, get_string, get_atom, api_macros, from_array, iolist_as_binary, resource, resource_binary, - resource_takeover, threading, send, send2, send_threaded, neg, is_checks, + resource_takeover, threading, send, send2, send3, send_threaded, neg, is_checks, get_length, make_atom, make_string]. %%init_per_testcase(_Case, Config) -> @@ -916,7 +916,164 @@ forwarder(To, N) -> other_term() -> {fun(X,Y) -> X*Y end, make_ref()}. - + +send3(doc) -> ["Message sending stress test"]; +send3(Config) when is_list(Config) -> + %% Let a number of processes send random message blobs between each other + %% using enif_send. Kill and spawn new ones randomly to keep a ~constant + %% number of workers running. + Seed = now(), + io:format("seed: ~p\n",[Seed]), + random:seed(Seed), + ets:new(nif_SUITE,[named_table,public]), + ?line true = ets:insert(nif_SUITE,{send3,0,0,0,0}), + timer:send_after(10000, timeout), % Run for 10 seconds + SpawnCnt = send3_controller(0, [], [], 20), + ?line [{_,Rcv,SndOk,SndFail,Balance}] = ets:lookup(nif_SUITE,send3), + io:format("spawns=~p received=~p, sent=~p send-failure=~p balance=~p\n", + [SpawnCnt,Rcv,SndOk,SndFail,Balance]), + ets:delete(nif_SUITE). + +send3_controller(SpawnCnt, [], _, infinity) -> + SpawnCnt; +send3_controller(SpawnCnt0, Mons0, Pids0, Tick) -> + receive + timeout -> + io:format("Timeout. Sending 'halt' to ~p\n",[Pids0]), + lists:foreach(fun(P) -> P ! {halt,self()} end, Pids0), + lists:foreach(fun(P) -> receive {halted,P} -> ok end end, Pids0), + QTot = lists:foldl(fun(P,QSum) -> + {message_queue_len,QLen} = + erlang:process_info(P,message_queue_len), + QSum + QLen + end, 0, Pids0), + io:format("Total queue length ~p\n",[QTot]), + lists:foreach(fun(P) -> P ! die end, Pids0), + send3_controller(SpawnCnt0, Mons0, [], infinity); + {'DOWN', MonRef, process, _Pid, _} -> + Mons1 = lists:delete(MonRef, Mons0), + %%io:format("Got DOWN from ~p. Monitors left: ~p\n",[Pid,Mons1]), + send3_controller(SpawnCnt0, Mons1, Pids0, Tick) + after Tick -> + Max = 20, + N = length(Pids0), + PidN = random:uniform(Max), + %%io:format("N=~p PidN=~p Pids0=~p\n", [N,PidN,Pids0]), + case PidN > N of + true -> + {NewPid,Mon} = spawn_opt(fun send3_proc/0, [link,monitor]), + lists:foreach(fun(P) -> P ! {is_born,NewPid} end, Pids0), + ?line Balance = ets:lookup_element(nif_SUITE,send3,5), + Inject = (Balance =< 0), + case Inject of + true -> ok; + false -> ets:update_element(nif_SUITE,send3,{5,-1}) + end, + NewPid ! {pids,Pids0,Inject}, + send3_controller(SpawnCnt0+1, [Mon|Mons0], [NewPid|Pids0], Tick); + false -> + KillPid = lists:nth(PidN,Pids0), + KillPid ! die, + Pids1 = lists:delete(KillPid, Pids0), + lists:foreach(fun(P) -> P ! {is_dead,KillPid} end, Pids1), + send3_controller(SpawnCnt0, Mons0, Pids1, Tick) + end + end. + +send3_proc() -> + %%io:format("Process ~p spawned\n",[self()]), + send3_proc([self()], {0,0,0}, {1,2,3,4,5}). +send3_proc(Pids0, Counters={Rcv,SndOk,SndFail}, State0) -> + %%io:format("~p: Pids0=~p", [self(), Pids0]), + %%timer:sleep(10), + receive + {pids, Pids1, Inject} -> + %%io:format("~p: got ~p Inject=~p\n", [self(), Pids1, Inject]), + ?line Pids0 = [self()], + Pids2 = [self() | Pids1], + case Inject of + true -> send3_proc_send(Pids2, Counters, State0); + false -> send3_proc(Pids2, Counters, State0) + end; + {is_born, Pid} -> + %%io:format("~p: is_born ~p, got ~p\n", [self(), Pid, Pids0]), + send3_proc([Pid | Pids0], Counters, State0); + {is_dead, Pid} -> + Pids1 = lists:delete(Pid,Pids0), + %%io:format("~p: is_dead ~p, got ~p\n", [self(), Pid, Pids1]), + send3_proc(Pids1, Counters, State0); + {blob, Blob0} -> + %%io:format("~p: blob ~p\n", [self(), Blob0]), + State1 = send3_new_state(State0, Blob0), + send3_proc_send(Pids0, {Rcv+1,SndOk,SndFail}, State1); + die -> + %%io:format("Process ~p terminating, stats = ~p\n",[self(),Counters]), + {message_queue_len,Dropped} = erlang:process_info(self(),message_queue_len), + _R = ets:update_counter(nif_SUITE,send3, + [{2,Rcv},{3,SndOk},{4,SndFail},{5,1-Dropped}]), + %%io:format("~p: dies R=~p\n", [self(), R]), + ok; + {halt,Papa} -> + Papa ! {halted,self()}, + io:format("~p halted\n",[self()]), + receive die -> ok end, + io:format("~p dying\n",[self()]) + end. + +send3_proc_send(Pids, {Rcv,SndOk,SndFail}, State0) -> + To = lists:nth(random:uniform(length(Pids)),Pids), + Blob = send3_make_blob(), + State1 = send3_new_state(State0,Blob), + case send3_send(To, Blob) of + true -> + send3_proc(Pids, {Rcv,SndOk+1,SndFail}, State1); + false -> + send3_proc(Pids, {Rcv,SndOk,SndFail+1}, State1) + end. + + +send3_make_blob() -> + case random:uniform(20)-1 of + 0 -> {term,[]}; + N -> + MsgEnv = alloc_msgenv(), + repeat(N bsr 1, + fun(_) -> grow_blob(MsgEnv,other_term(),random:uniform(1 bsl 20)) + end, void), + case (N band 1) of + 0 -> {term,copy_blob(MsgEnv)}; + 1 -> {msgenv,MsgEnv} + end + end. + +send3_send(Pid, Msg) -> + %% 90% enif_send and 10% normal bang + case random:uniform(10) of + 1 -> send3_send_bang(Pid,Msg); + _ -> send3_send_nif(Pid,Msg) + end. +send3_send_nif(Pid, {term,Blob}) -> + %%io:format("~p send term nif\n",[self()]), + send_term(Pid, {blob, Blob}) =:= 1; +send3_send_nif(Pid, {msgenv,MsgEnv}) -> + %%io:format("~p send blob nif\n",[self()]), + send3_blob(MsgEnv, Pid, blob) =:= 1. + +send3_send_bang(Pid, {term,Blob}) -> + %%io:format("~p send term bang\n",[self()]), + Pid ! {blob, Blob}, + true; +send3_send_bang(Pid, {msgenv,MsgEnv}) -> + %%io:format("~p send blob bang\n",[self()]), + Pid ! {blob, copy_blob(MsgEnv)}, + true. + +send3_new_state(State, Blob) -> + case random:uniform(5+2) of + N when N =< 5-> setelement(N, State, Blob); + _ -> State % Don't store blob + end. + neg(doc) -> ["Negative testing of load_nif"]; neg(Config) when is_list(Config) -> TmpMem = tmpmem(), @@ -1070,10 +1227,13 @@ send_new_blob(_,_) -> ?nif_stub. alloc_msgenv() -> ?nif_stub. clear_msgenv(_) -> ?nif_stub. grow_blob(_,_) -> ?nif_stub. +grow_blob(_,_,_) -> ?nif_stub. send_blob(_,_) -> ?nif_stub. +send3_blob(_,_,_) -> ?nif_stub. send_blob_thread(_,_,_) -> ?nif_stub. join_send_thread(_) -> ?nif_stub. - +copy_blob(_) -> ?nif_stub. +send_term(_,_) -> ?nif_stub. nif_stub_error(Line) -> exit({nif_not_loaded,module,?MODULE,line,Line}). diff --git a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c index c8cd323b7e..17f644829f 100644 --- a/erts/emulator/test/nif_SUITE_data/nif_SUITE.c +++ b/erts/emulator/test/nif_SUITE_data/nif_SUITE.c @@ -1038,36 +1038,41 @@ static ERL_NIF_TERM make_term_copy(struct make_term_info* mti, int n) { return enif_make_copy(mti->dst_env, mti->other_term); } + +typedef ERL_NIF_TERM Make_term_Func(struct make_term_info*, int); +static Make_term_Func* make_funcs[] = { + make_term_binary, + make_term_int, + make_term_ulong, + make_term_double, + make_term_atom, + make_term_existing_atom, + make_term_string, + //make_term_ref, + make_term_sub_binary, + make_term_uint, + make_term_long, + make_term_tuple0, + make_term_list0, + make_term_resource, + make_term_new_binary, + make_term_caller_pid, + make_term_tuple, + make_term_list, + make_term_list_cell, + make_term_tuple_from_array, + make_term_list_from_array, + make_term_garbage, + make_term_copy +}; +static unsigned num_of_make_funcs() +{ + return sizeof(make_funcs)/sizeof(*make_funcs); +} static int make_term_n(struct make_term_info* mti, int n, ERL_NIF_TERM* res) { - typedef ERL_NIF_TERM Make_term_Func(struct make_term_info*, int); - static Make_term_Func* funcs[] = { - make_term_binary, - make_term_int, - make_term_ulong, - make_term_double, - make_term_atom, - make_term_existing_atom, - make_term_string, - //make_term_ref, - make_term_sub_binary, - make_term_uint, - make_term_long, - make_term_tuple0, - make_term_list0, - make_term_resource, - make_term_new_binary, - make_term_caller_pid, - make_term_tuple, - make_term_list, - make_term_list_cell, - make_term_tuple_from_array, - make_term_list_from_array, - make_term_garbage, - make_term_copy - }; - if (n < sizeof(funcs)/sizeof(*funcs)) { - *res = funcs[n](mti, n); + if (n < num_of_make_funcs()) { + *res = make_funcs[n](mti, n); push_term(mti, *res); return 1; } @@ -1167,14 +1172,14 @@ static ERL_NIF_TERM grow_blob(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[ { union { void* vp; struct make_term_info* p; }mti; ERL_NIF_TERM term; - if (!enif_get_resource(env, argv[0], msgenv_resource_type, &mti.vp)) { + if (!enif_get_resource(env, argv[0], msgenv_resource_type, &mti.vp) + || (argc>2 && !enif_get_uint(env,argv[2], &mti.p->n))) { return enif_make_badarg(env); } mti.p->caller_env = env; mti.p->other_term = argv[1]; - while (!make_term_n(mti.p, mti.p->n++, &term)) { - mti.p->n = 0; - } + mti.p->n %= num_of_make_funcs(); + make_term_n(mti.p, mti.p->n++, &term); mti.p->blob = enif_make_list_cell(mti.p->dst_env, term, mti.p->blob); return atom_ok; } @@ -1194,6 +1199,23 @@ static ERL_NIF_TERM send_blob(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[ return enif_make_tuple3(env, atom_ok, enif_make_int(env,res), copy); } +static ERL_NIF_TERM send3_blob(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + union { void* vp; struct make_term_info* p; }mti; + ErlNifPid to; + ERL_NIF_TERM copy; + int res; + if (!enif_get_resource(env, argv[0], msgenv_resource_type, &mti.vp) + || !enif_get_local_pid(env, argv[1], &to)) { + return enif_make_badarg(env); + } + mti.p->blob = enif_make_tuple2(mti.p->dst_env, + enif_make_copy(mti.p->dst_env, argv[2]), + mti.p->blob); + res = enif_send(env, &to, mti.p->dst_env, mti.p->blob); + return enif_make_int(env,res); +} + void* threaded_sender(void *arg) { @@ -1253,6 +1275,28 @@ static ERL_NIF_TERM join_send_thread(ErlNifEnv* env, int argc, const ERL_NIF_TER return enif_make_tuple2(env, atom_ok, enif_make_int(env, mti.p->send_res)); } +static ERL_NIF_TERM copy_blob(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + union { void* vp; struct make_term_info* p; }mti; + if (!enif_get_resource(env, argv[0], msgenv_resource_type, &mti.vp)) { + return enif_make_badarg(env); + } + return enif_make_copy(env, mti.p->blob); +} + +static ERL_NIF_TERM send_term(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) +{ + ErlNifEnv* menv; + ErlNifPid pid; + int ret; + if (!enif_get_local_pid(env, argv[0], &pid)) { + return enif_make_badarg(env); + } + menv = enif_alloc_env(); + ret = enif_send(env, &pid, menv, enif_make_copy(menv, argv[1])); + enif_free_env(menv); + return enif_make_int(env, ret); +} static ErlNifFunc nif_funcs[] = { @@ -1291,9 +1335,13 @@ static ErlNifFunc nif_funcs[] = {"alloc_msgenv", 0, alloc_msgenv}, {"clear_msgenv", 1, clear_msgenv}, {"grow_blob", 2, grow_blob}, + {"grow_blob", 3, grow_blob}, {"send_blob", 2, send_blob}, + {"send3_blob", 3, send3_blob}, {"send_blob_thread", 3, send_blob_thread}, - {"join_send_thread", 1, join_send_thread} + {"join_send_thread", 1, join_send_thread}, + {"copy_blob", 1, copy_blob}, + {"send_term", 2, send_term} }; ERL_NIF_INIT(nif_SUITE,nif_funcs,load,reload,upgrade,unload) diff --git a/erts/emulator/test/port_SUITE.erl b/erts/emulator/test/port_SUITE.erl index 66aff307a3..77fa75b78f 100644 --- a/erts/emulator/test/port_SUITE.erl +++ b/erts/emulator/test/port_SUITE.erl @@ -878,13 +878,20 @@ env2(Config) -> "nisse" = os:getenv(Long) end), - + ?line env_slave(Temp, [{"must_define_something","some_value"}, - {"certainly_not_existing",false}, + {"certainly_not_existing",false}, {"ends_with_equal", "value="}, {Long,false}, {"glurf","a glorfy string"}]), + %% A lot of non existing variables (mingled with existing) + NotExistingList = [{lists:flatten(io_lib:format("V~p_not_existing",[X])),false} + || X <- lists:seq(1,150)], + ExistingList = [{lists:flatten(io_lib:format("V~p_existing",[X])),"a_value"} + || X <- lists:seq(1,150)], + ?line env_slave(Temp, lists:sort(ExistingList ++ NotExistingList)), + ?line test_server:timetrap_cancel(Dog), ok. diff --git a/erts/emulator/test/send_term_SUITE.erl b/erts/emulator/test/send_term_SUITE.erl index 489adbd660..5fd01a9ac5 100644 --- a/erts/emulator/test/send_term_SUITE.erl +++ b/erts/emulator/test/send_term_SUITE.erl @@ -76,40 +76,43 @@ basic(Config) when is_list(Config) -> ?line ExpectedBinTup = term(P, 7), %% single terms - ?line [] = term(P, 8), % ERL_DRV_NIL - ?line '' = term(P, 9), % ERL_DRV_ATOM - ?line an_atom = term(P, 10), % ERL_DRV_ATOM - ?line -4711 = term(P, 11), % ERL_DRV_INT - ?line 4711 = term(P, 12), % ERL_DRV_UINT - ?line P = term(P, 13), % ERL_DRV_PORT - ?line <<>> = term(P, 14), % ERL_DRV_BINARY - ?line <<"hejsan">> = term(P, 15), % ERL_DRV_BINARY - ?line <<>> = term(P, 16), % ERL_DRV_BUF2BINARY - ?line <<>> = term(P, 17), % ERL_DRV_BUF2BINARY - ?line <<"hoppsan">> = term(P, 18), % ERL_DRV_BUF2BINARY - ?line "" = term(P, 19), % ERL_DRV_STRING - ?line "" = term(P, 20), % ERL_DRV_STRING - ?line "hippsan" = term(P, 21), % ERL_DRV_STRING - ?line {} = term(P, 22), % ERL_DRV_TUPLE - ?line [] = term(P, 23), % ERL_DRV_LIST - ?line Self = term(P, 24), % ERL_DRV_PID - ?line [] = term(P, 25), % ERL_DRV_STRING_CONS - ?line AFloat = term(P, 26), % ERL_DRV_FLOAT + Singles = [{[], 8}, % ERL_DRV_NIL + {'', 9}, % ERL_DRV_ATOM + {an_atom, 10}, % ERL_DRV_ATOM + {-4711, 11}, % ERL_DRV_INT + {4711, 12}, % ERL_DRV_UINT + {P, 13}, % ERL_DRV_PORT + {<<>>, 14}, % ERL_DRV_BINARY + {<<"hejsan">>, 15}, % ERL_DRV_BINARY + {<<>>, 16}, % ERL_DRV_BUF2BINARY + {<<>>, 17}, % ERL_DRV_BUF2BINARY + {<<"hoppsan">>, 18}, % ERL_DRV_BUF2BINARY + {"", 19}, % ERL_DRV_STRING + {"", 20}, % ERL_DRV_STRING + {"hippsan", 21}, % ERL_DRV_STRING + {{}, 22}, % ERL_DRV_TUPLE + {[], 23}, % ERL_DRV_LIST + {Self, 24}, % ERL_DRV_PID + {[], 25}, % ERL_DRV_STRING_CONS + {[], 27}, % ERL_DRV_EXT2TERM + {18446744073709551615, 28}, % ERL_DRV_UINT64 + {20233590931456, 29}, % ERL_DRV_UINT64 + {4711, 30}, % ERL_DRV_UINT64 + {0, 31}, % ERL_DRV_UINT64 + {9223372036854775807, 32}, % ERL_DRV_INT64 + {20233590931456, 33}, % ERL_DRV_INT64 + {4711, 34}, % ERL_DRV_INT64 + {0, 35}, % ERL_DRV_INT64 + {-1, 36}, % ERL_DRV_INT64 + {-4711, 37}, % ERL_DRV_INT64 + {-20233590931456, 38}, % ERL_DRV_INT64 + {-9223372036854775808, 39}], % ERL_DRV_INT64 + ?line {Terms, Ops} = lists:unzip(Singles), + ?line Terms = term(P,Ops), + + AFloat = term(P, 26), % ERL_DRV_FLOAT ?line true = AFloat < 0.001, ?line true = AFloat > -0.001, - ?line [] = term(P, 27), % ERL_DRV_EXT2TERM - ?line 18446744073709551615 = term(P, 28), % ERL_DRV_UINT64 - ?line 20233590931456 = term(P, 29), % ERL_DRV_UINT64 - ?line 4711 = term(P, 30), % ERL_DRV_UINT64 - ?line 0 = term(P, 31), % ERL_DRV_UINT64 - ?line 9223372036854775807 = term(P, 32), % ERL_DRV_INT64 - ?line 20233590931456 = term(P, 33), % ERL_DRV_INT64 - ?line 4711 = term(P, 34), % ERL_DRV_INT64 - ?line 0 = term(P, 35), % ERL_DRV_INT64 - ?line -1 = term(P, 36), % ERL_DRV_INT64 - ?line -4711 = term(P, 37), % ERL_DRV_INT64 - ?line -20233590931456 = term(P, 38), % ERL_DRV_INT64 - ?line -9223372036854775808 = term(P, 39), % ERL_DRV_INT64 %% Failure cases. ?line [] = term(P, 127), diff --git a/erts/emulator/test/send_term_SUITE_data/send_term_drv.c b/erts/emulator/test/send_term_SUITE_data/send_term_drv.c index 6638de0560..165cce2e9d 100644 --- a/erts/emulator/test/send_term_SUITE_data/send_term_drv.c +++ b/erts/emulator/test/send_term_SUITE_data/send_term_drv.c @@ -17,6 +17,7 @@ */ #include "erl_driver.h" +#include <stdio.h> #include <errno.h> #include <string.h> @@ -65,12 +66,21 @@ static void fail_term(ErlDrvTermData* msg, int len, int line); static void send_term_drv_run(ErlDrvData port, char *buf, int count) { - ErlDrvTermData msg[1024]; - - switch (*buf) { + char buf7[1024]; + ErlDrvTermData spec[1024]; + ErlDrvTermData* msg = spec; + ErlDrvBinary* bins[15]; + int bin_ix = 0; + ErlDrvSInt64 s64[15]; + int s64_ix = 0; + ErlDrvUInt64 u64[15]; + int u64_ix = 0; + int i = 0; + + for (i=0; i<count; i++) switch (buf[i]) { case 0: msg[0] = ERL_DRV_NIL; - output_term(msg, 1); + msg += 1; break; case 1: /* Most term types inside a tuple. */ @@ -102,7 +112,7 @@ static void send_term_drv_run(ErlDrvData port, char *buf, int count) msg[22] = driver_connected(erlang_port); msg[23] = ERL_DRV_TUPLE; msg[24] = (ErlDrvTermData) 7; - output_term(msg, 25); + msg += 25; } break; @@ -117,7 +127,7 @@ static void send_term_drv_run(ErlDrvData port, char *buf, int count) msg[i] = ERL_DRV_NIL; msg[i+1] = ERL_DRV_LIST; msg[i+2] = (ErlDrvTermData) 201; - output_term(msg, i+3); + msg += i+3; } break; @@ -126,7 +136,7 @@ static void send_term_drv_run(ErlDrvData port, char *buf, int count) ErlDrvBinary* bin; int i; - bin = driver_alloc_binary(256); + bin = bins[bin_ix++] = driver_alloc_binary(256); for (i = 0; i < 256; i++) { bin->orig_bytes[i] = i; } @@ -140,8 +150,7 @@ static void send_term_drv_run(ErlDrvData port, char *buf, int count) msg[7] = (ErlDrvTermData) 23; msg[8] = ERL_DRV_TUPLE; msg[9] = (ErlDrvTermData) 2; - output_term(msg, 10); - driver_free_binary(bin); + msg += 10; } break; @@ -152,11 +161,11 @@ static void send_term_drv_run(ErlDrvData port, char *buf, int count) msg[3] = driver_caller(erlang_port); msg[4] = ERL_DRV_TUPLE; msg[5] = (ErlDrvTermData) 2; - output_term(msg, 6); + msg += 6; break; case 5: - output_term(msg, make_ext_term_list(msg, 0)); + msg += make_ext_term_list(msg, 0); break; case 6: @@ -166,94 +175,91 @@ static void send_term_drv_run(ErlDrvData port, char *buf, int count) msg[3] = ~((ErlDrvTermData) 0); msg[4] = ERL_DRV_TUPLE; msg[5] = (ErlDrvTermData) 2; - output_term(msg, 6); + msg += 6; break; case 7: { int len = 0; - char buf[1024]; - memset(buf, 17, sizeof(buf)); + memset(buf7, 17, sizeof(buf7)); /* empty heap binary */ msg[len++] = ERL_DRV_BUF2BINARY; msg[len++] = (ErlDrvTermData) NULL; /* NULL is ok if size == 0 */ msg[len++] = (ErlDrvTermData) 0; /* empty heap binary again */ msg[len++] = ERL_DRV_BUF2BINARY; - msg[len++] = (ErlDrvTermData) &buf[0]; /* ptr is ok if size == 0 */ + msg[len++] = (ErlDrvTermData) buf7; /* ptr is ok if size == 0 */ msg[len++] = (ErlDrvTermData) 0; /* heap binary */ msg[len++] = ERL_DRV_BUF2BINARY; - msg[len++] = (ErlDrvTermData) &buf[0]; + msg[len++] = (ErlDrvTermData) buf7; msg[len++] = (ErlDrvTermData) 17; /* off heap binary */ msg[len++] = ERL_DRV_BUF2BINARY; - msg[len++] = (ErlDrvTermData) &buf[0]; - msg[len++] = (ErlDrvTermData) sizeof(buf); + msg[len++] = (ErlDrvTermData) buf7; + msg[len++] = (ErlDrvTermData) sizeof(buf7); msg[len++] = ERL_DRV_TUPLE; msg[len++] = (ErlDrvTermData) 4; - output_term(msg, len); + msg += len; break; } case 8: msg[0] = ERL_DRV_NIL; - output_term(msg, 1); + msg += 1; break; case 9: msg[0] = ERL_DRV_ATOM; msg[1] = (ErlDrvTermData) driver_mk_atom(""); - output_term(msg, 2); + msg += 2; break; case 10: msg[0] = ERL_DRV_ATOM; msg[1] = (ErlDrvTermData) driver_mk_atom("an_atom"); - output_term(msg, 2); + msg += 2; break; case 11: msg[0] = ERL_DRV_INT; msg[1] = (ErlDrvTermData) -4711; - output_term(msg, 2); + msg += 2; break; case 12: msg[0] = ERL_DRV_UINT; msg[1] = (ErlDrvTermData) 4711; - output_term(msg, 2); + msg += 2; break; case 13: msg[0] = ERL_DRV_PORT; msg[1] = driver_mk_port(erlang_port); - output_term(msg, 2); + msg += 2; break; case 14: { - ErlDrvBinary *dbin = driver_alloc_binary(0); + ErlDrvBinary *dbin = bins[bin_ix++] = driver_alloc_binary(0); msg[0] = ERL_DRV_BINARY; msg[1] = (ErlDrvTermData) dbin; msg[2] = (ErlDrvTermData) 0; msg[3] = (ErlDrvTermData) 0; - output_term(msg, 4); - driver_free_binary(dbin); + msg += 4; break; } case 15: { - char buf[] = "hejsan"; - ErlDrvBinary *dbin = driver_alloc_binary(sizeof(buf)-1); + static const char buf[] = "hejsan"; + ErlDrvBinary *dbin = bins[bin_ix++] = driver_alloc_binary(sizeof(buf)-1); if (dbin) memcpy((void *) dbin->orig_bytes, (void *) buf, sizeof(buf)-1); msg[0] = ERL_DRV_BINARY; msg[1] = (ErlDrvTermData) dbin; msg[2] = (ErlDrvTermData) (dbin ? sizeof(buf)-1 : 0); msg[3] = (ErlDrvTermData) 0; - output_term(msg, 4); - driver_free_binary(dbin); + msg += 4; break; } @@ -261,24 +267,24 @@ static void send_term_drv_run(ErlDrvData port, char *buf, int count) msg[0] = ERL_DRV_BUF2BINARY; msg[1] = (ErlDrvTermData) NULL; msg[2] = (ErlDrvTermData) 0; - output_term(msg, 3); + msg += 3; break; case 17: { - char buf[] = ""; + static const char buf[] = ""; msg[0] = ERL_DRV_BUF2BINARY; msg[1] = (ErlDrvTermData) buf; msg[2] = (ErlDrvTermData) sizeof(buf)-1; - output_term(msg, 3); + msg += 3; break; } case 18: { - char buf[] = "hoppsan"; + static const char buf[] = "hoppsan"; msg[0] = ERL_DRV_BUF2BINARY; msg[1] = (ErlDrvTermData) buf; msg[2] = (ErlDrvTermData) sizeof(buf)-1; - output_term(msg, 3); + msg += 3; break; } @@ -286,44 +292,44 @@ static void send_term_drv_run(ErlDrvData port, char *buf, int count) msg[0] = ERL_DRV_STRING; msg[1] = (ErlDrvTermData) buf; msg[2] = (ErlDrvTermData) 0; - output_term(msg, 3); + msg += 3; break; case 20: { - char buf[] = ""; + static const char buf[] = ""; msg[0] = ERL_DRV_STRING; msg[1] = (ErlDrvTermData) buf; msg[2] = (ErlDrvTermData) sizeof(buf)-1; - output_term(msg, 3); + msg += 3; break; } case 21: { - char buf[] = "hippsan"; + static const char buf[] = "hippsan"; msg[0] = ERL_DRV_STRING; msg[1] = (ErlDrvTermData) buf; msg[2] = (ErlDrvTermData) sizeof(buf)-1; - output_term(msg, 3); + msg += 3; break; } case 22: msg[0] = ERL_DRV_TUPLE; msg[1] = (ErlDrvTermData) 0; - output_term(msg, 2); + msg += 2; break; case 23: msg[0] = ERL_DRV_NIL; msg[1] = ERL_DRV_LIST; msg[2] = (ErlDrvTermData) 1; - output_term(msg, 3); + msg += 3; break; case 24: msg[0] = ERL_DRV_PID; msg[1] = driver_connected(erlang_port); - output_term(msg, 2); + msg += 2; break; case 25: @@ -331,132 +337,131 @@ static void send_term_drv_run(ErlDrvData port, char *buf, int count) msg[1] = ERL_DRV_STRING_CONS; msg[2] = (ErlDrvTermData) ""; msg[3] = (ErlDrvTermData) 0; - output_term(msg, 4); + msg += 4; break; case 26: { - double my_float = 0.0; + static double my_float = 0.0; msg[0] = ERL_DRV_FLOAT; msg[1] = (ErlDrvTermData) &my_float; - output_term(msg, 2); + msg += 2; break; } case 27: { - char buf[] = {131, 106}; /* [] */ + static char buf[] = {131, 106}; /* [] */ msg[0] = ERL_DRV_EXT2TERM; msg[1] = (ErlDrvTermData) buf; msg[2] = (ErlDrvTermData) sizeof(buf); - output_term(msg, 3); + msg += 3; break; } case 28: { - ErlDrvUInt64 x = ~((ErlDrvUInt64) 0); + ErlDrvUInt64* x = &u64[u64_ix++]; + *x = ~((ErlDrvUInt64) 0); msg[0] = ERL_DRV_UINT64; - msg[1] = (ErlDrvTermData) &x; - output_term(msg, 2); - + msg[1] = (ErlDrvTermData) x; + msg += 2; break; } case 29: { - ErlDrvUInt64 x = ((ErlDrvUInt64) 4711) << 32; + ErlDrvUInt64* x = &u64[u64_ix++]; + *x = ((ErlDrvUInt64) 4711) << 32; msg[0] = ERL_DRV_UINT64; - msg[1] = (ErlDrvTermData) &x; - output_term(msg, 2); - + msg[1] = (ErlDrvTermData) x; + msg += 2; break; } case 30: { - ErlDrvUInt64 x = 4711; + ErlDrvUInt64* x = &u64[u64_ix++]; + *x = 4711; msg[0] = ERL_DRV_UINT64; - msg[1] = (ErlDrvTermData) &x; - output_term(msg, 2); - + msg[1] = (ErlDrvTermData) x; + msg += 2; break; } case 31: { - ErlDrvUInt64 x = 0; + ErlDrvUInt64* x = &u64[u64_ix++]; + *x = 0; msg[0] = ERL_DRV_UINT64; - msg[1] = (ErlDrvTermData) &x; - output_term(msg, 2); - + msg[1] = (ErlDrvTermData) x; + msg += 2; break; } case 32: { - ErlDrvSInt64 x = ((((ErlDrvUInt64) 0x7fffffff) << 32) - | ((ErlDrvUInt64) 0xffffffff)); + ErlDrvSInt64* x = &s64[s64_ix++]; + *x = ((((ErlDrvUInt64) 0x7fffffff) << 32) | ((ErlDrvUInt64) 0xffffffff)); msg[0] = ERL_DRV_INT64; - msg[1] = (ErlDrvTermData) &x; - output_term(msg, 2); - + msg[1] = (ErlDrvTermData) x; + msg += 2; break; } case 33: { - ErlDrvSInt64 x = (ErlDrvSInt64) (((ErlDrvUInt64) 4711) << 32); + ErlDrvSInt64* x = &s64[s64_ix++]; + *x = (ErlDrvSInt64) (((ErlDrvUInt64) 4711) << 32); msg[0] = ERL_DRV_INT64; - msg[1] = (ErlDrvTermData) &x; - output_term(msg, 2); - + msg[1] = (ErlDrvTermData) x; + msg += 2; break; } case 34: { - ErlDrvSInt64 x = 4711; + ErlDrvSInt64* x = &s64[s64_ix++]; + *x = 4711; msg[0] = ERL_DRV_INT64; - msg[1] = (ErlDrvTermData) &x; - output_term(msg, 2); - + msg[1] = (ErlDrvTermData) x; + msg += 2; break; } case 35: { - ErlDrvSInt64 x = 0; + ErlDrvSInt64* x = &s64[s64_ix++]; + *x = 0; msg[0] = ERL_DRV_INT64; - msg[1] = (ErlDrvTermData) &x; - output_term(msg, 2); - + msg[1] = (ErlDrvTermData) x; + msg += 2; break; } case 36: { - ErlDrvSInt64 x = -1; + ErlDrvSInt64* x = &s64[s64_ix++]; + *x = -1; msg[0] = ERL_DRV_INT64; - msg[1] = (ErlDrvTermData) &x; - output_term(msg, 2); - + msg[1] = (ErlDrvTermData) x; + msg += 2; break; } case 37: { - ErlDrvSInt64 x = -4711; + ErlDrvSInt64* x = &s64[s64_ix++]; + *x = -4711; msg[0] = ERL_DRV_INT64; - msg[1] = (ErlDrvTermData) &x; - output_term(msg, 2); - + msg[1] = (ErlDrvTermData) x; + msg += 2; break; } case 38: { - ErlDrvSInt64 x = ((ErlDrvSInt64) ((ErlDrvUInt64) 4711) << 32)*-1; + ErlDrvSInt64* x = &s64[s64_ix++]; + *x = ((ErlDrvSInt64) ((ErlDrvUInt64) 4711) << 32)*-1; msg[0] = ERL_DRV_INT64; - msg[1] = (ErlDrvTermData) &x; - output_term(msg, 2); - + msg[1] = (ErlDrvTermData) x; + msg += 2; break; } case 39: { - ErlDrvSInt64 x = ((ErlDrvSInt64) 1) << 63; + ErlDrvSInt64* x = &s64[s64_ix++]; + *x = ((ErlDrvSInt64) 1) << 63; msg[0] = ERL_DRV_INT64; - msg[1] = (ErlDrvTermData) &x; - output_term(msg, 2); - + msg[1] = (ErlDrvTermData) x; + msg += 2; break; } @@ -464,7 +469,7 @@ static void send_term_drv_run(ErlDrvData port, char *buf, int count) case 127: /* Error cases */ { long refc; - ErlDrvBinary* bin = driver_alloc_binary(256); + ErlDrvBinary* bin = bins[bin_ix++] = driver_alloc_binary(256); FAIL_TERM(msg, 0); @@ -537,7 +542,7 @@ static void send_term_drv_run(ErlDrvData port, char *buf, int count) refc = driver_binary_get_refc(bin); if (refc > 3) { char sbuf[128]; - sprintf(sbuf, "bad_refc:%d", refc); + sprintf(sbuf, "bad_refc:%ld", refc); driver_failure_atom(erlang_port, sbuf); } driver_free_binary(bin); @@ -644,6 +649,7 @@ static void send_term_drv_run(ErlDrvData port, char *buf, int count) /* Signal end of test case */ msg[0] = ERL_DRV_NIL; driver_output_term(erlang_port, msg, 1); + return; } break; @@ -651,6 +657,16 @@ static void send_term_drv_run(ErlDrvData port, char *buf, int count) driver_failure_atom(erlang_port, "bad_request"); break; } + if (count > 1) { + *msg++ = ERL_DRV_NIL; + *msg++ = ERL_DRV_LIST; + *msg++ = count + 1; + } + output_term(spec, msg-spec); + if ((bin_ix|s64_ix|u64_ix) > 15) abort(); + while (bin_ix) { + driver_free_binary(bins[--bin_ix]); + } } static void output_term(ErlDrvTermData* msg, int len) diff --git a/erts/epmd/src/epmd.c b/erts/epmd/src/epmd.c index c509c49b39..6ddf30efe3 100644 --- a/erts/epmd/src/epmd.c +++ b/erts/epmd/src/epmd.c @@ -226,6 +226,12 @@ int main(int argc, char** argv) else usage(g); epmd_cleanup_exit(g,0); + } else if (strcmp(argv[0], "-stop") == 0) { + if (argc == 2) + stop_cli(g, argv[1]); + else + usage(g); + epmd_cleanup_exit(g,0); } else usage(g); @@ -382,7 +388,7 @@ static void run_daemon(EpmdVars *g) static void usage(EpmdVars *g) { fprintf(stderr, "usage: epmd [-d|-debug] [DbgExtra...] [-port No] [-daemon]\n"); - fprintf(stderr, " [-d|-debug] [-port No] [-names|-kill]\n\n"); + fprintf(stderr, " [-d|-debug] [-port No] [-names|-kill|-stop name]\n\n"); fprintf(stderr, "See the Erlang epmd manual page for info about the usage.\n"); fprintf(stderr, "The -port and DbgExtra options are\n\n"); fprintf(stderr, " -port No\n"); diff --git a/erts/epmd/src/epmd_cli.c b/erts/epmd/src/epmd_cli.c index c12f711bc5..2aed861390 100644 --- a/erts/epmd/src/epmd_cli.c +++ b/erts/epmd/src/epmd_cli.c @@ -54,6 +54,42 @@ void kill_epmd(EpmdVars *g) } } +void stop_cli(EpmdVars *g, char *name) +{ + char buf[1024]; + int fd, rval, bsize; + + bsize = strlen(name); + if (bsize > 1000) { + printf("epmd: Name too long!"); + epmd_cleanup_exit(g, 1); + } + + fd = conn_to_epmd(g); + bsize++; + put_int16(bsize, buf); + buf[2] = EPMD_STOP_REQ; + bsize += 2; + strcpy(buf+3, name); + + if (write(fd, buf, bsize) != bsize) { + printf("epmd: Can't write to epmd\n"); + epmd_cleanup_exit(g,1); + } + if ((rval = read_fill(fd,buf,7)) == 7) { + buf[7] = '\000'; + printf("%s\n", buf); + epmd_cleanup_exit(g,0); + } else if (rval < 0) { + printf("epmd: failed to read answer from local epmd\n"); + epmd_cleanup_exit(g,1); + } else { /* rval is now 0 or 1 */ + buf[rval] = '\0'; + printf("epmd: local epmd responded with <%s>\n", buf); + epmd_cleanup_exit(g,1); + } +} + /* what == EPMD_NAMES_REQ || EPMD_DUMP_REQ */ void epmd_call(EpmdVars *g,int what) diff --git a/erts/epmd/src/epmd_srv.c b/erts/epmd/src/epmd_srv.c index 34f657fb16..c836bf0bb7 100644 --- a/erts/epmd/src/epmd_srv.c +++ b/erts/epmd/src/epmd_srv.c @@ -591,7 +591,7 @@ static void do_request(g, fd, s, buf, bsize) if (bsize <= 1) { - dbg_printf(g,0,"packet to small for request PORT2_REQ (%d)", bsize); + dbg_printf(g,0,"packet too small for request PORT2_REQ (%d)", bsize); return; } @@ -740,7 +740,7 @@ static void do_request(g, fd, s, buf, bsize) dbg_printf(g,1,"** got STOP_REQ"); if (bsize <= 1 ) { - dbg_printf(g,0,"packet to small for request STOP_REQ (%d)",bsize); + dbg_printf(g,0,"packet too small for request STOP_REQ (%d)",bsize); return; } @@ -902,7 +902,7 @@ static void node_init(EpmdVars *g) /* We have got a close on a connection and it may be a - EPMD_ALIVE_CLOSE_REQ. Note that this call shouild be called + EPMD_ALIVE_CLOSE_REQ. Note that this call should be called *before* calling conn_close() */ static int node_unreg(EpmdVars *g,char *name) diff --git a/erts/etc/common/Makefile.in b/erts/etc/common/Makefile.in index d2a5080c68..6551b2999a 100644 --- a/erts/etc/common/Makefile.in +++ b/erts/etc/common/Makefile.in @@ -178,7 +178,7 @@ MC_OUTPUTS= \ MT_FLAG="-MD" endif INET_GETHOST = $(BINDIR)/inet_gethost.exe -INSTALL_EMBEDDED_PROGS += $(BINDIR)/typer.exe $(BINDIR)/dialyzer.exe $(BINDIR)/erlc.exe $(BINDIR)/start_erl.exe $(BINDIR)/escript.exe +INSTALL_EMBEDDED_PROGS += $(BINDIR)/typer.exe $(BINDIR)/dialyzer.exe $(BINDIR)/erlc.exe $(BINDIR)/start_erl.exe $(BINDIR)/escript.exe $(BINDIR)/run_test.exe INSTALL_SRC = $(WINETC)/start_erl.c $(WINETC)/Nmakefile.start_erl ERLEXECDIR=. INSTALL_LIBS = @@ -211,7 +211,7 @@ ERLSRV_OBJECTS= MC_OUTPUTS= INET_GETHOST = $(BINDIR)/inet_gethost@EXEEXT@ INSTALL_EMBEDDED_PROGS += $(BINDIR)/typer@EXEEXT@ $(BINDIR)/dialyzer@EXEEXT@ \ - $(BINDIR)/erlc@EXEEXT@ $(BINDIR)/escript@EXEEXT@ \ + $(BINDIR)/erlc@EXEEXT@ $(BINDIR)/escript@EXEEXT@ $(BINDIR)/run_test@EXEEXT@ \ $(BINDIR)/run_erl $(BINDIR)/to_erl $(BINDIR)/dyn_erl INSTALL_EMBEDDED_DATA = ../unix/start.src ../unix/start_erl.src INSTALL_TOP = Install @@ -274,6 +274,7 @@ endif rm -f $(ERL_TOP)/erts/obj*/$(TARGET)/dyn_erl.o rm -f $(ERL_TOP)/erts/obj*/$(TARGET)/safe_string.o rm -f $(ERL_TOP)/erts/obj*/$(TARGET)/typer.o + rm -f $(ERL_TOP)/erts/obj*/$(TARGET)/run_test.o rm -f $(ERL_TOP)/erts/obj*/$(TARGET)/vxcall.o rm -f $(ERL_TOP)/erts/obj*/$(TARGET)/erl.o rm -f $(ERL_TOP)/erts/obj*/$(TARGET)/werl.o @@ -349,6 +350,13 @@ $(BINDIR)/escript@EXEEXT@: $(OBJDIR)/escript.o $(OBJDIR)/escript.o: escript.c $(CC) $(CFLAGS) -o $@ -c escript.c +$(BINDIR)/run_test@EXEEXT@: $(OBJDIR)/run_test.o + $(PURIFY) $(LD) $(LDFLAGS) -o $@ $(OBJDIR)/run_test.o -L$(OBJDIR) $(LIBS) + +$(OBJDIR)/run_test.o: run_test.c + $(CC) $(CFLAGS) -o $@ -c run_test.c + + #------------------------------------------------------------------------ # Windows specific targets # The windows platform is quite different from the others. erl/werl are small C programs diff --git a/erts/etc/common/run_test.c b/erts/etc/common/run_test.c new file mode 100644 index 0000000000..016d9c6afd --- /dev/null +++ b/erts/etc/common/run_test.c @@ -0,0 +1,514 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2010. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ +/* + * Purpose: Common Test front-end. + */ +#ifdef HAVE_CONFIG_H +# include "config.h" +#endif + +#include "sys.h" +#ifdef __WIN32__ +#include <winbase.h> +#endif + +#include <ctype.h> + +#define NO 0 +#define YES 1 + +#define ASIZE(a) (sizeof(a)/sizeof(a[0])) + +static int debug = 0; /* Bit flags for debug printouts. */ + +static char** eargv_base; /* Base of vector. */ +static char** eargv; /* First argument for erl. */ + +static int eargc; /* Number of arguments in eargv. */ + +#ifdef __WIN32__ +# define QUOTE(s) possibly_quote(s) +# define IS_DIRSEP(c) ((c) == '/' || (c) == '\\') +# define ERL_NAME "erl.exe" +#else +# define QUOTE(s) s +# define IS_DIRSEP(c) ((c) == '/') +# define ERL_NAME "erl" +#endif + +#define UNSHIFT(s) eargc++, eargv--; eargv[0] = QUOTE(s) +#define PUSH(s) eargv[eargc++] = QUOTE(s) +#define PUSH2(s, t) PUSH(s); PUSH(t) +#define PUSH3(s, t, u) PUSH2(s, t); PUSH(u) +#define PUSH4(s, t, u, v) PUSH2(s, t); PUSH2(u, v) + +/* + * The possible modes to start Common Test + */ + +#define NORMAL_MODE 0 +#define VTS_MODE 1 +#define CT_SHELL_MODE 2 +#define MASTER_MODE 3 +#define ERL_SHELL_MODE 4 + +/* + * Distribution + */ + +#define SHORT_NAME 0 +#define FULL_NAME 1 + +/* + * Local functions. + */ + +static void error(char* format, ...); +static char* emalloc(size_t size); +static char* strsave(char* string); +static void push_words(char* src); +static int run_erlang(char* name, char** argv); +static char* get_default_emulator(char* progname); +#ifdef __WIN32__ +static char* possibly_quote(char* arg); +#endif + +/* + * Supply a strerror() function if libc doesn't. + */ +#ifndef HAVE_STRERROR + +extern int sys_nerr; + +#ifndef SYS_ERRLIST_DECLARED +extern const char * const sys_errlist[]; +#endif /* !SYS_ERRLIST_DECLARED */ + +char *strerror(int errnum) +{ + static char *emsg[1024]; + + if (errnum != 0) { + if (errnum > 0 && errnum < sys_nerr) + sprintf((char *) &emsg[0], "(%s)", sys_errlist[errnum]); + else + sprintf((char *) &emsg[0], "errnum = %d ", errnum); + } + else { + emsg[0] = '\0'; + } + return (char *) &emsg[0]; +} +#endif /* !HAVE_STRERROR */ + +int +main(int argc, char** argv) +{ + int eargv_size; + int eargc_base; /* How many arguments in the base of eargv. */ + char* emulator; + char nodename[100]; + char browser[100]; + int ct_mode; + int dist_mode; + int cnt; + int erl_args; + char** argv0 = argv; + + emulator = get_default_emulator(argv[0]); + + /* + * Allocate the argv vector to be used for arguments to Erlang. + * Arrange for starting to pushing information in the middle of + * the array, to allow easy addition of commands in the beginning. + */ + + eargv_size = argc*4+100; + eargv_base = (char **) emalloc(eargv_size*sizeof(char*)); + eargv = eargv_base; + eargc = 0; + push_words(emulator); + eargc_base = eargc; + eargv = eargv + eargv_size/2; + eargc = 0; + + strcpy(nodename, "ct"); + dist_mode = SHORT_NAME; + browser[0] = '\0'; + ct_mode = NORMAL_MODE; + erl_args = argc; + cnt = 1; + + /* + * Check various flags before building command line + */ + + while (cnt < argc) { + if (strcmp(argv[1], "-erl_args") == 0) { + erl_args = cnt; + } + else if (strcmp(argv[1], "-sname") == 0) { + strcpy(nodename, argv[2]); + cnt++, argv++; + } + else if (strcmp(argv[1], "-name") == 0) { + strcpy(nodename, argv[2]); + dist_mode = FULL_NAME; + cnt++, argv++; + } + else { + if (cnt < erl_args) { + if (strcmp(argv[1], "-vts") == 0) { + ct_mode = VTS_MODE; + } + else if (strcmp(argv[1], "-browser") == 0) { + strcpy(browser, argv[2]); + cnt++, argv++; + } + else if (strcmp(argv[1], "-shell") == 0) { + ct_mode = CT_SHELL_MODE; + } + else if (strcmp(argv[1], "-ctmaster") == 0) { + strcpy(nodename, "ct_master"); + ct_mode = MASTER_MODE; + } + else if (strcmp(argv[1], "-ctname") == 0) { + strcpy(nodename, argv[2]); + ct_mode = ERL_SHELL_MODE; + cnt++, argv++; + } + } + } + cnt++, argv++; + } + + argv = argv0; + + /* + * Push initial arguments. + */ + + if (dist_mode == FULL_NAME) { + PUSH2("-name", nodename); + } + else { + PUSH2("-sname", nodename); + } + + /* + * Push everything else + */ + + if (ct_mode == VTS_MODE) { + PUSH4("-s", "webtool", "script_start", "vts"); + if (browser[0] != '\0') PUSH(browser); + PUSH3("-s", "ct_run", "script_start"); + } + else if (ct_mode == CT_SHELL_MODE) { + PUSH3("-s", "ct_run", "script_start"); + } + else if (ct_mode == NORMAL_MODE) { + PUSH3("-s", "ct_run", "script_start"); + PUSH3("-s", "erlang", "halt"); + } + + cnt = 1; + while (cnt < argc) { + if (strcmp(argv[1], "-erl_args") == 0) { + PUSH("-ct_erl_args"); + } + else if ((strcmp(argv[1], "-sname") == 0) || (strcmp(argv[1], "-name") == 0)) { + cnt++, argv++; + } + else if (cnt < erl_args) { + if (strcmp(argv[1], "-config") == 0) + PUSH("-ct_config"); + else if (strcmp(argv[1], "-decrypt_key") == 0) + PUSH("-ct_decrypt_key"); + else if (strcmp(argv[1], "-decrypt_file") == 0) + PUSH("-ct_decrypt_file"); + else + PUSH(argv[1]); + } + else { + PUSH(argv[1]); + } + cnt++, argv++; + } + + /* + * Move up the commands for invoking the emulator and adjust eargv + * accordingly. + */ + + while (--eargc_base >= 0) { + UNSHIFT(eargv_base[eargc_base]); + } + + /* + * Invoke Erlang with the collected options. + */ + + PUSH(NULL); + + return run_erlang(eargv[0], eargv); +} + +static void +push_words(char* src) +{ + char sbuf[1024]; + char* dst; + + dst = sbuf; + while ((*dst++ = *src++) != '\0') { + if (isspace((int)*src)) { + *dst = '\0'; + PUSH(strsave(sbuf)); + dst = sbuf; + do { + src++; + } while (isspace((int)*src)); + } + } + if (sbuf[0]) + PUSH(strsave(sbuf)); +} +#ifdef __WIN32__ +char *make_commandline(char **argv) +{ + static char *buff = NULL; + static int siz = 0; + int num = 0; + char **arg, *p; + + if (*argv == NULL) { + return ""; + } + for (arg = argv; *arg != NULL; ++arg) { + num += strlen(*arg)+1; + } + if (!siz) { + siz = num; + buff = malloc(siz*sizeof(char)); + } else if (siz < num) { + siz = num; + buff = realloc(buff,siz*sizeof(char)); + } + p = buff; + for (arg = argv; *arg != NULL; ++arg) { + strcpy(p,*arg); + p+=strlen(*arg); + *p++=' '; + } + *(--p) = '\0'; + + if (debug) { + printf("Processed commandline:%s\n",buff); + } + return buff; +} + +int my_spawnvp(char **argv) +{ + STARTUPINFO siStartInfo; + PROCESS_INFORMATION piProcInfo; + DWORD ec; + + memset(&siStartInfo,0,sizeof(STARTUPINFO)); + siStartInfo.cb = sizeof(STARTUPINFO); + siStartInfo.dwFlags = STARTF_USESTDHANDLES; + siStartInfo.hStdInput = GetStdHandle(STD_INPUT_HANDLE); + siStartInfo.hStdOutput = GetStdHandle(STD_OUTPUT_HANDLE); + siStartInfo.hStdError = GetStdHandle(STD_ERROR_HANDLE); + siStartInfo.wShowWindow = SW_HIDE; + siStartInfo.dwFlags |= STARTF_USESHOWWINDOW; + + + if (!CreateProcess(NULL, + make_commandline(argv), + NULL, + NULL, + TRUE, + 0, + NULL, + NULL, + &siStartInfo, + &piProcInfo)) { + return -1; + } + CloseHandle(piProcInfo.hThread); + + WaitForSingleObject(piProcInfo.hProcess,INFINITE); + if (!GetExitCodeProcess(piProcInfo.hProcess,&ec)) { + return 0; + } + return (int) ec; +} +#endif /* __WIN32__ */ + + +static int +run_erlang(char* progname, char** argv) +{ +#ifdef __WIN32__ + int status; +#endif + + if (debug) { + int i = 0; + while (argv[i] != NULL) + printf(" %s", argv[i++]); + printf("\n"); + } + +#ifdef __WIN32__ + /* + * Alas, we must wait here for the program to finish. + * Otherwise, the shell from which we were executed will think + * we are finished and print a prompt and read keyboard input. + */ + + status = my_spawnvp(argv)/*_spawnvp(_P_WAIT,progname,argv)*/; + if (status == -1) { + fprintf(stderr, "run_test: Error executing '%s': %d", progname, + GetLastError()); + } + return status; +#else + execvp(progname, argv); + error("Error %d executing \'%s\'.", errno, progname); + return 2; +#endif +} + +static void +error(char* format, ...) +{ + char sbuf[1024]; + va_list ap; + + va_start(ap, format); + vsprintf(sbuf, format, ap); + va_end(ap); + fprintf(stderr, "run_test: %s\n", sbuf); + exit(1); +} + +static char* +emalloc(size_t size) +{ + char *p = malloc(size); + if (p == NULL) + error("Insufficient memory"); + return p; +} + +static char* +strsave(char* string) +{ + char* p = emalloc(strlen(string)+1); + strcpy(p, string); + return p; +} + +static char* +get_default_emulator(char* progname) +{ + char sbuf[MAXPATHLEN]; + char* s; + + strcpy(sbuf, progname); + for (s = sbuf+strlen(sbuf); s >= sbuf; s--) { + if (IS_DIRSEP(*s)) { + strcpy(s+1, ERL_NAME); +#ifdef __WIN32__ + if (_access(sbuf, 0) != -1) { + return strsave(sbuf); + } +#else + if (access(sbuf, 1) != -1) { + return strsave(sbuf); + } +#endif + break; + } + } + return ERL_NAME; +} + +#ifdef __WIN32__ +static char* +possibly_quote(char* arg) +{ + int mustQuote = NO; + int n = 0; + char* s; + char* narg; + + if (arg == NULL) { + return arg; + } + + /* + * Scan the string to find out if it needs quoting and return + * the original argument if not. + */ + + for (s = arg; *s; s++, n++) { + switch(*s) { + case ' ': + mustQuote = YES; + continue; + case '"': + mustQuote = YES; + n++; + continue; + case '\\': + if(s[1] == '"') + n++; + continue; + default: + continue; + } + } + if (!mustQuote) { + return arg; + } + + /* + * Insert the quotes and put a backslash in front of every quote + * inside the string. + */ + + s = narg = emalloc(n+2+1); + for (*s++ = '"'; *arg; arg++, s++) { + if (*arg == '"' || (*arg == '\\' && arg[1] == '"')) { + *s++ = '\\'; + } + *s = *arg; + } + if (s[-1] == '\\') { + *s++ ='\\'; + } + *s++ = '"'; + *s = '\0'; + return narg; +} +#endif /* __WIN32__ */ diff --git a/erts/etc/unix/Install.src b/erts/etc/unix/Install.src index 83f9690782..7dead62ab0 100644 --- a/erts/etc/unix/Install.src +++ b/erts/etc/unix/Install.src @@ -89,6 +89,7 @@ cp -p $ERL_ROOT/erts-%I_VSN%/bin/erl . cp -p $ERL_ROOT/erts-%I_VSN%/bin/erlc . cp -p $ERL_ROOT/erts-%I_VSN%/bin/dialyzer . cp -p $ERL_ROOT/erts-%I_VSN%/bin/typer . +cp -p $ERL_ROOT/erts-%I_VSN%/bin/run_test . cp -p $ERL_ROOT/erts-%I_VSN%/bin/escript . # diff --git a/erts/etc/win32/Install.c b/erts/etc/win32/Install.c index 4a559cd8a2..ca814e3f80 100644 --- a/erts/etc/win32/Install.c +++ b/erts/etc/win32/Install.c @@ -1,19 +1,19 @@ /* * %CopyrightBegin% - * - * Copyright Ericsson AB 2003-2009. All Rights Reserved. - * + * + * Copyright Ericsson AB 2003-2010. All Rights Reserved. + * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in * compliance with the License. You should have received a copy of the * Erlang Public License along with this software. If not, it can be * retrieved online at http://www.erlang.org/. - * + * * Software distributed under the License is distributed on an "AS IS" * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See * the License for the specific language governing rights and limitations * under the License. - * + * * %CopyrightEnd% */ /* @@ -45,8 +45,8 @@ int main(int argc, char **argv) InitSection *ini_section; HANDLE module = GetModuleHandle(NULL); char *binaries[] = { "erl.exe", "werl.exe", "erlc.exe", - "dialyzer.exe", "typer.exe", - "escript.exe", NULL }; + "dialyzer.exe", "typer.exe", + "escript.exe", "run_test.exe", NULL }; char *scripts[] = { "start_clean.boot", "start_sasl.boot", NULL }; char fromname[MAX_PATH]; char toname[MAX_PATH]; diff --git a/erts/vsn.mk b/erts/vsn.mk index 6e1338cff5..ce6330e6dd 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -17,8 +17,8 @@ # %CopyrightEnd% # -VSN = 5.8 -SYSTEM_VSN = R14A +VSN = 5.8.1 +SYSTEM_VSN = R14B # Port number 4365 in 4.2 # Port number 4366 in 4.3 diff --git a/lib/appmon/doc/src/notes.xml b/lib/appmon/doc/src/notes.xml index 219b5671a4..29e66411a6 100644 --- a/lib/appmon/doc/src/notes.xml +++ b/lib/appmon/doc/src/notes.xml @@ -30,6 +30,21 @@ </header> <p>This document describes the changes made to the Appmon application.</p> +<section><title>Appmon 2.1.12</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Warnings due to new autoimported BIFs removed</p> + <p> + Own Id: OTP-8674 Aux Id: OTP-8579 </p> + </item> + </list> + </section> + +</section> + <section><title>Appmon 2.1.11</title> <section><title>Improvements and New Features</title> diff --git a/lib/appmon/vsn.mk b/lib/appmon/vsn.mk index 78b95e5688..cfcb5d3eb6 100644 --- a/lib/appmon/vsn.mk +++ b/lib/appmon/vsn.mk @@ -16,4 +16,4 @@ # # %CopyrightEnd% -APPMON_VSN = 2.1.11 +APPMON_VSN = 2.1.12 diff --git a/lib/asn1/doc/src/notes.xml b/lib/asn1/doc/src/notes.xml index 5eadb0c90c..c5ec682e0f 100644 --- a/lib/asn1/doc/src/notes.xml +++ b/lib/asn1/doc/src/notes.xml @@ -31,6 +31,25 @@ <p>This document describes the changes made to the asn1 application.</p> +<section><title>Asn1 1.6.14</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + By default, the ASN.1 compiler is now silent in the + absence of warnings or errors. The new '<c>verbose</c>' + option or the '<c>-v</c>' option for <c>erlc</c> can be + given to show extra information (for instance, about the + files that are generated). (Thanks to Tuncer Ayaz.)</p> + <p> + Own Id: OTP-8565</p> + </item> + </list> + </section> + +</section> + <section><title>Asn1 1.6.13</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/asn1/vsn.mk b/lib/asn1/vsn.mk index 32151a0cac..b7e91e42a0 100644 --- a/lib/asn1/vsn.mk +++ b/lib/asn1/vsn.mk @@ -1,7 +1,12 @@ -#next version number to use is 1.6.14 | 1.7 | 2.0 -ASN1_VSN = 1.6.13 +#next version number to use is 1.6.15 | 1.7 | 2.0 +ASN1_VSN = 1.6.14 -TICKETS = OTP-8463 +TICKETS = OTP-8565 \ + OTP-8516 + +TICKETS_1.6.14 = \ + OTP-8565 \ + OTP-8516 TICKETS_1.6.13 = \ OTP-8463 diff --git a/lib/common_test/Makefile b/lib/common_test/Makefile index ebca4523ab..aecec1a50d 100644 --- a/lib/common_test/Makefile +++ b/lib/common_test/Makefile @@ -1,19 +1,19 @@ # # %CopyrightBegin% -# -# Copyright Ericsson AB 2003-2009. All Rights Reserved. -# +# +# Copyright Ericsson AB 2003-2010. All Rights Reserved. +# # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in # compliance with the License. You should have received a copy of the # Erlang Public License along with this software. If not, it can be # retrieved online at http://www.erlang.org/. -# +# # Software distributed under the License is distributed on an "AS IS" # basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See # the License for the specific language governing rights and limitations # under the License. -# +# # %CopyrightEnd% # diff --git a/lib/common_test/doc/src/Makefile b/lib/common_test/doc/src/Makefile index a2c014418d..6322860088 100644 --- a/lib/common_test/doc/src/Makefile +++ b/lib/common_test/doc/src/Makefile @@ -45,7 +45,8 @@ CT_MODULES = \ ct_ssh \ ct_rpc \ ct_snmp \ - unix_telnet + unix_telnet \ + ct_slave CT_XML_FILES = $(CT_MODULES:=.xml) diff --git a/lib/common_test/doc/src/common_test_app.xml b/lib/common_test/doc/src/common_test_app.xml index 7b52883f8a..e30eef2488 100644 --- a/lib/common_test/doc/src/common_test_app.xml +++ b/lib/common_test/doc/src/common_test_app.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>2003</year><year>2009</year> + <year>2003</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Common Test</title> @@ -337,7 +337,7 @@ </func> <func> - <name>Module:testcase() -> [Info] </name> + <name>Module:Testcase() -> [Info] </name> <fsummary>Test case info function. </fsummary> <type> <v> Info = {timetrap,Time} | {require,Required} | @@ -396,7 +396,7 @@ <func> - <name>Module:testcase(Config) -> void() | {skip,Reason} | {comment,Comment} | {save_config,SaveConfig} | {skip_and_save,Reason,SaveConfig} | exit() </name> + <name>Module:Testcase(Config) -> void() | {skip,Reason} | {comment,Comment} | {save_config,SaveConfig} | {skip_and_save,Reason,SaveConfig} | exit() </name> <fsummary>A test case</fsummary> <type> <v> Config = SaveConfig = [{Key,Value}]</v> diff --git a/lib/common_test/doc/src/config_file_chapter.xml b/lib/common_test/doc/src/config_file_chapter.xml index a22a5270c1..850bc74e19 100644 --- a/lib/common_test/doc/src/config_file_chapter.xml +++ b/lib/common_test/doc/src/config_file_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2009</year> + <year>2004</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Config Files</title> @@ -180,7 +180,126 @@ </section> <section> - <title>Examples</title> + <title>User specific configuration data formats</title> + + <p>It is possible for the user to specify configuration data on a + different format than key-value tuples in a text file, as described + so far. The data can e.g. be read from arbitrary files, fetched from + the web over http, or requested from a user specific process. + To support this, Common Test provides a callback module plugin + mechanism to handle configuration data.</p> + + <section> + <title>Default callback modules for handling configuration data</title> + <p>The Common Test application includes default callback modules + for handling configuration data specified in standard config files + (see above) and in xml files:</p> + <list> + <item> + <c>ct_config_plain</c> - for reading configuration files with + key-value tuples (standard format). This handler will be used to + parse configuration files if no user callback is specified. + </item> + <item> + <c>ct_config_xml</c> - for reading configuration data from XML + files. + </item> + </list> + </section> + + <section> + <title>Using XML configuration files</title> + <p>This is an example of an XML configuration file:</p> + <pre><![CDATA[ +<config> + <ftp_host> + <ftp>"targethost"</ftp> + <username>"tester"</username> + <password>"letmein"</password> + </ftp_host> + <lm_directory>"/test/loadmodules"</lm_directory> +</config>]]></pre> + + <p>This configuration file, once read, will produce the same configuration + variables as the following text file:</p> + <pre> +{ftp_host, [{ftp,"targethost"}, + {username,"tester"}, + {password,"letmein"}]}. + +{lm_directory, "/test/loadmodules"}.</pre> + </section> + + <section> + <title>How to implement a user specific handler</title> + + <p>The user specific handler can be written to handle special + configuration file formats. The parameter can be either file + name(s) or configuration string(s) (the empty list is valid).</p> + + <p>The callback module implementing the handler is responsible for + checking correctness of configuration strings.</p> + + <p>To perform validation of the configuration strings, the callback module + should have the following function exported:</p> + + <p><c>Callback:check_parameter/1</c></p> + <p>The input argument will be passed from Common Test, as defined in the test + specification or given as an option to <c>run_test</c>.</p> + + <p>The return value should be any of the following values indicating if given + configuration parameter is valid:</p> + <list> + <item> + <c>{ok, {file, FileName}}</c> - parameter is a file name and + the file exists, + </item> + <item> + <c>{ok, {config, ConfigString}}</c> - parameter is a config string + and it is correct, + </item> + <item> + <c>{error, {nofile, FileName}}</c> - there is no file with the given + name in the current directory, + </item> + <item> + <c>{error, {wrong_config, ConfigString}}</c> - the configuration string + is wrong. + </item> + </list> + + <p>To perform reading of configuration data - initially before the tests + start, or as a result of data being reloaded during test execution - + the following function should be exported from the callback module:</p> + + <p><c>Callback:read_config/1</c></p> + + <p>The input argument is the same as for the <c>check_parameter/1</c> function.</p> + <p>The return value should be either:</p> + + <list> + <item> + <c>{ok, Config}</c> - if the configuration variables are read successfully, + </item> + <item> + <c>{error, Error, ErrorDetails}</c> - if the callback module fails to + proceed with the given configuration parameters. + </item> + </list> + <p><c>Config</c> is the proper Erlang key-value list, with possible + key-value sublists as values, like for the configuration file + example above:</p> + + <pre> + [{ftp_host, [{ftp, "targethost"}, {username, "tester"}, {password, "letmein"}]}, + {lm_directory, "/test/loadmodules"}]</pre> + + </section> + + </section> + + <section> + <title>Examples of configuration data handling</title> <p>A config file for using the FTP client to access files on a remote host could look like this:</p> @@ -188,28 +307,33 @@ <pre> {ftp_host, [{ftp,"targethost"}, {username,"tester"}, - {password,"letmein"}]}. + {password,"letmein"}]}. {lm_directory, "/test/loadmodules"}.</pre> - <p>Example of how to assert that the configuration data is available and + + <p>The XML version shown in the chapter above can also be used, but it should be + explicitly specified that the <c>ct_config_xml</c> callback module is to be + used by Common Test.</p> + + <p>Example of how to assert that the configuration data is available and use it for an FTP session:</p> <pre> init_per_testcase(ftptest, Config) -> {ok,_} = ct_ftp:open(ftp), - Config. + Config. end_per_testcase(ftptest, _Config) -> ct_ftp:close(ftp). ftptest() -> [{require,ftp,ftp_host}, - {require,lm_directory}]. + {require,lm_directory}]. ftptest(Config) -> - Remote = filename:join(ct:get_config(lm_directory), "loadmodX"), + Remote = filename:join(ct:get_config(lm_directory), "loadmodX"), Local = filename:join(?config(priv_dir,Config), "loadmodule"), ok = ct_ftp:recv(ftp, Remote, Local), - ...</pre> + ...</pre> <p>An example of how the above functions could be rewritten if necessary to open multiple connections to the FTP server:</p> @@ -217,7 +341,7 @@ init_per_testcase(ftptest, Config) -> {ok,Handle1} = ct_ftp:open(ftp_host), {ok,Handle2} = ct_ftp:open(ftp_host), - [{ftp_handles,[Handle1,Handle2]} | Config]. + [{ftp_handles,[Handle1,Handle2]} | Config]. end_per_testcase(ftptest, Config) -> lists:foreach(fun(Handle) -> ct_ftp:close(Handle) end, @@ -225,17 +349,117 @@ ftptest() -> [{require,ftp_host}, - {require,lm_directory}]. + {require,lm_directory}]. ftptest(Config) -> - Remote = filename:join(ct:get_config(lm_directory), "loadmodX"), + Remote = filename:join(ct:get_config(lm_directory), "loadmodX"), Local = filename:join(?config(priv_dir,Config), "loadmodule"), [Handle | MoreHandles] = ?config(ftp_handles,Config), ok = ct_ftp:recv(Handle, Remote, Local), - ...</pre> + ...</pre> </section> + <section> + <title>Example of user specific configuration handler</title> + <p>A simple configuration handling driver which will ask an external server for + configuration data can be implemented this way:</p> + <pre> +-module(config_driver). +-export([read_config/1, check_parameter/1]). + +read_config(ServerName)-> + ServerModule = list_to_atom(ServerName), + ServerModule:start(), + ServerModule:get_config(). + +check_parameter(ServerName)-> + ServerModule = list_to_atom(ServerName), + case code:is_loaded(ServerModule) of + {file, _}-> + {ok, {config, ServerName}}; + false-> + case code:load_file(ServerModule) of + {module, ServerModule}-> + {ok, {config, ServerName}}; + {error, nofile}-> + {error, {wrong_config, "File not found: " ++ ServerName ++ ".beam"}} + end + end.</pre> + + <p>The configuration string for this driver may be "config_server", if the + config_server.erl module below is compiled and exists in the code path + during test execution:</p> + <pre> +-module(config_server). +-export([start/0, stop/0, init/1, get_config/0, loop/0]). + +-define(REGISTERED_NAME, ct_test_config_server). + +start()-> + case whereis(?REGISTERED_NAME) of + undefined-> + spawn(?MODULE, init, [?REGISTERED_NAME]), + wait(); + _Pid-> + ok + end, + ?REGISTERED_NAME. + +init(Name)-> + register(Name, self()), + loop(). + +get_config()-> + call(self(), get_config). + +stop()-> + call(self(), stop). + +call(Client, Request)-> + case whereis(?REGISTERED_NAME) of + undefined-> + {error, not_started, Request}; + Pid-> + Pid ! {Client, Request}, + receive + Reply-> + {ok, Reply} + after 4000-> + {error, timeout, Request} + end + end. + +loop()-> + receive + {Pid, stop}-> + Pid ! ok; + {Pid, get_config}-> + {D,T} = erlang:localtime(), + Pid ! + [{localtime, [{date, D}, {time, T}]}, + {node, erlang:node()}, + {now, erlang:now()}, + {config_server_pid, self()}, + {config_server_vsn, ?vsn}], + ?MODULE:loop() + end. + +wait()-> + case whereis(?REGISTERED_NAME) of + undefined-> + wait(); + _Pid-> + ok + end.</pre> + + <p>In this example, the handler also provides the ability to dynamically reload + configuration variables. If <c>ct:reload_config(localtime)</c> is called from + the test case function, all variables loaded with <c>config_driver:read_config/1</c> + will be updated with their latest values, and the new value for variable + <c>localtime</c> will be returned.</p> + </section> + </chapter> diff --git a/lib/common_test/doc/src/ct_master_chapter.xml b/lib/common_test/doc/src/ct_master_chapter.xml index 79288cfe4c..01f8e61d36 100644 --- a/lib/common_test/doc/src/ct_master_chapter.xml +++ b/lib/common_test/doc/src/ct_master_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2006</year><year>2009</year> + <year>2006</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Using Common Test for Large Scale Testing</title> @@ -30,6 +30,7 @@ </header> <section> + <marker id="general"></marker> <title>General</title> <p>Large scale automated testing requires running multiple independent test sessions in parallel. This is accomplished by running @@ -102,9 +103,10 @@ <p><c>ct_master:abort()</c> (stop all) or <c>ct_master:abort(Nodes)</c></p> <p>For detailed information about the <c>ct_master</c> API, please see the - manual page for this module.</p> + <seealso marker="ct_master">manual page</seealso> for this module.</p> </section> <section> + <marker id="test_specifications"></marker> <title>Test Specifications</title> <p>The test specifications used as input to CT Master are fully compatible with the specifications used as input to the regular CT server. The syntax is described in the @@ -186,7 +188,7 @@ <seealso marker="run_test_chapter#test_specifications">Running Test Suites</seealso> chapter). The result is that any test specified to run on a node with the same name as the Common Test node in question (typically <c>ct@somehost</c> if started - with the <c>run_test</c> script), will be performed. Tests without explicit + with the <c>run_test</c> program), will be performed. Tests without explicit node association will always be performed too of course!</p> <note><p>It is recommended that absolute paths are used for log directories, @@ -194,6 +196,56 @@ current working directory settings are not important.</p></note> </section> + <section> + <title>Automatic startup of test target nodes</title> + <marker id="ct_slave"></marker> + <p>Is is possible to automatically start, and perform initial actions, on + test target nodes by using the test specification term <c>init</c>.</p> + <p>Currently, two sub-terms are supported, <c>node_start</c> and <c>eval</c>.</p> + <p>Example:</p> + <pre> + {node, node1, node1@host1}. + {node, node2, node1@host2}. + {node, node3, node2@host2}. + {node, node4, node1@host3}. + {init, node1, [{node_start, [{callback_module, my_slave_callback}]}]}. + {init, [node2, node3], {node_start, [{username, "ct_user"}, {password, "ct_password"}]}}. + {init, node4, {eval, {module, function, []}}}.</pre> + + <p>This test specification declares that <c>node1@host1</c> is to be started using + the user callback function <c>callback_module:my_slave_callback/0</c>, and nodes + <c>node1@host2</c> and <c>node2@host2</c> will be started with the default callback + module <c>ct_slave</c>. The given user name and password is used to log into remote + host <c>host2</c>. Also, the function <c>module:function/0</c> will be evaluated on + <c>node1@host3</c>, and the result of this call will be printed to the log.</p> + + <p>The default <seealso marker="ct_slave">ct_slave</seealso> callback module, + which is part of the Common Test application, has the following features: + <list> + <item>Starting Erlang target nodes on local or remote hosts + (ssh is used for communication). + </item> + <item>Ability to start an Erlang emulator with additional flags + (any flags supported by <c>erl</c> are supported). + </item> + <item>Supervision of a node being started by means of internal callback + functions. Used to prevent hanging nodes. (Configurable). + </item> + <item>Monitoring of the master node by the slaves. A slave node may be + stopped in case the master node terminates. (Configurable). + </item> + <item>Execution of user functions after a slave node is started. + Functions can be given as a list of {Module, Function, Arguments} tuples. + </item> + </list> + </p> + <p>Note that it is possible to specify an <c>eval</c> term for the node as well + as <c>startup_functions</c> in the <c>node_start</c> options list. In this + case first the node will be started, then the <c>startup_functions</c> are + executed, and finally functions specified with <c>eval</c> are called. + </p> + </section> + </chapter> diff --git a/lib/common_test/doc/src/event_handler_chapter.xml b/lib/common_test/doc/src/event_handler_chapter.xml index a550810850..7f5144b760 100644 --- a/lib/common_test/doc/src/event_handler_chapter.xml +++ b/lib/common_test/doc/src/event_handler_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2006</year><year>2009</year> + <year>2006</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Event Handling</title> @@ -68,29 +68,27 @@ Example:</p> <p><c>$ run_test -suite test/my_SUITE -event_handler handlers/my_evh1 handlers/my_evh2 -pa $PWD/handlers</c></p> + <p>Use the <c><![CDATA[run_test -event_handler_init]]></c> option instead of + <c><![CDATA[-event_handler]]></c> to pass start arguments to the event handler + init function.</p> <p>All event handler modules must have gen_event behaviour. Note also that these modules must be precompiled, and that their locations must be added explicitly to the Erlang code server search path (like in the example).</p> - <p>It is not possible to specify start arguments to the event handlers when - using the <c>run_test</c> script. You may however pass along start arguments - if you use the <c>ct:run_test/1</c> function. An event_handler tuple in the argument - <c>Opts</c> has the following definition (see also <c>ct:run_test/1</c> in the - reference manual):</p> + <p>An event_handler tuple in the argument <c>Opts</c> has the following + definition (see also <c>ct:run_test/1</c> in the reference manual):</p> <pre> {event_handler,EventHandlers} EventHandlers = EH | [EH] EH = atom() | {atom(),InitArgs} | {[atom()],InitArgs} - InitArgs = [term()] - </pre> + InitArgs = [term()]</pre> <p>Example:</p> <pre> - 1> ct:run_test([{suite,"test/my_SUITE"},{event_handler,[my_evh1,{my_evh2,[node()]}]}]). - </pre> + 1> ct:run_test([{suite,"test/my_SUITE"},{event_handler,[my_evh1,{my_evh2,[node()]}]}]).</pre> <p>This will install two event handlers for the <c>my_SUITE</c> test. Event handler <c>my_evh1</c> is started with <c>[]</c> as argument to the init function. Event handler <c>my_evh2</c> is started with the name of the current node in the init argument list.</p> diff --git a/lib/common_test/doc/src/install_chapter.xml b/lib/common_test/doc/src/install_chapter.xml index e1ff5abf6a..828588a673 100644 --- a/lib/common_test/doc/src/install_chapter.xml +++ b/lib/common_test/doc/src/install_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2007</year><year>2009</year> + <year>2007</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Installation</title> @@ -33,82 +33,77 @@ <marker id="general"></marker> <title>General information</title> - <p>The two main interfaces for running tests with Common Test - are an executable Bourne shell script named <c>run_test</c> and an - erlang module named <c>ct</c>. The shell script will work on Unix/Linux - (and Linux-like environments such as Cygwin on Windows) and the - <c>ct</c> interface functions can be called from the Erlang shell - (or from any Erlang function) on any supported platform.</p> - - <p>The Common Test application is installed with the Erlang/OTP - system and no explicit installation is required to start using - Common Test by means of the interface functions in the <c>ct</c> - module. If you wish to use <c>run_test</c>, however, this script - needs to be generated first, according to the instructions below.</p> - </section> + <p>The two main interfaces for running tests with Common Test + are an executable program named run_test and an + erlang module named <c>ct</c>. The run_test program + is compiled for the underlying operating system (e.g. Unix/Linux + or Windows) during the build of the Erlang/OTP system, and is + installed automatically with other executable programs in + the top level <c>bin</c> directory of Erlang/OTP. + The <c>ct</c> interface functions can be called from the Erlang shell, + or from any Erlang function, on any supported platform.</p> + + <p>A legacy Bourne shell script - also named run_test - exists, + which may be manually generated and installed. This script may be used + instead of the run_test program mentioned above, e.g. if the user + wishes to modify or customize the Common Test start flags in a simpler + way than making changes to the run_test C program.</p> - <section> - <title>Unix/Linux</title> - - <p>Go to the <c><![CDATA[common_test-<vsn>]]></c> directory, located - among the other OTP applications (under the OTP lib directory). Here you - execute the <c>install.sh</c> script with argument <c>local</c>:</p> - - <p><c> - $ ./install.sh local - </c></p> + <p>The Common Test application is installed with the Erlang/OTP + system and no additional installation step is required to start using + Common Test by means of the run_test executable program, and/or the interface + functions in the <c>ct</c> module. If you wish to use the legacy Bourne + shell script version of run_test, however, this script needs to be + generated first, according to the instructions below.</p> + + <p><note>Before reading on, please note that since Common Test version + 1.5, the run_test shell script is no longer required for starting + tests with Common Test from the OS command line. The run_test + program (descibed above) is the new recommended command line interface + for Common Test. The shell script exists mainly for legacy reasons and + may not be updated in future releases of Common Test. It may even be removed. + </note></p> + + <p>Optional step to generate a shell script for starting Common Test:</p> + <p>To generate the run_test shell script, navigate to the + <c><![CDATA[common_test-<vsn>]]></c> directory, located among the other + OTP applications (under the OTP lib directory). Here execute the + <c>install.sh</c> script with argument <c>local</c>:</p> + + <p><c> + $ ./install.sh local + </c></p> - <p>This generates the executable <c>run_test</c> script in the - <c><![CDATA[common_test-<vsn>/priv/bin]]></c> directory. The script - will include absolute paths to the Common Test and Test Server - application directories, so it's possible to copy or move the script to - a different location on the file system, if desired, without having to - update it. It's of course possible to leave the script under the - <c>priv/bin</c> directory and update the PATH variable accordingly (or - create a link or alias to it).</p> - - <p>If you, for any reason, have copied Common Test and Test Server - to a different location than the default OTP lib directory, you can - generate a <c>run_test</c> script with a different top level directory, - simply by specifying the directory, instead of <c>local</c>, when running - <c>install.sh</c>. Example:</p> - - <p><c> - $ install.sh /usr/local/test_tools + <p>This generates the executable run_test script in the + <c><![CDATA[common_test-<vsn>/priv/bin]]></c> directory. The script + will include absolute paths to the Common Test and Test Server + application directories, so it's possible to copy or move the script to + a different location on the file system, if desired, without having to + update it. It's of course possible to leave the script under the + <c>priv/bin</c> directory and update the PATH variable accordingly (or + create a link or alias to it).</p> + + <p>If you, for any reason, have copied Common Test and Test Server + to a different location than the default OTP lib directory, you can + generate a run_test script with a different top level directory, + simply by specifying the directory, instead of <c>local</c>, when running + <c>install.sh</c>. Example:</p> + + <p><c> + $ install.sh /usr/local/test_tools </c></p> <p>Note that the <c><![CDATA[common_test-<vsn>]]></c> and - <c><![CDATA[test_server-<vsn>]]></c> directories must be located under the - same top directory. Note also that the install script does not copy files - or update environment variables. It only generates the <c>run_test</c> - script.</p> + <c><![CDATA[test_server-<vsn>]]></c> directories must be located under the + same top directory. Note also that the install script does not copy files + or update environment variables. It only generates the run_test + script.</p> - <p>Whenever you install a new version of Erlang/OTP, the <c>run_test</c> - script needs to be regenerated, or updated manually with new directory names - (new version numbers), for it to "see" the latest Common Test and Test Server - versions.</p> + <p>Whenever you install a new version of Erlang/OTP, the run_test + script needs to be regenerated, or updated manually with new directory names + (new version numbers), for it to "see" the latest Common Test and Test Server + versions.</p> - <p>For more information on the <c>run_test</c> script and the <c>ct</c> - module, please see the reference manual.</p> - </section> - - <section> - <title>Windows</title> - - <p>On Windows it is very convenient to use Cygwin (<c>www.cygwin.com</c>) - for running Common Test and Erlang, since it enables you to use the - <c>run_test</c> script for starting Common Test. If you are a Cygwin - user, simply follow the instructions above for generating the <c>run_test</c> - script.</p> - - <p>If you do not use Cygwin, you have to rely on the API functions - in the <c>ct</c> module (instead of <c>run_test</c>) for running - Common Test as described initially in this chapter.</p> - - <p>If you, for any reason, have chosen to store Common Test and Test Server - in a different location than the default OTP lib directory, make - sure the <c>ebin</c> directories of these applications are included - in the Erlang code server path (so the application modules can be loaded).</p> </section> </chapter> diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index 4f5f6caa8c..eadfc83c07 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -32,6 +32,152 @@ <file>notes.xml</file> </header> +<section><title>Common_Test 1.5</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Process calls using monitors in Common Test would not + clear the inbox of remaining DOWN messages. This has been + fixed.</p> + <p> + Own Id: OTP-8621 Aux Id: seq11560 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + It is now possible for the user to provide specific + callback modules that handle test configuration data, so + that data on arbitray form can be accessed (e.g. by + reading files or by communicating with a configuration + server process). Two default callback modules have been + introduced in Common Test: ct_config_plain and + ct_config_xml. The former is used to handle the + traditional Common Test configuration files (with terms + on key-value tuple form) and the latter to handle + configuration data on XML representation.</p> + <p> + Own Id: OTP-8485</p> + </item> + <item> + <p> + It is now possible to execute test suites that are not + necessarily available on the local file system, but have + been loaded on the test node in advance (e.g. sent as + binaries from a remote node and loaded by RPC). A + requirement is that the no_auto_compile (or + {auto_compile,false}) parameter has been set.</p> + <p> + Own Id: OTP-8490 Aux Id: seq11500 </p> + </item> + <item> + <p> + Test Server will now call the end_per_testcase/2 function + even if the test case has been terminated explicitly + (with abort_current_testcase/1), or after a timetrap + timeout. Under these circumstances the return value of + end_per_testcase is completely ignored. Therefore the + function will not be able to change the reason for test + case termination by returning {fail,Reason}, nor will it + be able to save data with {save_config,Data}.</p> + <p> + Own Id: OTP-8500 Aux Id: seq11521 </p> + </item> + <item> + <p> + It is now possible to use the test specification term + 'init' to start Common Test nodes automatically, as well + as have initial function calls evaluated on the nodes. A + default callback module for the 'init' term, ct_slave, + has been introduced to enable Common Test Master to + perform host login and node startup operations over ssh.</p> + <p> + Own Id: OTP-8570</p> + </item> + <item> + <p> + The run_test script has been replaced by a program (with + the same name) which can be executed without explicit + installation. The start flags are the same as for the + legacy start script.</p> + <p> + Own Id: OTP-8650</p> + </item> + <item> + <p> + Previously, a repeat property of a test case group + specified the number of times the group should be + repeated after the main test run. I.e. {repeat,N} would + case the group to execute 1+N times. To be consistent + with the behaviour of the run_test repeat option, this + has been changed. N now specifies the absolute number of + executions instead.</p> + <p> + Own Id: OTP-8689 Aux Id: seq11502 </p> + </item> + <item> + <p> + With the run_test -erl_args option, it's possible to + divide the options on the run_test command line into ones + that Common Test should process (those preceding + -erl_args, and ones it should ignore (those succeeding + -erl_args). Options preceding -erl_args that Common Test + doesn't recognize are also ignored (i.e. the same + behaviour as earlier versions of Common Test).</p> + <p> + Own Id: OTP-8690 Aux Id: OTP-8650 </p> + </item> + <item> + <p> + Directories added with -pa or -pz in the pre-erl_args + part of the run_test command line will be converted from + relative to absolute, this to avoid problems loading user + modules when Common Test switches working directory + during the test run.</p> + <p> + Own Id: OTP-8691 Aux Id: OTP-8650 </p> + </item> + <item> + <p> + The timetrap handling has been made more user + controllable by means of new start options and new ct + interface functions. With the 'multiply_timetraps' start + option, it's possible to specify a value which all + timetrap timeout values get multiplied by. This is useful + e.g. to extend the timetraps temporarily while running + cover or trace. The 'scale_timetraps' start option + switches on or off the Test Server timetrap scaling + feature (which tries to detect if the tests may benefit + from extended timetraps, e.g. due to running certain test + tools, and performs the scaling automatically). + Furthermore, the ct:timetrap/1 function has been + introduced, which makes it possible to set/reset + timetraps during test execution. Also, a ct:sleep/1 + function is now available, which takes the timetrap + parameters into account when calculating the time to + suspend the process.</p> + <p> + Own Id: OTP-8693</p> + </item> + <item> + <p> + A new run_test start option, event_handler_init, has been + added that takes a start argument which gets passed to + the init function of the event handler.</p> + <p> + Own Id: OTP-8694</p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.4.7</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/common_test/doc/src/ref_man.xml b/lib/common_test/doc/src/ref_man.xml index beb3ed3247..8be234d979 100644 --- a/lib/common_test/doc/src/ref_man.xml +++ b/lib/common_test/doc/src/ref_man.xml @@ -4,7 +4,7 @@ <application xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>2003</year><year>2009</year> + <year>2003</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Common Test Reference Manual</title> @@ -75,6 +75,7 @@ <xi:include href="ct_snmp.xml"/> <xi:include href="ct_telnet.xml"/> <xi:include href="unix_telnet.xml"/> + <xi:include href="ct_slave.xml"/> </application> diff --git a/lib/common_test/doc/src/run_test.xml b/lib/common_test/doc/src/run_test.xml index d9dd22d411..2f0a94afba 100644 --- a/lib/common_test/doc/src/run_test.xml +++ b/lib/common_test/doc/src/run_test.xml @@ -4,7 +4,7 @@ <comref> <header> <copyright> - <year>2007</year><year>2009</year> + <year>2007</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,62 +13,95 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> - <title>The run_test shell script</title> + <title>The run_test program</title> <prepared>Peter Andersson</prepared> <responsible>Peter Andersson</responsible> <docno></docno> <approved></approved> <checked></checked> - <date>2007-07-04</date> - <rev>PA1</rev> + <date>2010-04-01</date> + <rev>PA2</rev> <file>run_test.xml</file> </header> - <com>run_test</com> - <comsummary>Shell script used for starting - Common Test from the Unix command line. + <com>run_test</com> + <comsummary>Program used for starting Common Test from the + OS command line. </comsummary> <description> - <p>The <c>run_test</c> script is automatically generated as Common - Test is installed (please see the Installation chapter in the Common - Test User's Guide for more information). The script accepts a number - of different start flags. Some flags trigger <c>run_test</c> - to start the Common Test application and pass on data to it. Some - flags start an Erlang node prepared for running Common Test in a - particular mode.</p> - <p><c>run_test</c> also accepts Erlang emulator - flags. These are used when <c>run_test</c> calls <c>erl</c> to - start the Erlang node (making it possible to e.g. add directories to - the code server path, change the cookie on the node, start - additional applications, etc).</p> - <p>If <c>run_test</c> is called without parameters, it prints all valid - start flags to stdout.</p> + <p>The <c>run_test</c> program is automatically installed with Erlang/OTP + and Common Test (please see the Installation chapter in the Common + Test User's Guide for more information). The program accepts a number + of different start flags. Some flags trigger <c>run_test</c> + to start the Common Test application and pass on data to it. Some + flags start an Erlang node prepared for running Common Test in a + particular mode.</p> + + <p>There is an interface function that corresponds to this program, + called <c>ct:run_test/1</c>, for starting Common Test from the Erlang + shell (or an Erlang program). Please see the <c>ct</c> man page for + details.</p> + + <p><c>run_test</c> also accepts Erlang emulator flags. These are used + when <c>run_test</c> calls <c>erl</c> to start the Erlang node + (making it possible to e.g. add directories to the code server path, + change the cookie on the node, start additional applications, etc).</p> + + <p>With the optional flag:</p> + <pre>-erl_args</pre> + <p>it's possible to divide the options on the <c>run_test</c> command line into + two groups, one that Common Test should process (those preceding <c>-erl_args</c>), + and one it should completely ignore and pass on directly to the emulator + (those following <c>-erl_args</c>). Options preceding <c>-erl_args</c> that Common Test + doesn't recognize, also get passed on to the emulator untouched. + By means of <c>-erl_args</c> the user may specify flags with the same name, but + with different destinations, on the <c>run_test</c> command line.</p> + <p>If <c>-pa</c> or <c>-pz</c> flags are specified in the Common Test group of options + (preceding <c>-erl_args</c>), relative directories will be converted to + absolute and re-inserted into the code path by Common Test (to avoid + problems loading user modules when Common Test changes working directory + during test runs). Common Test will however ignore <c>-pa</c> and <c>-pz</c> flags + following <c>-erl_args</c> on the command line. These directories are added + to the code path normally (i.e. on specified form)</p> + + <p>If <c>run_test</c> is called with option:</p> + <pre>-help</pre> + <p>it prints all valid start flags to stdout.</p> </description> + <marker id="run_test"></marker> + <section> <title>Run tests from command line</title> <pre> - run_test [-dir TestDir1 TestDir2 .. TestDirN] | - [-suite Suite1 Suite2 .. SuiteN + run_test [-dir TestDir1 TestDir2 .. TestDirN] | + [-suite Suite1 Suite2 .. SuiteN [[-group Group1 Group2 .. GroupN] [-case Case1 Case2 .. CaseN]]] [-step [config | keep_inactive]] [-config ConfigFile1 ConfigFile2 .. ConfigFileN] + [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 + ConfigString2 and .. and CallbackModuleN ConfigStringN] [-decrypt_key Key] | [-decrypt_file KeyFile] + [-label Label] [-logdir LogDir] [-silent_connections [ConnType1 ConnType2 .. ConnTypeN]] [-stylesheet CSSFile] [-cover CoverCfgFile] - [-event_handler EvHandler1 EvHandler2 .. EvHandlerN] + [-event_handler EvHandler1 EvHandler2 .. EvHandlerN] | + [-event_handler_init EvHandler1 InitArg1 and + EvHandler2 InitArg2 and .. EvHandlerN InitArgN] [-include InclDir1 InclDir2 .. InclDirN] [-no_auto_compile] + [-muliply_timetraps Multiplier] + [-scale_timetraps] [-repeat N [-force_stop]] | [-duration HHMMSS [-force_stop]] | [-until [YYMoMoDD]HHMMSS [-force_stop]] @@ -79,15 +112,22 @@ <pre> run_test -spec TestSpec1 TestSpec2 .. TestSpecN [-config ConfigFile1 ConfigFile2 .. ConfigFileN] + [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 + ConfigString2 and .. and CallbackModuleN ConfigStringN] [-decrypt_key Key] | [-decrypt_file KeyFile] + [-label Label] [-logdir LogDir] [-allow_user_terms] [-silent_connections [ConnType1 ConnType2 .. ConnTypeN]] [-stylesheet CSSFile] [-cover CoverCfgFile] - [-event_handler EvHandler1 EvHandler2 .. EvHandlerN] + [-event_handler EvHandler1 EvHandler2 .. EvHandlerN] | + [-event_handler_init EvHandler1 InitArg1 and + EvHandler2 InitArg2 and .. EvHandlerN InitArgN] [-include InclDir1 InclDir2 .. InclDirN] [-no_auto_compile] + [-muliply_timetraps Multiplier] + [-scale_timetraps] [-repeat N [-force_stop]] | [-duration HHMMSS [-force_stop]] | [-until [YYMoMoDD]HHMMSS [-force_stop]] @@ -97,12 +137,16 @@ <title>Run tests in web based GUI</title> <pre> run_test -vts [-browser Browser] - [-config ConfigFile1 ConfigFile2 .. ConfigFileN] - [-decrypt_key Key] | [-decrypt_file KeyFile] [-dir TestDir1 TestDir2 .. TestDirN] | [-suite Suite [[-group Group] [-case Case]]] + [-config ConfigFile1 ConfigFile2 .. ConfigFileN] + [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 + ConfigString2 and .. and CallbackModuleN ConfigStringN] + [-decrypt_key Key] | [-decrypt_file KeyFile] [-include InclDir1 InclDir2 .. InclDirN] [-no_auto_compile] + [-muliply_timetraps Multiplier] + [-scale_timetraps] [-basic_html]</pre> </section> <section> @@ -113,28 +157,23 @@ <section> <title>Run CT in interactive mode</title> <pre> - run_test -shell + run_test -shell [-config ConfigFile1 ConfigFile2 ... ConfigFileN] + [-userconfig CallbackModule1 ConfigString1 and CallbackModule2 + ConfigString2 and .. and CallbackModuleN ConfigStringN] [-decrypt_key Key] | [-decrypt_file KeyFile]</pre> </section> <section> - <title>Start an Erlang node with a given name</title> - <pre> - run_test -ctname NodeName</pre> - </section> - <section> <title>Start a Common Test Master node</title> <pre> run_test -ctmaster</pre> </section> <section> - <title>See also</title> - <p>Please read the <seealso marker="run_test_chapter">Running Test Suites</seealso> - chapter in the Common Test User's Guide for information about the meaning of the - different start flags.</p> + <title>See also</title> + <p>Please read the <seealso marker="run_test_chapter">Running Test Suites</seealso> + chapter in the Common Test User's Guide for information about the meaning of the + different start flags.</p> </section> </comref> - - diff --git a/lib/common_test/doc/src/run_test_chapter.xml b/lib/common_test/doc/src/run_test_chapter.xml index d731d18783..1efff25f5b 100644 --- a/lib/common_test/doc/src/run_test_chapter.xml +++ b/lib/common_test/doc/src/run_test_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2009</year> + <year>2003</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,15 +13,15 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> - <title>Running Test Suites</title> + <title>Running Tests</title> <prepared>Peter Andersson, Kenneth Lundin</prepared> <docno></docno> <date></date> @@ -97,25 +97,32 @@ the <c><![CDATA[{auto_compile,false}]]></c> option with <c><![CDATA[ct:run_test/1]]></c>. With automatic compilation disabled, the user is responsible for compiling the test suite modules - (and any help modules) before the test run. Common Test will only verify - that the specified test suites exist before starting the tests.</p> + (and any help modules) before the test run. If the modules can not be loaded + from the local file system during startup of Common Test, the user needs to + pre-load the modules before starting the test. Common Test will only verify + that the specified test suites exist (i.e. that they are, or can be, loaded). + This is useful e.g. if the test suites are transferred and loaded as binaries via + RPC from a remote node.</p> </section> <section> - <title>Running tests from the UNIX command line</title> + <title>Running tests from the OS command line</title> - <p>The script <c>run_test</c> can be used for running tests from - the Unix/Linux command line, e.g. + <p>The <c>run_test</c> program can be used for running tests from + the OS command line, e.g. </p> <list> <item><c><![CDATA[run_test -config <configfilenames> -dir <dirs>]]></c></item> <item><c><![CDATA[run_test -config <configfilenames> -suite <suiteswithfullpath>]]></c> </item> + <item><c><![CDATA[run_test -userconfig <callbackmodulename> <configfilenames> -suite <suiteswithfullpath>]]></c> + </item> <item><c><![CDATA[run_test -config <configfilenames> -suite <suitewithfullpath> -group <groupnames> -case <casenames>]]></c></item> </list> <p>Examples:</p> <p><c>$ run_test -config $CFGS/sys1.cfg $CFGS/sys2.cfg -dir $SYS1_TEST $SYS2_TEST</c></p> + <p><c>$ run_test -userconfig ct_config_xml $CFGS/sys1.xml $CFGS/sys2.xml -dir $SYS1_TEST $SYS2_TEST</c></p> <p><c>$ run_test -suite $SYS1_TEST/setup_SUITE $SYS2_TEST/config_SUITE</c></p> <p><c>$ run_test -suite $SYS1_TEST/setup_SUITE -case start stop</c></p> <p><c>$ run_test -suite $SYS1_TEST/setup_SUITE -group installation -case start stop</c></p> @@ -123,6 +130,8 @@ <p>Other flags that may be used with <c>run_test</c>:</p> <list> <item><c><![CDATA[-logdir <dir>]]></c>, specifies where the HTML log files are to be written.</item> + <item><c><![CDATA[-label <name_of_test_run>]]></c>, associates the test run with a name that gets printed + in the overview HTML log files.</item> <item><c>-refresh_logs</c>, refreshes the top level HTML index files.</item> <item><c>-vts</c>, start web based GUI (see below).</item> <item><c>-shell</c>, start interactive shell mode (see below).</item> @@ -136,8 +145,14 @@ <seealso marker="cover_chapter#cover">Code Coverage Analysis</seealso>).</item> <item><c><![CDATA[-event_handler <event_handlers>]]></c>, to install <seealso marker="event_handler_chapter#event_handling">event handlers</seealso>.</item> + <item><c><![CDATA[-event_handler_init <event_handlers>]]></c>, to install + <seealso marker="event_handler_chapter#event_handling">event handlers</seealso> including start arguments.</item> <item><c><![CDATA[-include]]></c>, specifies include directories (see above).</item> <item><c><![CDATA[-no_auto_compile]]></c>, disables the automatic test suite compilation feature (see above).</item> + <item><c><![CDATA[-multiply_timetraps <n>]]></c>, extends <seealso marker="write_test_chapter#timetraps">timetrap + timeout</seealso> values.</item> + <item><c><![CDATA[-scale_timetraps <bool>]]></c>, enables automatic <seealso marker="write_test_chapter#timetraps">timetrap + timeout</seealso> scaling.</item> <item><c><![CDATA[-repeat <n>]]></c>, tells Common Test to repeat the tests n times (see below).</item> <item><c><![CDATA[-duration <time>]]></c>, tells Common Test to repeat the tests for duration of time (see below).</item> <item><c><![CDATA[-until <stop_time>]]></c>, tells Common Test to repeat the tests until stop_time (see below).</item> @@ -165,7 +180,7 @@ the current working directory of the Erlang Runtime System during the test run!</p> </note> - <p>For details on how to generate the <c>run_test</c> script, see the + <p>For more information about the <c>run_test</c> program, see the <seealso marker="install_chapter#general">Installation</seealso> chapter. </p> </section> @@ -174,7 +189,7 @@ <title>Running tests from the Web based GUI</title> <p>The web based GUI, VTS, is started with the <c>run_test</c> - script. From the GUI you can load config files, and select + program. From the GUI you can load config files, and select directories, suites and cases to run. You can also state the config files, directories, suites and cases on the command line when starting the web based GUI. @@ -198,10 +213,11 @@ <p>Example:</p> <p><c><![CDATA[$ run_test -vts -browser 'firefox&']]></c></p> <p>Note that the browser must run as a separate OS process or VTS will hang!</p> - <p>If no specific browser start command is specified, netscape will + <p>If no specific browser start command is specified, Firefox will be the default browser on Unix platforms and Internet Explorer on Windows. - If Common Test fails to start a browser automatically, start your - favourite browser manually instead and type in the URL that Common Test + If Common Test fails to start a browser automatically, or <c>'none'</c> is + specified as the value for -browser (i.e. <c>-browser none</c>), start your + favourite browser manually and type in the URL that Common Test displays in the shell.</p> </section> @@ -211,7 +227,7 @@ <p>Common Test provides an Erlang API for running tests. The main (and most flexible) function for specifying and executing tests is called <c>ct:run_test/1</c>. This function takes the same start parameters as - the <c>run_test</c> script described above, only the flags are instead + the <c>run_test</c> program described above, only the flags are instead given as options in a list of key-value tuples. E.g. a test specified with <c>run_test</c> like:</p> <p><c>$ run_test -suite ./my_SUITE -logdir ./results</c></p> @@ -237,13 +253,14 @@ manually and call <c>ct:install/1</c> to install any configuration data you might need (use <c>[]</c> as argument otherwise), then call <c>ct:start_interactive/0</c> to start Common Test. If you use - the <c>run_test</c> script, you may start the Erlang shell and Common Test + the <c>run_test</c> program, you may start the Erlang shell and Common Test in the same go by using the <c>-shell</c> and, optionally, the <c>-config</c> - flag: + and/or <c>-userconfig</c> flag. Examples: </p> <list> <item><c>run_test -shell</c></item> - <item><c><![CDATA[run_test -shell -config <configfilename>]]></c></item> + <item><c><![CDATA[run_test -shell -config cfg/db.cfg]]></c></item> + <item><c><![CDATA[run_test -shell -userconfig db_login testuser x523qZ]]></c></item> </list> <p>If no config file is given with the <c>run_test</c> command, @@ -268,7 +285,8 @@ 2> ct_telnet:open(unix_telnet). {ok,<0.105.0>} 4> ct_telnet:cmd(unix_telnet, "ls ."). - {ok,["ls .","file1 ...",...]}</pre> + {ok,["ls .","file1 ...",...]} + </pre> <p>Everything that Common Test normally prints in the test case logs, will in the interactive mode be written to a log named @@ -319,54 +337,99 @@ <marker id="test_specifications"></marker> <title>Using test specifications</title> - <p>The most expressive way to specify what to test is to use a so - called test specification. A test specification is a sequence of - Erlang terms. The terms may be declared in a text file or passed - to the test server at runtime as a list (see <c>run_testspec/1</c> - in the manual page for <c>ct</c>). There are two general types - of terms: configuration terms and test specification terms.</p> - <p>With configuration terms it is possible to import configuration - data (similar to <c>run_test -config</c>), specify HTML log - directories (similar to <c>run_test -logdir</c>), give aliases - to test nodes and test directories (to make a specification - easier to read and maintain), enable code coverage analysis - (see the <seealso marker="cover_chapter#cover">Code Coverage - Analysis</seealso> chapter) and specify event_handler plugins + <p>The most flexible way to specify what to test, is to use a so + called test specification. A test specification is a sequence of + Erlang terms. The terms may be declared in a text file or passed + to the test server at runtime as a list + (see <c>run_testspec/1</c> in the manual page + for <c>ct</c>). There are two general types of terms: + configuration terms and test specification terms.</p> + <p>With configuration terms it is possible to e.g. label the test + run (similar to <c>run_test -label</c>), evaluate arbitrary expressions + before starting a test, import configuration + data (similar to + <c>run_test -config/-userconfig</c>), specify HTML log directories (similar + to + <c>run_test -logdir</c>), give aliases to test nodes and test + directories (to make a specification easier to read and + maintain), enable code coverage analysis (see + the <seealso marker="cover_chapter#cover">Code Coverage + Analysis</seealso> chapter) and specify event_handler plugins (see the <seealso marker="event_handler_chapter#event_handling"> - Event Handling</seealso> chapter). There is also a term - for specifying include directories that should be passed on - to the compiler when automatic compilation is performed - (similar to <c>run_test -include</c>, see above).</p> - <p>With test specification terms it is possible to state exactly which - tests should run and in which order. A test term specifies either - one or more suites or one or more test cases. An arbitrary number of test - terms may be declared in sequence. A test term can also specify one or - more test suites or test cases to be skipped. Skipped suites and cases - are not executed and show up in the HTML test log as SKIPPED.</p> - - <note><p>It is not yet possible to specify test case groups in - test specifications. This will be supported in a soon upcoming - release.</p></note> + Event Handling</seealso> chapter). There is also a term for + specifying include directories that should be passed on to the + compiler when automatic compilation is performed (similar + to <c>run_test -include</c>, see above).</p> + <p>With test specification terms it is possible to state exactly + which tests should run and in which order. A test term specifies + either one or more suites, one or more test case groups, or one + or more test cases in a group or suite.</p> + <p>An arbitrary number of test terms may be declared in sequence. + Common Test will compile the terms into one or more tests to be + performed in one resulting test run. Note that a term that + specifies a set of test cases will "swallow" one that only + specifies a subset of these cases. E.g. the result of merging + one term that specifies that all cases in suite S should be + executed, with another term specifying only test case X and Y in + S, is a test of all cases in S. However, if a term specifying + test case X and Y in S is merged with a term specifying case Z + in S, the result is a test of X, Y and Z in S.</p> + <p>A test term can also specify one or more test suites, groups, + or test cases to be skipped. Skipped suites, groups and cases + are not executed and show up in the HTML test log files as + SKIPPED.</p> + <p>When a test case group is specified, the resulting test + executes the + <c>init_per_group</c> function, followed by all test cases and + sub groups (including their configuration functions), and + finally the <c>end_per_group</c> function. Also if particular + test cases in a group are specified, <c>init_per_group</c> + and <c>end_per_group</c> for the group in question are + called. If a group which is defined (in <c>Suite:group/0</c>) to + be a sub group of another group, is specified (or particular test + cases of a sub group are), Common Test will call the configuration + functions for the top level groups as well as for the sub group + in question (making it possible to pass configuration data all + the way from <c>init_per_suite</c> down to the test cases in the + sub group).</p> <p>Below is the test specification syntax. Test specifications can - be used to run tests both in a single test host environment and in - a distributed Common Test environment. Node parameters are only relevant in the - latter (see the chapter about running Common Test in distributed mode for information). - For details on the event_handler term, see the - <seealso marker="event_handler_chapter#event_handling">Event Handling</seealso> - chapter.</p> + be used to run tests both in a single test host environment and + in a distributed Common Test environment (Large Scale + Testing). The node parameters in the init term are only + relevant in the latter (see the + <seealso marker="ct_master_chapter#test_specifications">Large + Scale Testing</seealso> chapter for information). For details on + the event_handler term, see the + <seealso marker="event_handler_chapter#event_handling">Event + Handling</seealso> chapter.</p> <p>Config terms:</p> <pre> {node, NodeAlias, Node}. + + {init, InitOptions}. + {init, [NodeAlias], InitOptions}. + + {label, Label}. + {label, NodeRefs, Label}. + + {multiply_timetraps, N}. + {multiply_timetraps, NodeRefs, N}. + + {scale_timetraps, Bool}. + {scale_timetraps, NodeRefs, Bool}. {cover, CoverSpecFile}. - {cover, NodeRef, CoverSpecFile}. + {cover, NodeRefs, CoverSpecFile}. {include, IncludeDirs}. {include, NodeRefs, IncludeDirs}. {config, ConfigFiles}. {config, NodeRefs, ConfigFiles}. + + {userconfig, {CallbackModule, ConfigStrings}}. + {userconfig, NodeRefs, {CallbackModule, ConfigStrings}}. {alias, DirAlias, Dir}. @@ -383,6 +446,12 @@ {suites, DirRef, Suites}. {suites, NodeRefs, DirRef, Suites}. + {groups, DirRef, Suite, Groups}. + {groups, NodeRefsDirRef, Suite, Groups}. + + {groups, DirRef, Suite, Group, {cases,Cases}}. + {groups, NodeRefsDirRef, Suite, Group, {cases,Cases}}. + {cases, DirRef, Suite, Cases}. {cases, NodeRefs, DirRef, Suite, Cases}. @@ -395,9 +464,12 @@ <p>Types:</p> <pre> NodeAlias = atom() + InitOptions = term() Node = node() NodeRef = NodeAlias | Node | master NodeRefs = all_nodes | [NodeRef] | NodeRef + N = integer() + Bool = true | false CoverSpecFile = string() IncludeDirs = string() | [string()] ConfigFiles = string() | [string()] @@ -408,6 +480,9 @@ InitArgs = [term()] DirRef = DirAlias | Dir Suites = atom() | [atom()] | all + Suite = atom() + Groups = atom() | [atom()] | all + Group = atom() Cases = atom() | [atom()] | all Comment = string() | "" </pre> @@ -449,9 +524,14 @@ <item>Secondly, the test for system t2 should run. The included suites are t2B and t2C. Included are also test cases test4, test1 and test7 in suite t2A. Note that the test cases will be executed in the specified order.</item> - <item>Lastly, all suites for systems t3 are to be completely skipped and this + <item>Lastly, all suites for systems t3 are to be completely skipped and this should be explicitly noted in the log files.</item> </list> + <p>It is possible to specify initialization options for nodes defined in the + test specification. Currently, there are options to start the node and/or to + evaluate any function on the node. + See the <seealso marker="ct_master_chapter#ct_slave">Automatic startup of + the test target nodes</seealso> chapter for details.</p> <p>It is possible for the user to provide a test specification that includes (for Common Test) unrecognizable terms. If this is desired, the <c>-allow_user_terms</c> flag should be used when starting tests with diff --git a/lib/common_test/doc/src/test_structure_chapter.xml b/lib/common_test/doc/src/test_structure_chapter.xml index c8628b3a7a..cd38ae0c7c 100644 --- a/lib/common_test/doc/src/test_structure_chapter.xml +++ b/lib/common_test/doc/src/test_structure_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2006</year><year>2009</year> + <year>2006</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Test Structure</title> @@ -146,8 +146,8 @@ <tag><em>run_test</em></tag> <item> - The name of an executable Bourne shell script that may be - used on Linux/Unix as an interface for specifying and running + The name of an executable program that may be + used as an interface for specifying and running tests with Common Test. </item> diff --git a/lib/common_test/doc/src/write_test_chapter.xml b/lib/common_test/doc/src/write_test_chapter.xml index 212e3d85be..5afec6de6a 100644 --- a/lib/common_test/doc/src/write_test_chapter.xml +++ b/lib/common_test/doc/src/write_test_chapter.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2003</year><year>2009</year> + <year>2003</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Writing Test Suites</title> @@ -157,6 +157,15 @@ <c>{skipped,Reason}</c> (where Reason is a user specific term). </p> + <p>The <c>end_per_testcase/2</c> function is called even after a + test case terminates due to a call to <c>ct:abort_current_testcase/1</c>, + or after a timetrap timeout. However, <c>end_per_testcase</c> + will then execute on a different process than the test case + function, and in this situation, <c>end_per_testcase</c> will + not be able to change the reason for test case termination by + returning <c>{fail,Reason}</c>, nor will it be able to save data with + <c>{save_config,Data}</c>.</p> + <p>If <c>init_per_testcase</c> crashes, the test case itself is skipped automatically (so called <em>auto skipped</em>). If <c>init_per_testcase</c> returns a <c>skip</c> tuple, also then will the test case be skipped (so @@ -682,12 +691,33 @@ <c>end_per_suite</c> execute, like test cases, on dedicated Erlang processes. </p> + </section> + <section> + <title>Timetrap timeouts</title> + <marker id="timetraps"></marker> <p>The default time limit for a test case is 30 minutes, unless a - <c>timetrap</c> is specified either by the test case info function - or the <c>suite/0</c> function. - </p> - + <c>timetrap</c> is specified either by the suite info function + or a test case info function. The timetrap timeout value defined + in <c>suite/0</c> is the value that will be used for each test case + in the suite (as well as for the configuration functions + <c>init_per_suite/1</c> and <c>end_per_suite</c>). A timetrap timeout + value set with the test case info function will override the value set + by <c>suite/0</c>, but only for that particular test case.</p> + <p>It is also possible to set/reset a timetrap during test case (or + configuration function) execution. This is done by calling + <c>ct:timetrap/1</c>. This function will cancel the current timetrap + and start a new one.</p> + <p>Timetrap values can be extended with a multiplier value specified at + startup with the <c>multiply_timetraps</c> option. It is also possible + to let Test Server decide to scale up timetrap timeout values + automatically, e.g. if tools such as cover or trace are running during + the test. This feature is disabled by default and can be enabled with + the <c>scale_timetraps</c> start option.</p> + <p>If a test case needs to suspend itself for a time that also gets + multipled by <c>multiply_timetraps</c>, and possibly scaled up if + <c>scale_timetraps</c> is enabled, the function <c>ct:sleep/1</c> + may be called.</p> </section> <section> diff --git a/lib/common_test/priv/Makefile.in b/lib/common_test/priv/Makefile.in index f144847a29..6372bbc8d5 100644 --- a/lib/common_test/priv/Makefile.in +++ b/lib/common_test/priv/Makefile.in @@ -56,8 +56,9 @@ ifneq ($(findstring win32,$(TARGET)),win32) # # Files # -FILES = vts.tool run_test.in -SCRIPTS = install.sh +FILES = +SCRIPTS = +IMAGES = tile1.jpg # # Rules @@ -83,14 +84,12 @@ include $(ERL_TOP)/make/otp_release_targets.mk ifeq ($(XNIX),true) release_spec: opt - $(INSTALL_DIR) $(RELSYSDIR)/priv/bin - $(INSTALL_SCRIPT) $(SCRIPTS) $(RELSYSDIR) - $(INSTALL_DATA) $(FILES) $(RELSYSDIR)/priv + $(INSTALL_DIR) $(RELSYSDIR)/priv + $(INSTALL_DATA) $(FILES) $(IMAGES) $(RELSYSDIR)/priv else release_spec: opt - $(INSTALL_DIR) $(RELSYSDIR)/priv/bin - $(INSTALL_SCRIPT) $(SCRIPTS) $(RELSYSDIR) - $(INSTALL_DATA) $(FILES) $(RELSYSDIR)/priv + $(INSTALL_DIR) $(RELSYSDIR)/priv + $(INSTALL_DATA) $(FILES) $(IMAGES) $(RELSYSDIR)/priv endif release_docs_spec: @@ -105,6 +104,7 @@ else # Files # FILES = vts.tool +IMAGES = tile1.jpg # # Rules @@ -123,8 +123,8 @@ clean: include $(ERL_TOP)/make/otp_release_targets.mk release_spec: opt - $(INSTALL_DIR) $(RELSYSDIR)/priv/bin - $(INSTALL_DATA) $(FILES) $(RELSYSDIR)/priv + $(INSTALL_DIR) $(RELSYSDIR)/priv + $(INSTALL_DATA) $(FILES) $(IMAGES) $(RELSYSDIR)/priv release_docs_spec: diff --git a/lib/common_test/priv/tile1.jpg b/lib/common_test/priv/tile1.jpg Binary files differnew file mode 100644 index 0000000000..8749383716 --- /dev/null +++ b/lib/common_test/priv/tile1.jpg diff --git a/lib/common_test/src/Makefile b/lib/common_test/src/Makefile index e7e2d1275d..027667e6b0 100644 --- a/lib/common_test/src/Makefile +++ b/lib/common_test/src/Makefile @@ -1,19 +1,19 @@ # # %CopyrightBegin% -# -# Copyright Ericsson AB 2003-2009. All Rights Reserved. -# +# +# Copyright Ericsson AB 2003-2010. All Rights Reserved. +# # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in # compliance with the License. You should have received a copy of the # Erlang Public License along with this software. If not, it can be # retrieved online at http://www.erlang.org/. -# +# # Software distributed under the License is distributed on an "AS IS" # basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See # the License for the specific language governing rights and limitations # under the License. -# +# # %CopyrightEnd% # @@ -63,7 +63,11 @@ MODULES= \ ct_telnet_client \ ct_make \ vts \ - unix_telnet + unix_telnet \ + ct_config \ + ct_config_plain \ + ct_config_xml \ + ct_slave TARGET_MODULES= $(MODULES:%=$(EBIN)/%) diff --git a/lib/common_test/src/common_test.app.src b/lib/common_test/src/common_test.app.src index 7b72932ad4..b42173f412 100644 --- a/lib/common_test/src/common_test.app.src +++ b/lib/common_test/src/common_test.app.src @@ -1,19 +1,19 @@ % This is an -*- erlang -*- file. %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2009-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% {application, common_test, @@ -42,10 +42,15 @@ ct_testspec, ct_util, unix_telnet, - vts + vts, + ct_config, + ct_config_plain, + ct_config_xml, + ct_slave ]}, {registered, [ct_logs, ct_util_server, + ct_config_server, ct_make_ref, vts, ct_master, diff --git a/lib/common_test/src/ct.erl b/lib/common_test/src/ct.erl index 8ae041e5b4..8ae175f10d 100644 --- a/lib/common_test/src/ct.erl +++ b/lib/common_test/src/ct.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2003-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -59,18 +59,22 @@ %% Test suite API -export([require/1, require/2, get_config/1, get_config/2, get_config/3, + reload_config/1, log/1, log/2, log/3, print/1, print/2, print/3, pal/1, pal/2, pal/3, fail/1, comment/1, - testcases/2, userdata/2, userdata/3]). + testcases/2, userdata/2, userdata/3, + timetrap/1, sleep/1]). + +%% New API for manipulating with config handlers +-export([add_config/2, remove_config/2]). %% Other interface functions -export([get_status/0, abort_current_testcase/1, encrypt_config_file/2, encrypt_config_file/3, decrypt_config_file/2, decrypt_config_file/3]). - -export([get_target_name/1]). -export([parse_table/1, listenv/1]). @@ -93,7 +97,7 @@ %%% <code>install([{config,["config_node.ctc","config_user.ctc"]}])</code>.</p> %%% %%% <p>Note that this function is automatically run by the -%%% <code>run_test</code> script.</p> +%%% <code>run_test</code> program.</p> install(Opts) -> ct_run:install(Opts). @@ -134,22 +138,30 @@ run(TestDirs) -> %%%----------------------------------------------------------------- %%% @spec run_test(Opts) -> Result %%% Opts = [OptTuples] -%%% OptTuples = {config,CfgFiles} | {dir,TestDirs} | {suite,Suites} | -%%% {testcase,Cases} | {group,Groups} | {spec,TestSpecs} | -%%% {allow_user_terms,Bool} | {logdir,LogDir} | -%%% {silent_connections,Conns} | {cover,CoverSpecFile} | -%%% {step,StepOpts} | {event_handler,EventHandlers} | {include,InclDirs} | -%%% {auto_compile,Bool} | {repeat,N} | {duration,DurTime} | -%%% {until,StopTime} | {force_stop,Bool} | {decrypt,DecryptKeyOrFile} | +%%% OptTuples = {dir,TestDirs} | {suite,Suites} | {group,Groups} | +%%% {testcase,Cases} | {spec,TestSpecs} | {label,Label} | +%%% {config,CfgFiles} | {userconfig, UserConfig} | +%%% {allow_user_terms,Bool} | {logdir,LogDir} | +%%% {silent_connections,Conns} | {stylesheet,CSSFile} | +%%% {cover,CoverSpecFile} | {step,StepOpts} | +%%% {event_handler,EventHandlers} | {include,InclDirs} | +%%% {auto_compile,Bool} | {multiply_timetraps,M} | {scale_timetraps,Bool} | +%%% {repeat,N} | {duration,DurTime} | {until,StopTime} | +%%% {force_stop,Bool} | {decrypt,DecryptKeyOrFile} | %%% {refresh_logs,LogDir} | {basic_html,Bool} -%%% CfgFiles = [string()] | string() %%% TestDirs = [string()] | string() %%% Suites = [string()] | string() %%% Cases = [atom()] | atom() %%% Groups = [atom()] | atom() %%% TestSpecs = [string()] | string() +%%% Label = string() | atom() +%%% CfgFiles = [string()] | string() +%%% UserConfig = [{CallbackMod,CfgStrings}] | {CallbackMod,CfgStrings} +%%% CallbackMod = atom() +%%% CfgStrings = [string()] | string() %%% LogDir = string() %%% Conns = all | [atom()] +%%% CSSFile = string() %%% CoverSpecFile = string() %%% StepOpts = [StepOpt] | [] %%% StepOpt = config | keep_inactive @@ -157,6 +169,7 @@ run(TestDirs) -> %%% EH = atom() | {atom(),InitArgs} | {[atom()],InitArgs} %%% InitArgs = [term()] %%% InclDirs = [string()] | string() +%%% M = integer() %%% N = integer() %%% DurTime = string(HHMMSS) %%% StopTime = string(YYMoMoDDHHMMSS) | string(HHMMSS) @@ -165,11 +178,12 @@ run(TestDirs) -> %%% DecryptFile = string() %%% Result = [TestResult] | {error,Reason} %%% @doc Run tests as specified by the combination of options in <code>Opts</code>. -%%% The options are the same as those used with the <code>run_test</code> script. +%%% The options are the same as those used with the +%%% <seealso marker="run_test#run_test"><code>run_test</code></seealso> program. %%% Note that here a <code>TestDir</code> can be used to point out the path to %%% a <code>Suite</code>. Note also that the option <code>testcase</code> %%% corresponds to the <code>-case</code> option in the <code>run_test</code> -%%% script. Configuration files specified in <code>Opts</code> will be +%%% program. Configuration files specified in <code>Opts</code> will be %%% installed automatically at startup. run_test(Opts) -> ct_run:run_test(Opts). @@ -211,7 +225,7 @@ step(TestDir,Suite,Case,Opts) -> %%% %%% <p>From this mode all test case support functions can be executed %%% directly from the erlang shell. The interactive mode can also be -%%% started from the unix command line with <code>run_test -shell +%%% started from the OS command line with <code>run_test -shell %%% [-config File...]</code>.</p> %%% %%% <p>If any functions using "required config data" (e.g. telnet or @@ -269,7 +283,7 @@ stop_interactive() -> %%% @see get_config/2 %%% @see get_config/3 require(Required) -> - ct_util:require(Required). + ct_config:require(Required). %%%----------------------------------------------------------------- %%% @spec require(Name,Required) -> ok | {error,Reason} @@ -304,19 +318,19 @@ require(Required) -> %%% @see get_config/2 %%% @see get_config/3 require(Name,Required) -> - ct_util:require(Name,Required). + ct_config:require(Name,Required). %%%----------------------------------------------------------------- %%% @spec get_config(Required) -> Value %%% @equiv get_config(Required,undefined,[]) get_config(Required) -> - ct_util:get_config(Required,undefined,[]). + ct_config:get_config(Required,undefined,[]). %%%----------------------------------------------------------------- %%% @spec get_config(Required,Default) -> Value %%% @equiv get_config(Required,Default,[]) get_config(Required,Default) -> - ct_util:get_config(Required,Default,[]). + ct_config:get_config(Required,Default,[]). %%%----------------------------------------------------------------- %%% @spec get_config(Required,Default,Opts) -> ValueOrElement @@ -375,7 +389,26 @@ get_config(Required,Default) -> %%% @see require/1 %%% @see require/2 get_config(Required,Default,Opts) -> - ct_util:get_config(Required,Default,Opts). + ct_config:get_config(Required,Default,Opts). + +%%%----------------------------------------------------------------- +%%% @spec reload_config(Required) -> ValueOrElement +%%% Required = KeyOrName | {KeyOrName,SubKey} +%%% KeyOrName = atom() +%%% SubKey = atom() +%%% ValueOrElement = term() +%%% +%%% @doc Reload config file which contains specified configuration key. +%%% +%%% <p>This function performs updating of the configuration data from which the +%%% given configuration variable was read, and returns the (possibly) new +%%% value of this variable.</p> +%%% <p>Note that if some variables were present in the configuration but are not loaded +%%% using this function, they will be removed from the configuration table together +%%% with their aliases.</p> +%%% +reload_config(Required)-> + ct_config:reload_config(Required). %%%----------------------------------------------------------------- %%% @spec log(Format) -> ok @@ -734,7 +767,7 @@ abort_current_testcase(Reason) -> %%% <p>See the <code>crypto</code> application for details on DES3 %%% encryption/decryption.</p> encrypt_config_file(SrcFileName, EncryptFileName) -> - ct_util:encrypt_config_file(SrcFileName, EncryptFileName). + ct_config:encrypt_config_file(SrcFileName, EncryptFileName). %%%----------------------------------------------------------------- %%% @spec encrypt_config_file(SrcFileName, EncryptFileName, KeyOrFile) -> @@ -754,7 +787,7 @@ encrypt_config_file(SrcFileName, EncryptFileName) -> %%% <p>See the <code>crypto</code> application for details on DES3 %%% encryption/decryption.</p> encrypt_config_file(SrcFileName, EncryptFileName, KeyOrFile) -> - ct_util:encrypt_config_file(SrcFileName, EncryptFileName, KeyOrFile). + ct_config:encrypt_config_file(SrcFileName, EncryptFileName, KeyOrFile). %%%----------------------------------------------------------------- %%% @spec decrypt_config_file(EncryptFileName, TargetFileName) -> @@ -770,7 +803,7 @@ encrypt_config_file(SrcFileName, EncryptFileName, KeyOrFile) -> %%% <code>.ct_config.crypt</code> in the current directory, or the %%% home directory of the user (it is searched for in that order).</p> decrypt_config_file(EncryptFileName, TargetFileName) -> - ct_util:decrypt_config_file(EncryptFileName, TargetFileName). + ct_config:decrypt_config_file(EncryptFileName, TargetFileName). %%%----------------------------------------------------------------- %%% @spec decrypt_config_file(EncryptFileName, TargetFileName, KeyOrFile) -> @@ -785,5 +818,65 @@ decrypt_config_file(EncryptFileName, TargetFileName) -> %%% file contents is saved in the target file. The key must have the %%% the same value as that used for encryption.</p> decrypt_config_file(EncryptFileName, TargetFileName, KeyOrFile) -> - ct_util:decrypt_config_file(EncryptFileName, TargetFileName, KeyOrFile). + ct_config:decrypt_config_file(EncryptFileName, TargetFileName, KeyOrFile). + + +%%%----------------------------------------------------------------- +%%% @spec add_config(Callback, Config) -> ok | {error, Reason} +%%% Callback = atom() +%%% Config = string() +%%% Reason = term() +%%% +%%% @doc <p>This function loads configuration variables using the +%%% given callback module and configuration string. Callback module +%%% should be either loaded or present in the code part. Loaded +%%% configuration variables can later be removed using +%%% <code>remove_config/2</code> function.</p> +add_config(Callback, Config)-> + ct_config:add_config(Callback, Config). +%%%----------------------------------------------------------------- +%%% @spec remove_config(Callback, Config) -> ok +%%% Callback = atom() +%%% Config = string() +%%% Reason = term() +%%% +%%% @doc <p>This function removes configuration variables (together with +%%% their aliases) which were loaded with specified callback module and +%%% configuration string.</p> +remove_config(Callback, Config) -> + ct_config:remove_config(Callback, Config). + +%%%----------------------------------------------------------------- +%%% @spec timetrap(Time) -> ok +%%% Time = {hours,Hours} | {minutes,Mins} | {seconds,Secs} | Millisecs | infinity +%%% Hours = integer() +%%% Mins = integer() +%%% Secs = integer() +%%% Millisecs = integer() | float() +%%% +%%% @doc <p>Use this function to set a new timetrap for the running test case.</p> +timetrap(Time) -> + test_server:timetrap(Time). + +%%%----------------------------------------------------------------- +%%% @spec sleep(Time) -> ok +%%% Time = {hours,Hours} | {minutes,Mins} | {seconds,Secs} | Millisecs | infinity +%%% Hours = integer() +%%% Mins = integer() +%%% Secs = integer() +%%% Millisecs = integer() | float() +%%% +%%% @doc <p>This function, similar to <c>timer:sleep/1</c>, suspends the test +%%% case for specified time. However, this function also multiplies +%%% <c>Time</c> with the 'multiply_timetraps' value (if set) and under +%%% certain circumstances also scales up the time automatically +%%% if 'scale_timetraps' is set to true (default is false).</p> +sleep({hours,Hs}) -> + sleep(trunc(Hs * 1000 * 60 * 60)); +sleep({minutes,Ms}) -> + sleep(trunc(Ms * 1000 * 60)); +sleep({seconds,Ss}) -> + sleep(trunc(Ss * 1000)); +sleep(Time) -> + test_server:adjusted_sleep(Time). diff --git a/lib/common_test/src/ct_config.erl b/lib/common_test/src/ct_config.erl new file mode 100644 index 0000000000..dc6fcc66e5 --- /dev/null +++ b/lib/common_test/src/ct_config.erl @@ -0,0 +1,786 @@ +%%-------------------------------------------------------------------- +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%%---------------------------------------------------------------------- +%% File : ct_config.erl +%% Description : CT module for reading and manipulating of configuration +%% data +%% +%% Created : 15 February 2010 +%%---------------------------------------------------------------------- +-module(ct_config). + +-export([start/1, stop/0]). + +-export([read_config_files/1, + get_config_file_list/1]). + +-export([require/1, require/2]). + +-export([get_config/1, get_config/2, get_config/3, + get_all_config/0]). + +-export([set_default_config/2, set_default_config/3]). + +-export([delete_default_config/1]). + +-export([reload_config/1, update_config/2]). + +-export([release_allocated/0]). + +-export([encrypt_config_file/2, encrypt_config_file/3, + decrypt_config_file/2, decrypt_config_file/3, + get_crypt_key_from_file/0, get_crypt_key_from_file/1]). + +-export([get_ref_from_name/1, get_name_from_ref/1, get_key_from_name/1]). + +-export([check_config_files/1, prepare_config_list/1]). + +-export([add_config/2, remove_config/2]). + +-include("ct_util.hrl"). + +-define(cryptfile, ".ct_config.crypt"). + +-record(ct_conf,{key,value,handler,config,ref,name='_UNDEF',default=false}). + +start(Mode) -> + case whereis(ct_config_server) of + undefined -> + Me = self(), + Pid = spawn_link(fun() -> do_start(Me) end), + receive + {Pid,started} -> Pid; + {Pid,Error} -> exit(Error) + end; + Pid -> + case ct_util:get_mode() of + interactive when Mode==interactive -> + Pid; + interactive -> + {error,interactive_mode}; + _OtherMode -> + Pid + end + end. + +do_start(Parent) -> + process_flag(trap_exit,true), + register(ct_config_server,self()), + ct_util:create_table(?attr_table,bag,#ct_conf.key), + {ok,StartDir} = file:get_cwd(), + Opts = case ct_util:read_opts() of + {ok,Opts1} -> + Opts1; + Error -> + Parent ! {self(),Error}, + exit(Error) + end, + case read_config_files(Opts) of + ok -> + Parent ! {self(),started}, + loop(StartDir); + ReadError -> + Parent ! {self(),ReadError}, + exit(ReadError) + end. + +stop() -> + case whereis(ct_config_server) of + undefined -> ok; + _ -> call({stop}) + end. + +call(Msg) -> + MRef = erlang:monitor(process, whereis(ct_config_server)), + Ref = make_ref(), + ct_config_server ! {Msg,{self(),Ref}}, + receive + {Ref, Result} -> + erlang:demonitor(MRef, [flush]), + Result; + {'DOWN',MRef,process,_,Reason} -> + {error,{ct_util_server_down,Reason}} + end. + +return({To,Ref},Result) -> + To ! {Ref, Result}. + +loop(StartDir) -> + receive + {{require,Name,Tag,SubTags},From} -> + Result = do_require(Name,Tag,SubTags), + return(From,Result), + loop(StartDir); + {{set_default_config,{Config,Scope}},From} -> + set_config(Config,{true,Scope}), + return(From,ok), + loop(StartDir); + {{set_default_config,{Name,Config,Scope}},From} -> + set_config(Name,Config,{true,Scope}), + return(From,ok), + loop(StartDir); + {{delete_default_config,Scope},From} -> + delete_config({true,Scope}), + return(From,ok), + loop(StartDir); + {{update_config,{Name,NewConfig}},From} -> + update_conf(Name,NewConfig), + return(From,ok), + loop(StartDir); + {{reload_config, KeyOrName},From}-> + NewValue = reload_conf(KeyOrName), + return(From, NewValue), + loop(StartDir); + {{stop},From} -> + ets:delete(?attr_table), + file:set_cwd(StartDir), + return(From,ok) + end. + +set_default_config(NewConfig, Scope) -> + call({set_default_config, {NewConfig, Scope}}). + +set_default_config(Name, NewConfig, Scope) -> + call({set_default_config, {Name, NewConfig, Scope}}). + +delete_default_config(Scope) -> + call({delete_default_config, Scope}). + +update_config(Name, Config) -> + call({update_config, {Name, Config}}). + +reload_config(KeyOrName) -> + call({reload_config, KeyOrName}). + +process_default_configs(Opts) -> + case lists:keysearch(config, 1, Opts) of + {value,{_,Files=[File|_]}} when is_list(File) -> + Files; + {value,{_,File=[C|_]}} when is_integer(C) -> + [File]; + {value,{_,[]}} -> + []; + false -> + [] + end. + +process_user_configs(Opts, Acc) -> + case lists:keytake(userconfig, 1, Opts) of + false -> + lists:reverse(Acc); + {value, {userconfig, Config=[{_,_}|_]}, NewOpts} -> + Acc1 = lists:map(fun({_Callback, []}=Cfg) -> + Cfg; + ({Callback, Files=[File|_]}) when is_list(File) -> + {Callback, Files}; + ({Callback, File=[C|_]}) when is_integer(C) -> + {Callback, [File]} + end, Config), + process_user_configs(NewOpts, lists:reverse(Acc1)++Acc); + {value, {userconfig, {Callback, []}}, NewOpts} -> + process_user_configs(NewOpts, [{Callback, []} | Acc]); + {value, {userconfig, {Callback, Files=[File|_]}}, NewOpts} when is_list(File) -> + process_user_configs(NewOpts, [{Callback, Files} | Acc]); + {value, {userconfig, {Callback, File=[C|_]}}, NewOpts} when is_integer(C) -> + process_user_configs(NewOpts, [{Callback, [File]} | Acc]) + end. + +get_config_file_list(Opts) -> + DefaultConfigs = process_default_configs(Opts), + CfgFiles = + if + DefaultConfigs == []-> + []; + true-> + [{?ct_config_txt, DefaultConfigs}] + end ++ + process_user_configs(Opts, []), + CfgFiles. + +read_config_files(Opts) -> + AddCallback = fun(CallBack, []) -> + [{CallBack, []}]; + (CallBack, [F|_]=Files) when is_integer(F) -> + [{CallBack, Files}]; + (CallBack, [F|_]=Files) when is_list(F) -> + lists:map(fun(X) -> {CallBack, X} end, Files) + end, + ConfigFiles = case lists:keyfind(config, 1, Opts) of + {config, ConfigLists}-> + lists:foldr(fun({Callback,Files}, Acc) -> + AddCallback(Callback,Files) ++ Acc + end, + [], + ConfigLists); + false-> + [] + end, + read_config_files_int(ConfigFiles, fun store_config/3). + +read_config_files_int([{Callback, File}|Files], FunToSave) -> + case Callback:read_config(File) of + {ok, Config} -> + FunToSave(Config, Callback, File), + read_config_files_int(Files, FunToSave); + {error, ErrorName, ErrorDetail}-> + {user_error, {ErrorName, File, ErrorDetail}} + end; +read_config_files_int([], _FunToSave) -> + ok. + +store_config(Config, Callback, File) -> + [ets:insert(?attr_table, + #ct_conf{key=Key, + value=Val, + handler=Callback, + config=File, + ref=ct_util:ct_make_ref(), + default=false}) || + {Key,Val} <- Config]. + +keyfindall(Key, Pos, List) -> + [E || E <- List, element(Pos, E) =:= Key]. + +rewrite_config(Config, Callback, File) -> + OldRows = ets:match_object(?attr_table, + #ct_conf{handler=Callback, + config=File,_='_'}), + ets:match_delete(?attr_table, + #ct_conf{handler=Callback, + config=File,_='_'}), + Updater = fun({Key, Value}) -> + case keyfindall(Key, #ct_conf.key, OldRows) of + []-> + ets:insert(?attr_table, + #ct_conf{key=Key, + value=Value, + handler=Callback, + config=File, + ref=ct_util:ct_make_ref()}); + RowsToUpdate -> + Inserter = fun(Row) -> + ets:insert(?attr_table, + Row#ct_conf{value=Value, + ref=ct_util:ct_make_ref()}) + end, + lists:foreach(Inserter, RowsToUpdate) + end + end, + [Updater({Key, Value})||{Key, Value}<-Config]. + +set_config(Config,Default) -> + set_config('_UNDEF',Config,Default). + +set_config(Name,Config,Default) -> + [ets:insert(?attr_table, + #ct_conf{key=Key,value=Val,ref=ct_util:ct_make_ref(), + name=Name,default=Default}) || + {Key,Val} <- Config]. + +get_config(KeyOrName) -> + get_config(KeyOrName,undefined,[]). + +get_config(KeyOrName,Default) -> + get_config(KeyOrName,Default,[]). + +get_config(KeyOrName,Default,Opts) when is_atom(KeyOrName) -> + case lookup_config(KeyOrName) of + [] -> + Default; + [{_Ref,Val}|_] = Vals -> + case {lists:member(all,Opts),lists:member(element,Opts)} of + {true,true} -> + [{KeyOrName,V} || {_R,V} <- lists:sort(Vals)]; + {true,false} -> + [V || {_R,V} <- lists:sort(Vals)]; + {false,true} -> + {KeyOrName,Val}; + {false,false} -> + Val + end + end; + +get_config({KeyOrName,SubKey},Default,Opts) -> + case lookup_config(KeyOrName) of + [] -> + Default; + Vals -> + Vals1 = case [Val || {_Ref,Val} <- lists:sort(Vals)] of + Result=[L|_] when is_list(L) -> + case L of + [{_,_}|_] -> + Result; + _ -> + [] + end; + _ -> + [] + end, + case get_subconfig([SubKey],Vals1,[],Opts) of + {ok,[{_,SubVal}|_]=SubVals} -> + case {lists:member(all,Opts),lists:member(element,Opts)} of + {true,true} -> + [{{KeyOrName,SubKey},Val} || {_,Val} <- SubVals]; + {true,false} -> + [Val || {_SubKey,Val} <- SubVals]; + {false,true} -> + {{KeyOrName,SubKey},SubVal}; + {false,false} -> + SubVal + end; + _ -> + Default + end + end. + +get_subconfig(SubKeys,Values) -> + get_subconfig(SubKeys,Values,[],[]). + +get_subconfig(SubKeys,[Value|Rest],Mapped,Opts) -> + case do_get_config(SubKeys,Value,[]) of + {ok,SubMapped} -> + case lists:member(all,Opts) of + true -> + get_subconfig(SubKeys,Rest,Mapped++SubMapped,Opts); + false -> + {ok,SubMapped} + end; + _Error -> + get_subconfig(SubKeys,Rest,Mapped,Opts) + end; +get_subconfig(SubKeys,[],[],_) -> + {error,{not_available,SubKeys}}; +get_subconfig(_SubKeys,[],Mapped,_) -> + {ok,Mapped}. + +do_get_config([Key|Required],Available,Mapped) -> + case lists:keysearch(Key,1,Available) of + {value,{Key,Value}} -> + NewAvailable = lists:keydelete(Key,1,Available), + NewMapped = [{Key,Value}|Mapped], + do_get_config(Required,NewAvailable,NewMapped); + false -> + {error,{not_available,Key}} + end; +do_get_config([],_Available,Mapped) -> + {ok,lists:reverse(Mapped)}. + +get_all_config() -> + ets:select(?attr_table,[{#ct_conf{name='$1',key='$2',value='$3', + default='$4',_='_'}, + [], + [{{'$1','$2','$3','$4'}}]}]). + +lookup_config(KeyOrName) -> + case lookup_name(KeyOrName) of + [] -> + lookup_key(KeyOrName); + Values -> + Values + end. + +lookup_name(Name) -> + ets:select(?attr_table,[{#ct_conf{ref='$1',value='$2',name=Name,_='_'}, + [], + [{{'$1','$2'}}]}]). +lookup_key(Key) -> + ets:select(?attr_table,[{#ct_conf{key=Key,ref='$1',value='$2',name='_UNDEF',_='_'}, + [], + [{{'$1','$2'}}]}]). + +lookup_handler_for_config({Key, _Subkey}) -> + lookup_handler_for_config(Key); +lookup_handler_for_config(KeyOrName) -> + case lookup_handler_for_name(KeyOrName) of + [] -> + lookup_handler_for_key(KeyOrName); + Values -> + Values + end. + +lookup_handler_for_name(Name) -> + ets:select(?attr_table,[{#ct_conf{handler='$1',config='$2',name=Name,_='_'}, + [], + [{{'$1','$2'}}]}]). + +lookup_handler_for_key(Key) -> + ets:select(?attr_table,[{#ct_conf{handler='$1',config='$2',key=Key,_='_'}, + [], + [{{'$1','$2'}}]}]). + + +update_conf(Name, NewConfig) -> + Old = ets:select(?attr_table,[{#ct_conf{name=Name,_='_'},[],['$_']}]), + lists:foreach(fun(OldElem) -> + NewElem = OldElem#ct_conf{value=NewConfig}, + ets:delete_object(?attr_table, OldElem), + ets:insert(?attr_table, NewElem) + end, Old), + ok. + +reload_conf(KeyOrName) -> + case lookup_handler_for_config(KeyOrName) of + []-> + undefined; + HandlerList-> + HandlerList2 = lists:usort(HandlerList), + read_config_files_int(HandlerList2, fun rewrite_config/3), + get_config(KeyOrName) + end. + +release_allocated() -> + Allocated = ets:select(?attr_table,[{#ct_conf{name='$1',_='_'}, + [{'=/=','$1','_UNDEF'}], + ['$_']}]), + release_allocated(Allocated). +release_allocated([H|T]) -> + ets:delete_object(?attr_table,H), + ets:insert(?attr_table,H#ct_conf{name='_UNDEF'}), + release_allocated(T); +release_allocated([]) -> + ok. + +allocate(Name,Key,SubKeys) -> + case ets:match_object(?attr_table,#ct_conf{key=Key,name='_UNDEF',_='_'}) of + [] -> + {error,{not_available,Key}}; + Available -> + case allocate_subconfig(Name,SubKeys,Available,false) of + ok -> + ok; + Error -> + Error + end + end. + +allocate_subconfig(Name,SubKeys,[C=#ct_conf{value=Value}|Rest],Found) -> + case do_get_config(SubKeys,Value,[]) of + {ok,_SubMapped} -> + ets:insert(?attr_table,C#ct_conf{name=Name}), + allocate_subconfig(Name,SubKeys,Rest,true); + _Error -> + allocate_subconfig(Name,SubKeys,Rest,Found) + end; +allocate_subconfig(_Name,_SubKeys,[],true) -> + ok; +allocate_subconfig(_Name,SubKeys,[],false) -> + {error,{not_available,SubKeys}}. + +delete_config(Default) -> + ets:match_delete(?attr_table,#ct_conf{default=Default,_='_'}), + ok. + +require(Key) when is_atom(Key) -> + require({Key,[]}); +require({Key,SubKeys}) when is_atom(Key) -> + allocate('_UNDEF',Key,to_list(SubKeys)); +require(Key) -> + {error,{invalid,Key}}. + +require(Name,Key) when is_atom(Key) -> + require(Name,{Key,[]}); +require(Name,{Key,SubKeys}) when is_atom(Name), is_atom(Key) -> + call({require,Name,Key,to_list(SubKeys)}); +require(Name,Keys) -> + {error,{invalid,{Name,Keys}}}. + +to_list(X) when is_list(X) -> X; +to_list(X) -> [X]. + +do_require(Name,Key,SubKeys) when is_list(SubKeys) -> + case get_key_from_name(Name) of + {error,_} -> + allocate(Name,Key,SubKeys); + {ok,Key} -> + %% already allocated - check that it has all required subkeys + Vals = [Val || {_Ref,Val} <- lookup_name(Name)], + case get_subconfig(SubKeys,Vals) of + {ok,_SubMapped} -> + ok; + Error -> + Error + end; + {ok,OtherKey} -> + {error,{name_in_use,Name,OtherKey}} + end. + +encrypt_config_file(SrcFileName, EncryptFileName) -> + case get_crypt_key_from_file() of + {error,_} = E -> + E; + Key -> + encrypt_config_file(SrcFileName, EncryptFileName, {key,Key}) + end. + +get_ref_from_name(Name) -> + case ets:select(?attr_table,[{#ct_conf{name=Name,ref='$1',_='_'}, + [], + ['$1']}]) of + [Ref] -> + {ok,Ref}; + _ -> + {error,{no_such_name,Name}} + end. + +get_name_from_ref(Ref) -> + case ets:select(?attr_table,[{#ct_conf{name='$1',ref=Ref,_='_'}, + [], + ['$1']}]) of + [Name] -> + {ok,Name}; + _ -> + {error,{no_such_ref,Ref}} + end. + +get_key_from_name(Name) -> + case ets:select(?attr_table,[{#ct_conf{name=Name,key='$1',_='_'}, + [], + ['$1']}]) of + [Key|_] -> + {ok,Key}; + _ -> + {error,{no_such_name,Name}} + end. + +encrypt_config_file(SrcFileName, EncryptFileName, {file,KeyFile}) -> + case get_crypt_key_from_file(KeyFile) of + {error,_} = E -> + E; + Key -> + encrypt_config_file(SrcFileName, EncryptFileName, {key,Key}) + end; + +encrypt_config_file(SrcFileName, EncryptFileName, {key,Key}) -> + crypto:start(), + {K1,K2,K3,IVec} = make_crypto_key(Key), + case file:read_file(SrcFileName) of + {ok,Bin0} -> + Bin1 = term_to_binary({SrcFileName,Bin0}), + Bin2 = case byte_size(Bin1) rem 8 of + 0 -> Bin1; + N -> list_to_binary([Bin1,random_bytes(8-N)]) + end, + EncBin = crypto:des3_cbc_encrypt(K1, K2, K3, IVec, Bin2), + case file:write_file(EncryptFileName, EncBin) of + ok -> + io:format("~s --(encrypt)--> ~s~n", + [SrcFileName,EncryptFileName]), + ok; + {error,Reason} -> + {error,{Reason,EncryptFileName}} + end; + {error,Reason} -> + {error,{Reason,SrcFileName}} + end. + +decrypt_config_file(EncryptFileName, TargetFileName) -> + case get_crypt_key_from_file() of + {error,_} = E -> + E; + Key -> + decrypt_config_file(EncryptFileName, TargetFileName, {key,Key}) + end. + +decrypt_config_file(EncryptFileName, TargetFileName, {file,KeyFile}) -> + case get_crypt_key_from_file(KeyFile) of + {error,_} = E -> + E; + Key -> + decrypt_config_file(EncryptFileName, TargetFileName, {key,Key}) + end; + +decrypt_config_file(EncryptFileName, TargetFileName, {key,Key}) -> + crypto:start(), + {K1,K2,K3,IVec} = make_crypto_key(Key), + case file:read_file(EncryptFileName) of + {ok,Bin} -> + DecBin = crypto:des3_cbc_decrypt(K1, K2, K3, IVec, Bin), + case catch binary_to_term(DecBin) of + {'EXIT',_} -> + {error,bad_file}; + {_SrcFile,SrcBin} -> + case TargetFileName of + undefined -> + {ok,SrcBin}; + _ -> + case file:write_file(TargetFileName, SrcBin) of + ok -> + io:format("~s --(decrypt)--> ~s~n", + [EncryptFileName,TargetFileName]), + ok; + {error,Reason} -> + {error,{Reason,TargetFileName}} + end + end + end; + {error,Reason} -> + {error,{Reason,EncryptFileName}} + end. + +get_crypt_key_from_file(File) -> + case file:read_file(File) of + {ok,Bin} -> + case catch string:tokens(binary_to_list(Bin), [$\n,$\r]) of + [Key] -> + Key; + _ -> + {error,{bad_crypt_file,File}} + end; + {error,Reason} -> + {error,{Reason,File}} + end. + +get_crypt_key_from_file() -> + CwdFile = filename:join(".",?cryptfile), + {Result,FullName} = + case file:read_file(CwdFile) of + {ok,Bin} -> + {Bin,CwdFile}; + _ -> + case init:get_argument(home) of + {ok,[[Home]]} -> + HomeFile = filename:join(Home,?cryptfile), + case file:read_file(HomeFile) of + {ok,Bin} -> + {Bin,HomeFile}; + _ -> + {{error,no_crypt_file},noent} + end; + _ -> + {{error,no_crypt_file},noent} + end + end, + case FullName of + noent -> + Result; + _ -> + case catch string:tokens(binary_to_list(Result), [$\n,$\r]) of + [Key] -> + io:format("~nCrypt key file: ~s~n", [FullName]), + Key; + _ -> + {error,{bad_crypt_file,FullName}} + end + end. + +make_crypto_key(String) -> + <<K1:8/binary,K2:8/binary>> = First = erlang:md5(String), + <<K3:8/binary,IVec:8/binary>> = erlang:md5([First|lists:reverse(String)]), + {K1,K2,K3,IVec}. + +random_bytes(N) -> + {A,B,C} = now(), + random:seed(A, B, C), + random_bytes_1(N, []). + +random_bytes_1(0, Acc) -> Acc; +random_bytes_1(N, Acc) -> random_bytes_1(N-1, [random:uniform(255)|Acc]). + +check_callback_load(Callback) -> + case code:is_loaded(Callback) of + {file, _Filename}-> + check_exports(Callback); + false-> + case code:load_file(Callback) of + {module, Callback}-> + check_exports(Callback); + {error, Error}-> + {error, Error} + end + end. + +check_exports(Callback) -> + Fs = Callback:module_info(exports), + case {lists:member({check_parameter,1},Fs), + lists:member({read_config,1},Fs)} of + {true, true} -> + {ok, Callback}; + _ -> + {error, missing_callback_functions} + end. + +check_config_files(Configs) -> + ConfigChecker = fun + ({Callback, [F|_R]=Files}) -> + case check_callback_load(Callback) of + {ok, Callback} -> + if is_integer(F) -> + Callback:check_parameter(Files); + is_list(F) -> + lists:map(fun(File) -> + Callback:check_parameter(File) + end, + Files) + end; + {error, Why}-> + {error, {callback, {Callback,Why}}} + end; + ({Callback, []}) -> + case check_callback_load(Callback) of + {ok, Callback}-> + Callback:check_parameter([]); + {error, Why}-> + {error, {callback, {Callback,Why}}} + end + end, + lists:keysearch(error, 1, lists:flatten(lists:map(ConfigChecker, Configs))). + +prepare_user_configs([ConfigString|UserConfigs], Acc, new) -> + prepare_user_configs(UserConfigs, + [{list_to_atom(ConfigString), []}|Acc], + cur); +prepare_user_configs(["and"|UserConfigs], Acc, _) -> + prepare_user_configs(UserConfigs, Acc, new); +prepare_user_configs([ConfigString|UserConfigs], [{LastMod, LastList}|Acc], cur) -> + prepare_user_configs(UserConfigs, + [{LastMod, [ConfigString|LastList]}|Acc], + cur); +prepare_user_configs([], Acc, _) -> + Acc. + +prepare_config_list(Args) -> + ConfigFiles = case lists:keysearch(ct_config, 1, Args) of + {value,{ct_config,Files}}-> + [{?ct_config_txt,[filename:absname(F) || F <- Files]}]; + false-> + [] + end, + UserConfigs = case lists:keysearch(userconfig, 1, Args) of + {value,{userconfig,UserConfigFiles}}-> + prepare_user_configs(UserConfigFiles, [], new); + false-> + [] + end, + ConfigFiles ++ UserConfigs. + +% TODO: add logging of the loaded configuration file to the CT FW log!!! +add_config(Callback, []) -> + read_config_files_int([{Callback, []}], fun store_config/3); +add_config(Callback, [File|_Files]=Config) when is_list(File) -> + lists:foreach(fun(CfgStr) -> + read_config_files_int([{Callback, CfgStr}], fun store_config/3) end, + Config); +add_config(Callback, [C|_]=Config) when is_integer(C) -> + read_config_files_int([{Callback, Config}], fun store_config/3), + ok. + +remove_config(Callback, Config) -> + ets:match_delete(?attr_table, + #ct_conf{handler=Callback, + config=Config,_='_'}), + ok. diff --git a/lib/common_test/src/ct_config_plain.erl b/lib/common_test/src/ct_config_plain.erl new file mode 100644 index 0000000000..3fbc8af9fb --- /dev/null +++ b/lib/common_test/src/ct_config_plain.erl @@ -0,0 +1,109 @@ +%%-------------------------------------------------------------------- +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%%---------------------------------------------------------------------- +%% File : ct_config_plain.erl +%% Description : CT callback module for reading configs from text files +%% +%% Created : 15 February 2010 +%%---------------------------------------------------------------------- +-module(ct_config_plain). +-export([read_config/1, check_parameter/1]). + +read_config(ConfigFile) -> + case file:consult(ConfigFile) of + {ok,Config} -> + {ok, Config}; + {error,enoent} -> + {error, config_file_error, enoent}; + {error,Reason} -> + Key = + case application:get_env(common_test, decrypt) of + {ok,KeyOrFile} -> + case KeyOrFile of + {key,K} -> + K; + {file,F} -> + ct_config:get_crypt_key_from_file(F) + end; + _ -> + ct_config:get_crypt_key_from_file() + end, + case Key of + {error,no_crypt_file} -> + {error, config_file_error, Reason}; + {error,CryptError} -> + {error, decrypt_file_error, CryptError}; + _ when is_list(Key) -> + case ct_config:decrypt_config_file(ConfigFile, undefined, {key,Key}) of + {ok,CfgBin} -> + case read_config_terms(CfgBin) of + {error,ReadFail} -> + {error, config_file_error, ReadFail}; + Config -> + {ok, Config} + end; + {error,DecryptFail} -> + {error, decrypt_config_error, DecryptFail} + end; + _ -> + {error, bad_decrypt_key, Key} + end + end. + +% check if config file exists +check_parameter(File)-> + case filelib:is_file(File) of + true-> + {ok, {file, File}}; + false-> + {error, {nofile, File}} + end. + +read_config_terms(Bin) when is_binary(Bin) -> + case catch binary_to_list(Bin) of + {'EXIT',_} -> + {error,invalid_textfile}; + Lines -> + read_config_terms(Lines) + end; +read_config_terms(Lines) when is_list(Lines) -> + read_config_terms1(erl_scan:tokens([], Lines, 0), 1, [], []). + +read_config_terms1({done,{ok,Ts,EL},Rest}, L, Terms, _) -> + case erl_parse:parse_term(Ts) of + {ok,Term} when Rest == [] -> + lists:reverse([Term|Terms]); + {ok,Term} -> + read_config_terms1(erl_scan:tokens([], Rest, 0), + EL+1, [Term|Terms], Rest); + _ -> + {error,{bad_term,{L,EL}}} + end; +read_config_terms1({done,{eof,_},_}, _, Terms, Rest) when Rest == [] -> + lists:reverse(Terms); +read_config_terms1({done,{eof,EL},_}, L, _, _) -> + {error,{bad_term,{L,EL}}}; +read_config_terms1({done,{error,Info,EL},_}, L, _, _) -> + {error,{Info,{L,EL}}}; +read_config_terms1({more,_}, L, Terms, Rest) -> + case string:tokens(Rest, [$\n,$\r,$\t]) of + [] -> + lists:reverse(Terms); + _ -> + {error,{bad_term,L}} + end. diff --git a/lib/common_test/src/ct_config_xml.erl b/lib/common_test/src/ct_config_xml.erl new file mode 100644 index 0000000000..8a6e75e635 --- /dev/null +++ b/lib/common_test/src/ct_config_xml.erl @@ -0,0 +1,118 @@ +%%-------------------------------------------------------------------- +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%%---------------------------------------------------------------------- +%% File : ct_config_xml.erl +%% Description : CT callback module for reading configs from XML files +%% +%% Created : 16 February 2010 +%%---------------------------------------------------------------------- +-module(ct_config_xml). +-export([read_config/1, check_parameter/1]). + +% read config file +read_config(ConfigFile) -> + case catch do_read_xml_config(ConfigFile) of + {ok, Config}-> + {ok, Config}; + {error, Error, ErroneousString}-> + {error, Error, ErroneousString} + end. + +% check file exists +check_parameter(File)-> + case filelib:is_file(File) of + true-> + {ok, {file, File}}; + false-> + {error, {nofile, File}} + end. + +% actual reading of the config +do_read_xml_config(ConfigFile)-> + case catch xmerl_sax_parser:file(ConfigFile, + [{event_fun, fun event/3}, + {event_state, []}]) of + {ok, EntityList, _}-> + {ok, lists:reverse(transform_entity_list(EntityList))}; + Oops-> + {error, parsing_failed, Oops} + end. + +% event callback for xmerl_sax_parser +event(Event, _LineNo, State) -> + tag(Event, State). + +% document start +tag(startDocument, State) -> + State; + +% start of the config +tag({startElement, _Uri, "config", _QName, _Attributes}, []) -> + [{"config", []}]; + +% start tag +tag({startElement, _Uri, Name, _QName, _Attributes}, Tags) -> + [{Name, []}|Tags]; + +% value +tag({characters, String}, [{Tag, _Value}|Tags]) -> + [{Tag, String}|Tags]; + +% end tag +tag({endElement, _Uri, _Name, _QName}, + [Entity, {PrevEntityTag, PrevEntityValue}|Tags]) -> + NewHead = {PrevEntityTag, [Entity|PrevEntityValue]}, + [NewHead|Tags]; + +% end of the config +tag({endElement, _Uri, "config", _QName}, [{"config", Config}]) -> + Config; + +% end of document, return result +tag(endDocument, {_Tags, Result}) -> + Result; + +% default +tag(_El, State) -> + State. + +% transform of the ugly deeply nested entity list to the key-value "tree" +transform_entity_list(EntityList)-> + lists:map(fun transform_entity/1, EntityList). + +% transform entity from {list(), list()} to {atom(), term()} +transform_entity({Tag, [Value|Rest]}) when + is_tuple(Value)-> + {list_to_atom(Tag), transform_entity_list(lists:reverse([Value|Rest]))}; +transform_entity({Tag, String})-> + case list_to_term(String) of + {ok, Value}-> + {list_to_atom(Tag), Value}; + Error-> + throw(Error) + end. + +% transform a string with Erlang terms +list_to_term(String) -> + {ok, T, _} = erl_scan:string(String++"."), + case catch erl_parse:parse_term(T) of + {ok, Term} -> + {ok, Term}; + Error -> + {error, Error, String} + end. diff --git a/lib/common_test/src/ct_framework.erl b/lib/common_test/src/ct_framework.erl index ed8b564921..f2ca023cff 100644 --- a/lib/common_test/src/ct_framework.erl +++ b/lib/common_test/src/ct_framework.erl @@ -27,8 +27,12 @@ -export([init_tc/3, end_tc/3, get_suite/2, report/2, warn/1]). -export([error_notification/4]). +-export([overview_html_header/1]). + -export([error_in_suite/1, ct_init_per_group/2, ct_end_per_group/2]). +-export([make_all_conf/3, make_conf/5]). + -include("ct_event.hrl"). -include("ct_util.hrl"). @@ -101,7 +105,8 @@ init_tc1(Mod,Func,[Config0],DoInit) when is_list(Config0) -> [{saved_config,{LastFunc,SavedConfig}} | lists:keydelete(saved_config,1,Config0)]; {{LastSuite,InitOrEnd},SavedConfig} when InitOrEnd == init_per_suite ; - InitOrEnd == end_per_suite -> % last suite + InitOrEnd == end_per_suite -> + %% last suite [{saved_config,{LastSuite,SavedConfig}} | lists:keydelete(saved_config,1,Config0)]; undefined -> @@ -113,9 +118,9 @@ init_tc1(Mod,Func,[Config0],DoInit) when is_list(Config0) -> ok; true -> %% delete all default values used in previous suite - ct_util:delete_default_config(suite), + ct_config:delete_default_config(suite), %% release all name -> key bindings (once per suite) - ct_util:release_allocated() + ct_config:release_allocated() end, TestCaseInfo = case catch apply(Mod,Func,[]) of @@ -125,7 +130,7 @@ init_tc1(Mod,Func,[Config0],DoInit) when is_list(Config0) -> %% clear all config data default values set by previous %% testcase info function (these should only survive the %% testcase, not the whole suite) - ct_util:delete_default_config(testcase), + ct_config:delete_default_config(testcase), case add_defaults(Mod,Func,TestCaseInfo,DoInit) of Error = {suite0_failed,_} -> ct_logs:init_tc(), @@ -161,6 +166,7 @@ init_tc2(Mod,Func,SuiteInfo,MergeResult,Config,DoInit) -> _ -> MergeResult end, + %% timetrap must be handled before require MergedInfo1 = timetrap_first(MergedInfo, [], []), %% tell logger to use specified style sheet @@ -244,8 +250,8 @@ add_defaults(Mod,Func,FuncInfo,DoInit) -> _ -> {suite0_failed,bad_return_value} end. - -add_defaults1(_Mod,init_per_suite,[],SuiteInfo,_) -> + +add_defaults1(_Mod,init_per_suite,[],SuiteInfo,_DoInit) -> SuiteInfo; add_defaults1(Mod,Func,FuncInfo,SuiteInfo,DoInit) -> @@ -253,15 +259,27 @@ add_defaults1(Mod,Func,FuncInfo,SuiteInfo,DoInit) -> %% can result in weird behaviour (suite values get overwritten) SuiteReqs = [SDDef || SDDef <- SuiteInfo, - require == element(1,SDDef)], - case [element(2,Clash) || Clash <- SuiteReqs, - true == lists:keymember(element(2,Clash),2,FuncInfo)] of + ((require == element(1,SDDef)) or + (default_config == element(1,SDDef)))], + FuncReqs = + [FIDef || FIDef <- FuncInfo, + require == element(1,FIDef)], + case [element(2,Clash) || Clash <- SuiteReqs, + require == element(1, Clash), + true == lists:keymember(element(2,Clash),2, + FuncReqs)] of [] -> add_defaults2(Mod,Func,FuncInfo,SuiteInfo,SuiteReqs,DoInit); Clashes -> {error,{config_name_already_in_use,Clashes}} end. +add_defaults2(Mod,init_per_suite,IPSInfo,SuiteInfo,SuiteReqs,false) -> + %% not common practise to use a test case info function for + %% init_per_suite (usually handled by suite/0), but let's support + %% it just in case... + add_defaults2(Mod,init_per_suite,IPSInfo,SuiteInfo,SuiteReqs,true); + add_defaults2(_Mod,_Func,FuncInfo,SuiteInfo,_,false) -> %% include require elements from test case info, but not from suite/0 %% (since we've already required those vars) @@ -381,10 +399,10 @@ try_set_default(Name,Key,Info,Where) -> {_,[]} -> no_default; {'_UNDEF',_} -> - [ct_util:set_default_config([CfgVal],Where) || CfgVal <- CfgElems], + [ct_config:set_default_config([CfgVal],Where) || CfgVal <- CfgElems], ok; _ -> - [ct_util:set_default_config(Name,[CfgVal],Where) || CfgVal <- CfgElems], + [ct_config:set_default_config(Name,[CfgVal],Where) || CfgVal <- CfgElems], ok end. @@ -631,12 +649,12 @@ group_or_func(Func, _Config) -> %%% and every test case. If the former, all test cases in the suite %%% should be returned. -get_suite(Mod, all) -> +get_suite(Mod, all) -> case catch apply(Mod, groups, []) of {'EXIT',_} -> get_all(Mod, []); GroupDefs when is_list(GroupDefs) -> - case catch check_groups(Mod, GroupDefs) of + case catch find_groups(Mod, all, all, GroupDefs) of {error,_} = Error -> %% this makes test_server call error_in_suite as first %% (and only) test case so we can report Error properly @@ -651,96 +669,193 @@ get_suite(Mod, all) -> %%!============================================================ %%! Note: The handling of sequences in get_suite/2 and get_all/2 -%%! is deprecated and should be removed after OTP R13! +%%! is deprecated and should be removed at some point... %%!============================================================ -get_suite(Mod, Name) -> - %% Name may be name of a group or a test case. If it's a group, - %% it should be expanded to list of cases (in a conf term) +%% group +get_suite(Mod, Group={conf,Props,_Init,TCs,_End}) -> + Name = proplists:get_value(name, Props), case catch apply(Mod, groups, []) of {'EXIT',_} -> - get_seq(Mod, Name); + [Group]; GroupDefs when is_list(GroupDefs) -> - case catch check_groups(Mod, GroupDefs) of + case catch find_groups(Mod, Name, TCs, GroupDefs) of {error,_} = Error -> %% this makes test_server call error_in_suite as first %% (and only) test case so we can report Error properly [{?MODULE,error_in_suite,[[Error]]}]; + [] -> + {error,{invalid_group_spec,Name}}; ConfTests -> - FindConf = fun({conf,Props,_,_,_}) -> - case proplists:get_value(name, Props) of - Name -> true; - _ -> false - end - end, - case lists:filter(FindConf, ConfTests) of - [] -> % must be a test case - get_seq(Mod, Name); - [ConfTest|_] -> - ConfTest + case lists:member(skipped, Props) of + true -> + %% a *subgroup* specified *only* as skipped (and not + %% as an explicit test) should not be returned, or + %% init/end functions for top groups will be executed + case catch proplists:get_value(name, element(2, hd(ConfTests))) of + Name -> % top group + ConfTests; + _ -> + [] + end; + false -> + ConfTests end end; _ -> E = "Bad return value from "++atom_to_list(Mod)++":groups/0", [{?MODULE,error_in_suite,[[{error,list_to_atom(E)}]]}] - end. + end; -check_groups(_Mod, []) -> - []; -check_groups(Mod, Defs) -> - check_groups(Mod, Defs, Defs, []). +%% testcase +get_suite(Mod, Name) -> + get_seq(Mod, Name). -check_groups(Mod, [TC | Gs], Defs, Levels) when is_atom(TC), length(Levels)>0 -> - [TC | check_groups(Mod, Gs, Defs, Levels)]; +%%%----------------------------------------------------------------- -check_groups(Mod, [{group,SubName} | Gs], Defs, Levels) when is_atom(SubName) -> - case lists:member(SubName, Levels) of - true -> - E = "Cyclic reference to group "++atom_to_list(SubName)++ - " in "++atom_to_list(Mod)++":groups/0", - throw({error,list_to_atom(E)}); - false -> - case find_group(Mod, SubName, Defs) of - {error,_} = Error -> - throw(Error); - G -> - [check_groups(Mod, [G], Defs, Levels) | - check_groups(Mod, Gs, Defs, Levels)] - end +find_groups(Mod, Name, TCs, GroupDefs) -> + Found = find(Mod, Name, TCs, GroupDefs, [], GroupDefs, false), + Trimmed = trim(Found), + delete_subs(Trimmed, Trimmed). + +find(Mod, all, _TCs, [{Name,Props,Tests} | Gs], Known, Defs, _) -> + cyclic_test(Mod, Name, Known), + [make_conf(Mod, Name, Props, + find(Mod, all, all, Tests, [Name | Known], Defs, true)) | + find(Mod, all, all, Gs, [], Defs, true)]; + +find(Mod, Name, TCs, [{Name,Props,Tests} | _Gs], Known, Defs, false) + when is_atom(Name), is_list(Props), is_list(Tests) -> + cyclic_test(Mod, Name, Known), + case TCs of + all -> + [make_conf(Mod, Name, Props, + find(Mod, Name, TCs, Tests, [Name | Known], Defs, true))]; + _ -> + Tests1 = [TC || TC <- TCs, + lists:member(TC, Tests) == true], + [make_conf(Mod, Name, Props, Tests1)] end; -check_groups(Mod, [{Name,Tests} | Gs], Defs, Levels) when is_atom(Name), - is_list(Tests) -> - check_groups(Mod, [{Name,[],Tests} | Gs], Defs, Levels); - -check_groups(Mod, [{Name,Props,Tests} | Gs], Defs, Levels) when is_atom(Name), - is_list(Props), - is_list(Tests) -> - {TestSpec,Levels1} = - case Levels of - [] -> - {check_groups(Mod, Tests, Defs, [Name]),[]}; - _ -> - {check_groups(Mod, Tests, Defs, [Name|Levels]),Levels} - end, - [make_conf(Mod, Name, Props, TestSpec) | - check_groups(Mod, Gs, Defs, Levels1)]; +find(Mod, Name, TCs, [{Name1,Props,Tests} | Gs], Known, Defs, false) + when is_atom(Name1), is_list(Props), is_list(Tests) -> + cyclic_test(Mod, Name1, Known), + [make_conf(Mod, Name1, Props, + find(Mod, Name, TCs, Tests, [Name1 | Known], Defs, false)) | + find(Mod, Name, TCs, Gs, [], Defs, false)]; + +find(Mod, Name, _TCs, [{Name,_Props,_Tests} | _Gs], _Known, _Defs, true) + when is_atom(Name) -> + E = "Duplicate groups named "++atom_to_list(Name)++" in "++ + atom_to_list(Mod)++":groups/0", + throw({error,list_to_atom(E)}); + +find(Mod, Name, all, [{Name1,Props,Tests} | Gs], Known, Defs, true) + when is_atom(Name1), is_list(Props), is_list(Tests) -> + cyclic_test(Mod, Name1, Known), + [make_conf(Mod, Name1, Props, + find(Mod, Name, all, Tests, [Name1 | Known], Defs, true)) | + find(Mod, Name, all, Gs, [], Defs, true)]; + +find(Mod, Name, TCs, [{group,Name1} | Gs], Known, Defs, Found) when is_atom(Name1) -> + find(Mod, Name, TCs, [expand(Mod, Name1, Defs) | Gs], Known, Defs, Found); -check_groups(Mod, [BadTerm | _Gs], _Defs, Levels) -> - Where = if length(Levels) == 0 -> +find(Mod, Name, TCs, [{Name1,Tests} | Gs], Known, Defs, Found) + when is_atom(Name1), is_list(Tests) -> + find(Mod, Name, TCs, [{Name1,[],Tests} | Gs], Known, Defs, Found); + +find(Mod, Name, TCs, [TC | Gs], Known, Defs, false) when is_atom(TC) -> + find(Mod, Name, TCs, Gs, Known, Defs, false); + +find(Mod, Name, TCs, [TC | Gs], Known, Defs, true) when is_atom(TC) -> + [TC | find(Mod, Name, TCs, Gs, Known, Defs, true)]; + +find(Mod, _Name, _TCs, [BadTerm | _Gs], Known, _Defs, _Found) -> + Where = if length(Known) == 0 -> atom_to_list(Mod)++":groups/0"; true -> - "group "++atom_to_list(lists:last(Levels))++ + "group "++atom_to_list(lists:last(Known))++ " in "++atom_to_list(Mod)++":groups/0" end, Term = io_lib:format("~p", [BadTerm]), E = "Bad term "++lists:flatten(Term)++" in "++Where, throw({error,list_to_atom(E)}); -check_groups(_Mod, [], _Defs, _) -> +find(_Mod, _Name, _TCs, [], _Known, _Defs, false) -> + ['$NOMATCH']; + +find(_Mod, _Name, _TCs, [], _Known, _Defs, _Found) -> []. -find_group(Mod, Name, Defs) -> +delete_subs([Conf | Confs], All) -> + All1 = delete_conf(Conf, All), + case is_sub(Conf, All1) of + true -> + delete_subs(Confs, All1); + false -> + delete_subs(Confs, All) + end; + +delete_subs([], All) -> + All. + +delete_conf({conf,Props,_,_,_}, Confs) -> + Name = proplists:get_value(name, Props), + [Conf || Conf = {conf,Props0,_,_,_} <- Confs, + Name =/= proplists:get_value(name, Props0)]. + +is_sub({conf,Props,_,_,_}=Conf, [{conf,_,_,Tests,_} | Confs]) -> + Name = proplists:get_value(name, Props), + case lists:any(fun({conf,Props0,_,_,_}) -> + case proplists:get_value(name, Props0) of + N when N == Name -> + true; + _ -> + false + end; + (_) -> + false + end, Tests) of + true -> + true; + false -> + is_sub(Conf, Tests) or is_sub(Conf, Confs) + end; + +is_sub(Conf, [_TC | Tests]) -> + is_sub(Conf, Tests); + +is_sub(_Conf, []) -> + false. + +trim(['$NOMATCH' | Tests]) -> + trim(Tests); + +trim([{conf,Props,Init,Tests,End} | Confs]) -> + case trim(Tests) of + [] -> + trim(Confs); + Trimmed -> + [{conf,Props,Init,Trimmed,End} | trim(Confs)] + end; + +trim([TC | Tests]) -> + [TC | trim(Tests)]; + +trim([]) -> + []. + +cyclic_test(Mod, Name, Names) -> + case lists:member(Name, Names) of + true -> + E = "Cyclic reference to group "++atom_to_list(Name)++ + " in "++atom_to_list(Mod)++":groups/0", + throw({error,list_to_atom(E)}); + false -> + ok + end. + +expand(Mod, Name, Defs) -> case lists:keysearch(Name, 1, Defs) of {value,Def} -> Def; @@ -750,7 +865,48 @@ find_group(Mod, Name, Defs) -> throw({error,list_to_atom(E)}) end. +make_all_conf(Dir, Mod, _Props) -> + case code:is_loaded(Mod) of + false -> + code:load_abs(filename:join(Dir,atom_to_list(Mod))); + _ -> + ok + end, + make_all_conf(Mod). + +make_all_conf(Mod) -> + case catch apply(Mod, groups, []) of + {'EXIT',_} -> + {error,{invalid_group_definition,Mod}}; + GroupDefs when is_list(GroupDefs) -> + case catch find_groups(Mod, all, all, GroupDefs) of + {error,_} = Error -> + %% this makes test_server call error_in_suite as first + %% (and only) test case so we can report Error properly + [{?MODULE,error_in_suite,[[Error]]}]; + [] -> + {error,{invalid_group_spec,Mod}}; + ConfTests -> + [{conf,Props,Init,all,End} || {conf,Props,Init,_,End} <- ConfTests] + end + end. + +make_conf(Dir, Mod, Name, Props, TestSpec) -> + case code:is_loaded(Mod) of + false -> + code:load_abs(filename:join(Dir,atom_to_list(Mod))); + _ -> + ok + end, + make_conf(Mod, Name, Props, TestSpec). + make_conf(Mod, Name, Props, TestSpec) -> + case code:is_loaded(Mod) of + false -> + code:load_file(Mod); + _ -> + ok + end, {InitConf,EndConf} = case erlang:function_exported(Mod,init_per_group,2) of true -> @@ -761,6 +917,7 @@ make_conf(Mod, Name, Props, TestSpec) -> end, {conf,[{name,Name}|Props],InitConf,TestSpec,EndConf}. +%%%----------------------------------------------------------------- get_all(Mod, ConfTests) -> case catch apply(Mod, all, []) of @@ -1076,4 +1233,31 @@ add_data_dir(File,Config) when is_list(File) -> File end. +%%%----------------------------------------------------------------- +%%% @spec overview_html_header(TestName) -> Header +overview_html_header(TestName) -> + TestName1 = lists:flatten(io_lib:format("~p", [TestName])), + Label = case application:get_env(common_test, test_label) of + {ok,Lbl} when Lbl =/= undefined -> + "<H1><FONT color=\"green\">" ++ Lbl ++ "</FONT></H1>\n"; + _ -> + "" + end, + Bgr = case ct_logs:basic_html() of + true -> + ""; + false -> + CTPath = code:lib_dir(common_test), + TileFile = filename:join(filename:join(CTPath,"priv"),"tile1.jpg"), + " background=\"" ++ TileFile ++ "\"" + end, + + ["<html>\n", + "<head><title>Test ", TestName1, " results</title>\n", + "<meta http-equiv=\"cache-control\" content=\"no-cache\">\n", + "</head>\n", + "<body", Bgr, " bgcolor=\"white\" text=\"black\" ", + "link=\"blue\" vlink=\"purple\" alink=\"red\">\n", + Label, + "<H2>Results from test ", TestName1, "</H2>\n"]. diff --git a/lib/common_test/src/ct_gen_conn.erl b/lib/common_test/src/ct_gen_conn.erl index a31e57c7ea..5aab4dd2dd 100644 --- a/lib/common_test/src/ct_gen_conn.erl +++ b/lib/common_test/src/ct_gen_conn.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2003-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -76,7 +76,7 @@ start(Name,Address,InitData,CallbackMod) -> MRef = erlang:monitor(process,Pid), receive {connected,Pid} -> - erlang:demonitor(MRef), + erlang:demonitor(MRef, [flush]), ct_util:register_connection(Name,Address,CallbackMod,Pid), {ok,Pid}; {Error,Pid} -> @@ -182,7 +182,7 @@ call(Pid,Msg) -> Pid ! {Msg,{self(),Ref}}, receive {Ref, Result} -> - erlang:demonitor(MRef), + erlang:demonitor(MRef, [flush]), case Result of {retry,_Data} -> call(Pid,Result); diff --git a/lib/common_test/src/ct_logs.erl b/lib/common_test/src/ct_logs.erl index bd1a89ae1f..f8ae7202e6 100644 --- a/lib/common_test/src/ct_logs.erl +++ b/lib/common_test/src/ct_logs.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2003-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -36,7 +36,8 @@ -export([make_all_suites_index/1,make_all_runs_index/1]). %% Logging stuff directly from testcase --export([tc_log/3,tc_print/3,tc_pal/3]). +-export([tc_log/3,tc_print/3,tc_pal/3, + basic_html/0]). %% Simulate logger process for use without ct environment running -export([simulate/0]). @@ -57,7 +58,7 @@ -define(table_color2,"#E4F0FE"). -define(table_color3,"#F0F8FF"). --define(testname_width, 70). +-define(testname_width, 60). -define(abs(Name), filename:absname(Name)). @@ -80,7 +81,7 @@ init(Mode) -> MRef = erlang:monitor(process,Pid), receive {started,Pid,Result} -> - erlang:demonitor(MRef), + erlang:demonitor(MRef, [flush]), Result; {'DOWN',MRef,process,_,Reason} -> exit({could_not_start_process,?MODULE,Reason}) @@ -163,7 +164,7 @@ call(Msg) -> ?MODULE ! {Msg,{self(),Ref}}, receive {Ref, Result} -> - erlang:demonitor(MRef), + erlang:demonitor(MRef, [flush]), Result; {'DOWN',MRef,process,_,Reason} -> {error,{process_down,?MODULE,Reason}} @@ -505,7 +506,7 @@ logger_loop(State) -> logger_loop(State); {set_stylesheet,TC,SSFile} -> Fd = State#logger_state.ct_log_fd, - io:format(Fd, "~p uses external style sheet: ~s~n", [TC,SSFile]), + io:format(Fd, "~p loading external style sheet: ~s~n", [TC,SSFile]), logger_loop(State#logger_state{stylesheet=SSFile}); {clear_stylesheet,_} when State#logger_state.stylesheet == undefined -> logger_loop(State); @@ -716,7 +717,7 @@ make_last_run_index1(StartTime,IndexName) -> [Log]; Logs -> case read_totals_file(?totals_name) of - {_Node,Logs0,_Totals} -> + {_Node,_Lbl,Logs0,_Totals} -> insert_dirs(Logs,Logs0); _ -> %% someone deleted the totals file!? @@ -728,10 +729,15 @@ make_last_run_index1(StartTime,IndexName) -> {ok,Bin} -> binary_to_term(Bin); _ -> [] end, - {ok,Index0,Totals} = make_last_run_index(Logs1, index_header(StartTime), + Label = case application:get_env(common_test, test_label) of + {ok,Lbl} -> Lbl; + _ -> undefined + end, + {ok,Index0,Totals} = make_last_run_index(Logs1, + index_header(Label,StartTime), 0, 0, 0, 0, 0, Missing), %% write current Totals to file, later to be used in all_runs log - write_totals_file(?totals_name,Logs1,Totals), + write_totals_file(?totals_name,Label,Logs1,Totals), Index = [Index0|index_footer()], case force_write_file(IndexName, Index) of ok -> @@ -761,7 +767,7 @@ make_last_run_index([Name|Rest], Result, TotSucc, TotFail, UserSkip, AutoSkip, TotNotBuilt, Missing); LastLogDir -> SuiteName = filename:rootname(filename:basename(Name)), - case make_one_index_entry(SuiteName, LastLogDir, false, Missing) of + case make_one_index_entry(SuiteName, LastLogDir, "-", false, Missing) of {Result1,Succ,Fail,USkip,ASkip,NotBuilt} -> %% for backwards compatibility AutoSkip1 = case catch AutoSkip+ASkip of @@ -780,18 +786,18 @@ make_last_run_index([], Result, TotSucc, TotFail, UserSkip, AutoSkip, TotNotBuil {ok, [Result|total_row(TotSucc, TotFail, UserSkip, AutoSkip, TotNotBuilt, false)], {TotSucc,TotFail,UserSkip,AutoSkip,TotNotBuilt}}. -make_one_index_entry(SuiteName, LogDir, All, Missing) -> +make_one_index_entry(SuiteName, LogDir, Label, All, Missing) -> case count_cases(LogDir) of {Succ,Fail,UserSkip,AutoSkip} -> NotBuilt = not_built(SuiteName, LogDir, All, Missing), - NewResult = make_one_index_entry1(SuiteName, LogDir, Succ, Fail, + NewResult = make_one_index_entry1(SuiteName, LogDir, Label, Succ, Fail, UserSkip, AutoSkip, NotBuilt, All), {NewResult,Succ,Fail,UserSkip,AutoSkip,NotBuilt}; error -> error end. -make_one_index_entry1(SuiteName, Link, Success, Fail, UserSkip, AutoSkip, +make_one_index_entry1(SuiteName, Link, Label, Success, Fail, UserSkip, AutoSkip, NotBuilt, All) -> LogFile = filename:join(Link, ?suitelog_name ++ ".html"), CrashDumpName = SuiteName ++ "_erl_crash.dump", @@ -803,7 +809,7 @@ make_one_index_entry1(SuiteName, Link, Success, Fail, UserSkip, AutoSkip, false -> "" end, - {Timestamp,Node,AllInfo} = + {Lbl,Timestamp,Node,AllInfo} = case All of {true,OldRuns} -> [_Prefix,NodeOrDate|_] = string:tokens(Link,"."), @@ -811,20 +817,21 @@ make_one_index_entry1(SuiteName, Link, Success, Fail, UserSkip, AutoSkip, 0 -> "-"; _ -> NodeOrDate end, - N = ["<TD ALIGN=right>",Node1,"</TD>\n"], + N = ["<TD ALIGN=right><FONT SIZE=-1>",Node1,"</FONT></TD>\n"], CtRunDir = filename:dirname(filename:dirname(Link)), - T = ["<TD>",timestamp(CtRunDir),"</TD>\n"], + L = ["<TD ALIGN=center><FONT SIZE=-1><B>",Label,"</FONT></B></TD>\n"], + T = ["<TD><FONT SIZE=-1>",timestamp(CtRunDir),"</FONT></TD>\n"], CtLogFile = filename:join(CtRunDir,?ct_log_name), OldRunsLink = case OldRuns of [] -> "none"; _ -> "<A HREF=\""++?all_runs_name++"\">Old Runs</A>" end, - A=["<TD><A HREF=\"",CtLogFile,"\">CT Log</A></TD>\n", - "<TD>",OldRunsLink,"</TD>\n"], - {T,N,A}; + A=["<TD><FONT SIZE=-1><A HREF=\"",CtLogFile,"\">CT Log</A></FONT></TD>\n", + "<TD><FONT SIZE=-1>",OldRunsLink,"</FONT></TD>\n"], + {L,T,N,A}; false -> - {"","",""} + {"","","",""} end, NotBuiltStr = if NotBuilt == 0 -> @@ -851,7 +858,8 @@ make_one_index_entry1(SuiteName, Link, Success, Fail, UserSkip, AutoSkip, {UserSkip+AutoSkip,integer_to_list(UserSkip),ASStr} end, ["<TR valign=top>\n", - "<TD><A HREF=\"",LogFile,"\">",SuiteName,"</A>",CrashDumpLink,"</TD>\n", + "<TD><FONT SIZE=-1><A HREF=\"",LogFile,"\">",SuiteName,"</A>",CrashDumpLink,"</FONT></TD>\n", + Lbl, Timestamp, "<TD ALIGN=right>",integer_to_list(Success),"</TD>\n", "<TD ALIGN=right>",FailStr,"</TD>\n", @@ -862,12 +870,14 @@ make_one_index_entry1(SuiteName, Link, Success, Fail, UserSkip, AutoSkip, AllInfo, "</TR>\n"]. total_row(Success, Fail, UserSkip, AutoSkip, NotBuilt, All) -> - {TimestampCell,AllInfo} = + {Label,TimestampCell,AllInfo} = case All of - true -> - {"<TD> </TD>\n","<TD> </TD>\n<TD> </TD>\n"}; + true -> + {"<TD> </TD>\n", + "<TD> </TD>\n", + "<TD> </TD>\n<TD> </TD>\n"}; false -> - {"",""} + {"","",""} end, {AllSkip,UserSkipStr,AutoSkipStr} = @@ -877,6 +887,7 @@ total_row(Success, Fail, UserSkip, AutoSkip, NotBuilt, All) -> end, ["<TR valign=top>\n", "<TD><B>Total</B></TD>", + Label, TimestampCell, "<TD ALIGN=right><B>",integer_to_list(Success),"<B></TD>\n", "<TD ALIGN=right><B>",integer_to_list(Fail),"<B></TD>\n", @@ -937,13 +948,21 @@ term_to_text(Term) -> %%% Headers and footers. -index_header(StartTime) -> - [header("Test Results " ++ format_time(StartTime)) | +index_header(Label, StartTime) -> + Head = + case Label of + undefined -> + header("Test Results", format_time(StartTime)); + _ -> + header("Test Results for \"" ++ Label ++ "\"", + format_time(StartTime)) + end, + [Head | ["<CENTER>\n", "<P><A HREF=\"",?ct_log_name,"\">Common Test Framework Log</A></P>", "<TABLE border=\"3\" cellpadding=\"5\" " "BGCOLOR=\"",?table_color3,"\">\n" - "<th><B>Name</B></th>\n", + "<th><B>Test Name</B></th>\n", "<th><font color=\"",?table_color3,"\">_</font>Ok" "<font color=\"",?table_color3,"\">_</font></th>\n" "<th>Failed</th>\n", @@ -952,13 +971,17 @@ index_header(StartTime) -> "\n"]]. all_suites_index_header() -> + {ok,Cwd} = file:get_cwd(), + LogDir = filename:basename(Cwd), + AllRuns = "All test runs in \"" ++ LogDir ++ "\"", [header("Test Results") | ["<CENTER>\n", - "<A HREF=\"",?all_runs_name,"\">All Test Runs in this directory</A>\n", + "<A HREF=\"",?all_runs_name,"\">",AllRuns,"</A>\n", "<br><br>\n", "<TABLE border=\"3\" cellpadding=\"5\" " "BGCOLOR=\"",?table_color2,"\">\n" - "<th>Name</th>\n", + "<th>Test Name</th>\n", + "<th>Label</th>\n", "<th>Test Run Started</th>\n", "<th><font color=\"",?table_color2,"\">_</font>Ok" "<font color=\"",?table_color2,"\">_</font></th>\n" @@ -971,13 +994,17 @@ all_suites_index_header() -> "\n"]]. all_runs_header() -> - [header("All test runs in current directory") | + {ok,Cwd} = file:get_cwd(), + LogDir = filename:basename(Cwd), + Title = "All test runs in \"" ++ LogDir ++ "\"", + [header(Title) | ["<CENTER><TABLE border=\"3\" cellpadding=\"5\" " "BGCOLOR=\"",?table_color1,"\">\n" "<th><B>History</B></th>\n" "<th><B>Node</B></th>\n" + "<th><B>Label</B></th>\n" "<th>Tests</th>\n" - "<th><B>Names</B></th>\n" + "<th><B>Test Names</B></th>\n" "<th>Total</th>\n" "<th><font color=\"",?table_color1,"\">_</font>Ok" "<font color=\"",?table_color1,"\">_</font></th>\n" @@ -987,12 +1014,23 @@ all_runs_header() -> "\n"]]. header(Title) -> + header1(Title, ""). +header(Title, SubTitle) -> + header1(Title, SubTitle). + +header1(Title, SubTitle) -> + SubTitleHTML = if SubTitle =/= "" -> + ["<CENTER>\n", + "<H2>" ++ SubTitle ++ "</H2>\n", + "</CENTER>\n<BR>\n"]; + true -> "<BR>\n" + end, ["<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n" "<!-- autogenerated by '"++atom_to_list(?MODULE)++"'. -->\n" "<HTML>\n", "<HEAD>\n", - "<TITLE>" ++ Title ++ "</TITLE>\n", + "<TITLE>" ++ Title ++ " " ++ SubTitle ++ "</TITLE>\n", "<META HTTP-EQUIV=\"CACHE-CONTROL\" CONTENT=\"NO-CACHE\">\n", "</HEAD>\n", @@ -1004,6 +1042,7 @@ header(Title) -> "<CENTER>\n", "<H1>" ++ Title ++ "</H1>\n", "</CENTER>\n", + SubTitleHTML, "<!-- ---- CONTENT ---- -->\n"]. @@ -1013,19 +1052,28 @@ index_footer() -> footer() -> ["<P><CENTER>\n" - "<HR>\n" - "<P><FONT SIZE=-1>\n" - "Copyright © ", year(), - " <A HREF=\"http://erlang.ericsson.se\">Open Telecom Platform</A><BR>\n" - "Updated: <!date>", current_time(), "<!/date><BR>\n" - "</FONT>\n" - "</CENTER>\n" - "</body>\n"]. + "<BR><BR>\n" + "<HR>\n" + "<P><FONT SIZE=-1>\n" + "Copyright © ", year(), + " <A HREF=\"http://erlang.ericsson.se\">Open Telecom Platform</A><BR>\n" + "Updated: <!date>", current_time(), "<!/date><BR>\n" + "</FONT>\n" + "</CENTER>\n" + "</body>\n"]. body_tag() -> - "<body bgcolor=\"#FFFFFF\" text=\"#000000\" link=\"#0000FF\"" - "vlink=\"#800080\" alink=\"#FF0000\">\n". + case basic_html() of + true -> + "<body bgcolor=\"#FFFFFF\" text=\"#000000\" link=\"#0000FF\" " + "vlink=\"#800080\" alink=\"#FF0000\">\n"; + false -> + CTPath = code:lib_dir(common_test), + TileFile = filename:join(filename:join(CTPath,"priv"),"tile1.jpg"), + "<body background=\"" ++ TileFile ++ "\" bgcolor=\"#FFFFFF\" text=\"#000000\" link=\"#0000FF\" " + "vlink=\"#800080\" alink=\"#FF0000\">\n" + end. current_time() -> format_time(calendar:local_time()). @@ -1217,7 +1265,7 @@ runentry(Dir, BasicHtml) -> TotalsFile = filename:join(Dir,?totals_name), TotalsStr = case read_totals_file(TotalsFile) of - {Node,Logs,{TotSucc,TotFail,UserSkip,AutoSkip,NotBuilt}} -> + {Node,Label,Logs,{TotSucc,TotFail,UserSkip,AutoSkip,NotBuilt}} -> TotFailStr = if TotFail > 0 -> ["<FONT color=\"red\">", @@ -1263,6 +1311,7 @@ runentry(Dir, BasicHtml) -> end, Total = TotSucc+TotFail+AllSkip, A = ["<TD ALIGN=center><FONT SIZE=-1>",Node,"</FONT></TD>\n", + "<TD ALIGN=center><FONT SIZE=-1><B>",Label,"</B></FONT></TD>\n", "<TD ALIGN=right>",NoOfTests,"</TD>\n"], B = if BasicHtml -> ["<TD ALIGN=center><FONT SIZE=-1>",TestNamesTrunc,"</FONT></TD>\n"]; @@ -1283,17 +1332,19 @@ runentry(Dir, BasicHtml) -> end, Index = filename:join(Dir,?index_name), ["<TR>\n" - "<TD><A HREF=\"",Index,"\">",timestamp(Dir),"</A>",TotalsStr,"</TD>\n" + "<TD><FONT SIZE=-1><A HREF=\"",Index,"\">",timestamp(Dir),"</A>",TotalsStr,"</FONT></TD>\n" "</TR>\n"]. -write_totals_file(Name,Logs,Totals) -> +write_totals_file(Name,Label,Logs,Totals) -> AbsName = ?abs(Name), notify_and_lock_file(AbsName), force_write_file(AbsName, term_to_binary({atom_to_list(node()), - Logs,Totals})), + Label,Logs,Totals})), notify_and_unlock_file(AbsName). +%% this function needs to convert from old formats to new so that old +%% test results (prev ct versions) can be listed together with new read_totals_file(Name) -> AbsName = ?abs(Name), notify_and_lock_file(AbsName), @@ -1303,12 +1354,23 @@ read_totals_file(Name) -> case catch binary_to_term(Bin) of {'EXIT',_Reason} -> % corrupt file {"-",[],undefined}; - R = {Node,Ls,Tot} -> + {Node,Label,Ls,Tot} -> % all info available + Label1 = case Label of + undefined -> "-"; + _ -> Label + end, case Tot of - {_,_,_,_,_} -> % latest format - R; + {_Ok,_Fail,_USkip,_ASkip,_NoBuild} -> % latest format + {Node,Label1,Ls,Tot}; {TotSucc,TotFail,AllSkip,NotBuilt} -> - {Node,Ls,{TotSucc,TotFail,AllSkip,undefined,NotBuilt}} + {Node,Label1,Ls,{TotSucc,TotFail,AllSkip,undefined,NotBuilt}} + end; + {Node,Ls,Tot} -> % no label found + case Tot of + {_Ok,_Fail,_USkip,_ASkip,_NoBuild} -> % latest format + {Node,"-",Ls,Tot}; + {TotSucc,TotFail,AllSkip,NotBuilt} -> + {Node,"-",Ls,{TotSucc,TotFail,AllSkip,undefined,NotBuilt}} end; %% for backwards compatibility {Ls,Tot} -> {"-",Ls,Tot}; @@ -1411,7 +1473,7 @@ make_all_suites_index1(When,AllSuitesLogDirs) -> make_all_suites_index2(IndexName,AllSuitesLogDirs) -> {ok,Index0,_Totals} = make_all_suites_index3(AllSuitesLogDirs, all_suites_index_header(), - 0, 0, 0, 0, 0), + 0, 0, 0, 0, 0, []), Index = [Index0|index_footer()], case force_write_file(IndexName, Index) of ok -> @@ -1421,14 +1483,25 @@ make_all_suites_index2(IndexName,AllSuitesLogDirs) -> end. make_all_suites_index3([{SuiteName,[LastLogDir|OldDirs]}|Rest], - Result, TotSucc, TotFail, UserSkip, AutoSkip, TotNotBuilt) -> + Result, TotSucc, TotFail, UserSkip, AutoSkip, TotNotBuilt, + Labels) -> [EntryDir|_] = filename:split(LastLogDir), Missing = case file:read_file(filename:join(EntryDir,?missing_suites_info)) of {ok,Bin} -> binary_to_term(Bin); _ -> [] end, - case make_one_index_entry(SuiteName, LastLogDir, {true,OldDirs}, Missing) of + {Label,Labels1} = + case proplists:get_value(EntryDir, Labels) of + undefined -> + case read_totals_file(filename:join(EntryDir,?totals_name)) of + {_,Lbl,_,_} -> {Lbl,[{EntryDir,Lbl}|Labels]}; + _ -> {"-",[{EntryDir,"-"}|Labels]} + end; + Lbl -> + {Lbl,Labels} + end, + case make_one_index_entry(SuiteName, LastLogDir, Label, {true,OldDirs}, Missing) of {Result1,Succ,Fail,USkip,ASkip,NotBuilt} -> %% for backwards compatibility AutoSkip1 = case catch AutoSkip+ASkip of @@ -1437,13 +1510,13 @@ make_all_suites_index3([{SuiteName,[LastLogDir|OldDirs]}|Rest], end, make_all_suites_index3(Rest, [Result|Result1], TotSucc+Succ, TotFail+Fail, UserSkip+USkip, AutoSkip1, - TotNotBuilt+NotBuilt); + TotNotBuilt+NotBuilt,Labels1); error -> make_all_suites_index3(Rest, Result, TotSucc, TotFail, - UserSkip, AutoSkip, TotNotBuilt) + UserSkip, AutoSkip, TotNotBuilt,Labels1) end; make_all_suites_index3([], Result, TotSucc, TotFail, UserSkip, AutoSkip, - TotNotBuilt) -> + TotNotBuilt,_) -> {ok, [Result|total_row(TotSucc, TotFail, UserSkip, AutoSkip, TotNotBuilt,true)], {TotSucc,TotFail,UserSkip,AutoSkip,TotNotBuilt}}. diff --git a/lib/common_test/src/ct_master.erl b/lib/common_test/src/ct_master.erl index 7eb2c3cfef..42e4cf08f4 100644 --- a/lib/common_test/src/ct_master.erl +++ b/lib/common_test/src/ct_master.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2006-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2006-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -28,7 +28,7 @@ -export([abort/0,abort/1,progress/0]). --export([init_master/6, init_node_ctrl/3]). +-export([init_master/7, init_node_ctrl/3]). -export([status/2]). @@ -49,7 +49,8 @@ %%% OptTuples = {config,CfgFiles} | {dir,TestDirs} | {suite,Suites} | %%% {testcase,Cases} | {spec,TestSpecs} | {allow_user_terms,Bool} | %%% {logdir,LogDir} | {event_handler,EventHandlers} | -%%% {silent_connections,Conns} | {cover,CoverSpecFile} +%%% {silent_connections,Conns} | {cover,CoverSpecFile} | +%%% {userconfig, UserCfgFiles} %%% CfgFiles = string() | [string()] %%% TestDirs = string() | [string()] %%% Suites = atom() | [atom()] @@ -98,11 +99,14 @@ run([TS|TestSpecs],AllowUserTerms,InclNodes,ExclNodes) when is_list(TS), {error,Reason} -> {error,Reason}; TSRec=#testspec{logdir=AllLogDirs, - config=AllCfgFiles, + config=StdCfgFiles, + userconfig=UserCfgFiles, + init=AllInitOpts, event_handler=AllEvHs} -> + AllCfgFiles = {StdCfgFiles, UserCfgFiles}, RunSkipPerNode = ct_testspec:prepare_tests(TSRec), RunSkipPerNode2 = exclude_nodes(ExclNodes,RunSkipPerNode), - run_all(RunSkipPerNode2,AllLogDirs,AllCfgFiles,AllEvHs,[],[],TS1) + run_all(RunSkipPerNode2,AllLogDirs,AllCfgFiles,AllEvHs,[],[],AllInitOpts,TS1) end, [{TS,Result} | run(TestSpecs,AllowUserTerms,InclNodes,ExclNodes)]; run([],_,_,_) -> @@ -157,10 +161,13 @@ run_on_node([TS|TestSpecs],AllowUserTerms,Node) when is_list(TS),is_atom(Node) - {error,Reason} -> {error,Reason}; TSRec=#testspec{logdir=AllLogDirs, - config=AllCfgFiles, + config=StdCfgFiles, + init=AllInitOpts, + userconfig=UserCfgFiles, event_handler=AllEvHs} -> + AllCfgFiles = {StdCfgFiles, UserCfgFiles}, {Run,Skip} = ct_testspec:prepare_tests(TSRec,Node), - run_all([{Node,Run,Skip}],AllLogDirs,AllCfgFiles,AllEvHs,[],[],TS1) + run_all([{Node,Run,Skip}],AllLogDirs,AllCfgFiles,AllEvHs,[],[],AllInitOpts,TS1) end, [{TS,Result} | run_on_node(TestSpecs,AllowUserTerms,Node)]; run_on_node([],_,_) -> @@ -180,7 +187,9 @@ run_on_node(TestSpecs,Node) -> -run_all([{Node,Run,Skip}|Rest],AllLogDirs,AllCfgFiles,AllEvHs,NodeOpts,LogDirs,Specs) -> +run_all([{Node,Run,Skip}|Rest],AllLogDirs, + {AllStdCfgFiles, AllUserCfgFiles}=AllCfgFiles, + AllEvHs,NodeOpts,LogDirs,InitOptions,Specs) -> LogDir = lists:foldl(fun({N,Dir},_Found) when N == Node -> Dir; @@ -191,29 +200,36 @@ run_all([{Node,Run,Skip}|Rest],AllLogDirs,AllCfgFiles,AllEvHs,NodeOpts,LogDirs,S (_Dir,Found) -> Found end,".",AllLogDirs), - CfgFiles = + + StdCfgFiles = lists:foldr(fun({N,F},Fs) when N == Node -> [F|Fs]; ({_N,_F},Fs) -> Fs; (F,Fs) -> [F|Fs] - end,[],AllCfgFiles), + end,[],AllStdCfgFiles), + UserCfgFiles = + lists:foldr(fun({N,F},Fs) when N == Node -> [{userconfig, F}|Fs]; + ({_N,_F},Fs) -> Fs; + (F,Fs) -> [{userconfig, F}|Fs] + end,[],AllUserCfgFiles), EvHs = lists:foldr(fun({N,H,A},Hs) when N == Node -> [{H,A}|Hs]; ({_N,_H,_A},Hs) -> Hs; ({H,A},Hs) -> [{H,A}|Hs] end,[],AllEvHs), + NO = {Node,[{prepared_tests,{Run,Skip},Specs}, {logdir,LogDir}, - {config,CfgFiles}, - {event_handler,EvHs}]}, - run_all(Rest,AllLogDirs,AllCfgFiles,AllEvHs,[NO|NodeOpts],[LogDir|LogDirs],Specs); -run_all([],AllLogDirs,_,AllEvHs,NodeOpts,LogDirs,Specs) -> + {config,StdCfgFiles}, + {event_handler,EvHs}] ++ UserCfgFiles}, + run_all(Rest,AllLogDirs,AllCfgFiles,AllEvHs,[NO|NodeOpts],[LogDir|LogDirs],InitOptions,Specs); +run_all([],AllLogDirs,_,AllEvHs,NodeOpts,LogDirs,InitOptions,Specs) -> Handlers = [{H,A} || {Master,H,A} <- AllEvHs, Master == master], MasterLogDir = case lists:keysearch(master,1,AllLogDirs) of {value,{_,Dir}} -> Dir; false -> "." end, log(tty,"Master Logdir","~s",[MasterLogDir]), - start_master(lists:reverse(NodeOpts),Handlers,MasterLogDir,LogDirs,Specs), + start_master(lists:reverse(NodeOpts),Handlers,MasterLogDir,LogDirs,InitOptions,Specs), ok. @@ -251,17 +267,17 @@ progress() -> %%% MASTER, runs on central controlling node. %%%----------------------------------------------------------------- start_master(NodeOptsList) -> - start_master(NodeOptsList,[],".",[],[]). + start_master(NodeOptsList,[],".",[],[],[]). -start_master(NodeOptsList,EvHandlers,MasterLogDir,LogDirs,Specs) -> +start_master(NodeOptsList,EvHandlers,MasterLogDir,LogDirs,InitOptions,Specs) -> Master = spawn_link(?MODULE,init_master,[self(),NodeOptsList,EvHandlers, - MasterLogDir,LogDirs,Specs]), + MasterLogDir,LogDirs,InitOptions,Specs]), receive {Master,Result} -> Result end. %%% @hidden -init_master(Parent,NodeOptsList,EvHandlers,MasterLogDir,LogDirs,Specs) -> +init_master(Parent,NodeOptsList,EvHandlers,MasterLogDir,LogDirs,InitOptions,Specs) -> case whereis(ct_master) of undefined -> register(ct_master,self()), @@ -314,10 +330,10 @@ init_master(Parent,NodeOptsList,EvHandlers,MasterLogDir,LogDirs,Specs) -> Pid when is_pid(Pid) -> ok end, - init_master1(Parent,NodeOptsList,LogDirs). + init_master1(Parent,NodeOptsList,InitOptions,LogDirs). -init_master1(Parent,NodeOptsList,LogDirs) -> - {Inaccessible,NodeOptsList1} = ping_nodes(NodeOptsList,[],[]), +init_master1(Parent,NodeOptsList,InitOptions,LogDirs) -> + {Inaccessible,NodeOptsList1,InitOptions1} = init_nodes(NodeOptsList,InitOptions), case Inaccessible of [] -> init_master2(Parent,NodeOptsList,LogDirs); @@ -331,7 +347,7 @@ init_master1(Parent,NodeOptsList,LogDirs) -> "Proceeding without: ~p",[Inaccessible]), init_master2(Parent,NodeOptsList1,LogDirs); "r\n" -> - init_master1(Parent,NodeOptsList,LogDirs); + init_master1(Parent,NodeOptsList,InitOptions1,LogDirs); _ -> log(html,"Aborting Tests","",[]), ct_master_event:stop(), @@ -542,6 +558,9 @@ get_pid(Node,NodeCtrlPids) -> undefined end. +ping_nodes(NodeOptions)-> + ping_nodes(NodeOptions, [], []). + ping_nodes([NO={Node,_Opts}|NOs],Inaccessible,NodeOpts) -> case net_adm:ping(Node) of pong -> @@ -678,13 +697,80 @@ call(Pid,Msg) -> {'DOWN', Ref, _, _, _} -> {error,master_died} end, - erlang:demonitor(Ref), + erlang:demonitor(Ref, [flush]), Return. reply(Result,To) -> To ! {self(),Result}, ok. +init_nodes(NodeOptions, InitOptions)-> + ping_nodes(NodeOptions), + start_nodes(InitOptions), + eval_on_nodes(InitOptions), + {Inaccessible, NodeOptions1}=ping_nodes(NodeOptions), + InitOptions1 = filter_accessible(InitOptions, Inaccessible), + {Inaccessible, NodeOptions1, InitOptions1}. + +% only nodes which are inaccessible now, should be initiated later +filter_accessible(InitOptions, Inaccessible)-> + [{Node,Option}||{Node,Option}<-InitOptions, lists:member(Node, Inaccessible)]. + +start_nodes(InitOptions)-> + lists:foreach(fun({NodeName, Options})-> + [NodeS,HostS]=string:tokens(atom_to_list(NodeName), "@"), + Node=list_to_atom(NodeS), + Host=list_to_atom(HostS), + HasNodeStart = lists:keymember(node_start, 1, Options), + IsAlive = lists:member(NodeName, nodes()), + case {HasNodeStart, IsAlive} of + {false, false}-> + io:format("WARNING: Node ~p is not alive but has no node_start option~n", [NodeName]); + {false, true}-> + io:format("Node ~p is alive~n", [NodeName]); + {true, false}-> + {node_start, NodeStart} = lists:keyfind(node_start, 1, Options), + {value, {callback_module, Callback}, NodeStart2}= + lists:keytake(callback_module, 1, NodeStart), + case Callback:start(Host, Node, NodeStart2) of + {ok, NodeName} -> + io:format("Node ~p started successfully with callback ~p~n", [NodeName,Callback]); + {error, Reason, _NodeName} -> + io:format("Failed to start node ~p with callback ~p! Reason: ~p~n", [NodeName, Callback, Reason]) + end; + {true, true}-> + io:format("WARNING: Node ~p is alive but has node_start option~n", [NodeName]) + end + end, + InitOptions). + +eval_on_nodes(InitOptions)-> + lists:foreach(fun({NodeName, Options})-> + HasEval = lists:keymember(eval, 1, Options), + IsAlive = lists:member(NodeName, nodes()), + case {HasEval, IsAlive} of + {false,_}-> + ok; + {true,false}-> + io:format("WARNING: Node ~p is not alive but has eval option ~n", [NodeName]); + {true,true}-> + {eval, MFAs} = lists:keyfind(eval, 1, Options), + evaluate(NodeName, MFAs) + end + end, + InitOptions). + +evaluate(Node, [{M,F,A}|MFAs])-> + case rpc:call(Node, M, F, A) of + {badrpc,Reason}-> + io:format("WARNING: Failed to call ~p:~p/~p on node ~p due to ~p~n", [M,F,length(A),Node,Reason]); + Result-> + io:format("Called ~p:~p/~p on node ~p, result: ~p~n", [M,F,length(A),Node,Result]) + end, + evaluate(Node, MFAs); +evaluate(_Node, [])-> + ok. + %cast(Msg) -> % cast(whereis(ct_master),Msg). diff --git a/lib/common_test/src/ct_master_logs.erl b/lib/common_test/src/ct_master_logs.erl index 63f60b1182..244faace06 100644 --- a/lib/common_test/src/ct_master_logs.erl +++ b/lib/common_test/src/ct_master_logs.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2006-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2006-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -44,7 +44,7 @@ start(LogDir,Nodes) -> MRef = erlang:monitor(process,Pid), receive {started,Pid,Result} -> - erlang:demonitor(MRef), + erlang:demonitor(MRef, [flush]), {Pid,Result}; {'DOWN',MRef,process,_,Reason} -> exit({could_not_start_process,?MODULE,Reason}) @@ -435,7 +435,7 @@ call(Msg) -> ?MODULE ! {Msg,{self(),Ref}}, receive {Ref, Result} -> - erlang:demonitor(MRef), + erlang:demonitor(MRef, [flush]), Result; {'DOWN',MRef,process,_,Reason} -> {error,{process_down,?MODULE,Reason}} diff --git a/lib/common_test/src/ct_repeat.erl b/lib/common_test/src/ct_repeat.erl index 7ac6e045d7..be3c485b75 100644 --- a/lib/common_test/src/ct_repeat.erl +++ b/lib/common_test/src/ct_repeat.erl @@ -1,26 +1,26 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2007-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2007-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% %%% @doc Common Test Framework module that handles repeated test runs %%% %%% <p>This module exports functions for repeating tests. The following -%%% script flags (or equivalent ct:run_test/1 options) are supported: +%%% start flags (or equivalent ct:run_test/1 options) are supported: %%% -until <StopTime>, StopTime = YYMoMoDDHHMMSS | HHMMSS %%% -duration <DurTime>, DurTime = HHMMSS %%% -force_stop diff --git a/lib/common_test/src/ct_run.erl b/lib/common_test/src/ct_run.erl index 6b1063f74c..c5bfd01642 100644 --- a/lib/common_test/src/ct_run.erl +++ b/lib/common_test/src/ct_run.erl @@ -24,7 +24,6 @@ -module(ct_run). - %% Script interface -export([script_start/0,script_usage/0]). @@ -46,11 +45,28 @@ -define(abs(Name), filename:absname(Name)). -define(testdir(Name, Suite), ct_util:get_testdir(Name, Suite)). +-record(opts, {label, + vts, + shell, + cover, + coverspec, + step, + logdir, + config = [], + event_handlers = [], + include = [], + silent_connections, + stylesheet, + multiply_timetraps = 1, + scale_timetraps = false, + testspecs = [], + tests}). + %%%----------------------------------------------------------------- %%% @spec script_start() -> void() %%% -%%% @doc Start tests via the run_test script. -%%% +%%% @doc Start tests via the run_test program or script. +%%% %%% <p>Example:<br/><code>./run_test -config config.ctc -dir %%% $TEST_DIR</code></p> %%% @@ -59,13 +75,53 @@ %%% script_start() -> process_flag(trap_exit, true), - Args = merge_arguments(init:get_arguments()), + Init = init:get_arguments(), + CtArgs = lists:takewhile(fun({ct_erl_args,_}) -> false; + (_) -> true end, Init), + + %% convert relative dirs added with pa or pz (pre erl_args on + %% the run_test command line) to absolute so that app modules + %% can be found even after CT changes CWD to logdir + rel_to_abs(CtArgs), + + Args = + case application:get_env(common_test, run_test_start_opts) of + {ok,EnvStartOpts} -> + FlagFilter = fun(Flags) -> + lists:filter(fun({root,_}) -> false; + ({progname,_}) -> false; + ({home,_}) -> false; + ({noshell,_}) -> false; + ({noinput,_}) -> false; + (_) -> true + end, Flags) + end, + %% used for purpose of testing the run_test interface + io:format(user, "~n-------------------- START ARGS --------------------~n", []), + io:format(user, "--- Init args:~n~p~n", [FlagFilter(Init)]), + io:format(user, "--- CT args:~n~p~n", [FlagFilter(CtArgs)]), + EnvArgs = opts2args(EnvStartOpts), + io:format(user, "--- Env opts -> args:~n~p~n =>~n~p~n", + [EnvStartOpts,EnvArgs]), + Merged = merge_arguments(CtArgs ++ EnvArgs), + io:format(user, "--- Merged args:~n~p~n", [FlagFilter(Merged)]), + io:format(user, "----------------------------------------------------~n~n", []), + Merged; + _ -> + merge_arguments(CtArgs) + end, + case proplists:get_value(help, Args) of + undefined -> script_start(Args); + _ -> script_usage() + end. + +script_start(Args) -> Tracing = start_trace(Args), - Res = + Res = case ct_repeat:loop_test(script, Args) of - false -> + false -> {ok,Cwd} = file:get_cwd(), - CTVsn = + CTVsn = case filename:basename(code:lib_dir(common_test)) of CTBase when is_list(CTBase) -> case string:tokens(CTBase, "-") of @@ -76,7 +132,7 @@ script_start() -> io:format("~nCommon Test~s starting (cwd is ~s)~n~n", [CTVsn,Cwd]), Self = self(), Pid = spawn_link(fun() -> script_start1(Self, Args) end), - receive + receive {'EXIT',Pid,Reason} -> case Reason of {user_error,What} -> @@ -98,296 +154,335 @@ script_start() -> Result end, stop_trace(Tracing), + timer:sleep(1000), Res. script_start1(Parent, Args) -> - case lists:keymember(preload, 1, Args) of - true -> preload(); - false -> ok - end, - - VtsOrShell = - case lists:keymember(vts, 1, Args) of - true -> - vts; - false -> - case lists:keymember(shell, 1, Args) of - true -> shell; - false -> false - end - end, - LogDir = - case lists:keysearch(logdir, 1, Args) of - {value,{logdir,[LogD]}} -> LogD; - false -> "." - end, - EvHandlers = - case lists:keysearch(event_handler, 1, Args) of - {value,{event_handler,Handlers}} -> - lists:map(fun(H) -> {list_to_atom(H),[]} end, Handlers); - false -> - [] - end, - Cover = - case lists:keysearch(cover, 1, Args) of - {value,{cover,CoverFile}} -> - {cover,?abs(CoverFile)}; - false -> - false - end, - - case lists:keysearch(ct_decrypt_key, 1, Args) of - {value,{_,[DecryptKey]}} -> + %% read general start flags + Label = get_start_opt(label, fun([Lbl]) -> Lbl end, Args), + Vts = get_start_opt(vts, true, Args), + Shell = get_start_opt(shell, true, Args), + Cover = get_start_opt(cover, fun([CoverFile]) -> ?abs(CoverFile) end, Args), + LogDir = get_start_opt(logdir, fun([LogD]) -> LogD end, Args), + MultTT = get_start_opt(multiply_timetraps, + fun([MT]) -> list_to_integer(MT) end, 1, Args), + ScaleTT = get_start_opt(scale_timetraps, + fun([CT]) -> list_to_atom(CT); + ([]) -> true + end, false, Args), + EvHandlers = event_handler_args2opts(Args), + + %% check flags and set corresponding application env variables + + %% ct_decrypt_key | ct_decrypt_file + case proplists:get_value(ct_decrypt_key, Args) of + [DecryptKey] -> application:set_env(common_test, decrypt, {key,DecryptKey}); - false -> - case lists:keysearch(ct_decrypt_file, 1, Args) of - {value,{_,[DecryptFile]}} -> - application:set_env(common_test, decrypt, - {file,filename:absname(DecryptFile)}); - false -> + undefined -> + case proplists:get_value(ct_decrypt_file, Args) of + [DecryptFile] -> + application:set_env(common_test, decrypt, + {file,?abs(DecryptFile)}); + undefined -> application:unset_env(common_test, decrypt) end end, - - case lists:keysearch(no_auto_compile, 1, Args) of - {value,_} -> - application:set_env(common_test, auto_compile, false); - false -> - application:set_env(common_test, auto_compile, true), - - InclDirs = - case lists:keysearch(include,1,Args) of - {value,{include,Incl}} when is_list(hd(Incl)) -> - Incl; - {value,{include,Incl}} when is_list(Incl) -> - [Incl]; + %% no_auto_compile + include + IncludeDirs = + case proplists:get_value(no_auto_compile, Args) of + undefined -> + application:set_env(common_test, auto_compile, true), + InclDirs = + case proplists:get_value(include, Args) of + Incl when is_list(hd(Incl)) -> + Incl; + Incl when is_list(Incl) -> + [Incl]; + undefined -> + [] + end, + case os:getenv("CT_INCLUDE_PATH") of false -> - [] - end, - case os:getenv("CT_INCLUDE_PATH") of - false -> - application:set_env(common_test, include, InclDirs); - CtInclPath -> - InclDirs1 = string:tokens(CtInclPath,[$:,$ ,$,]), - application:set_env(common_test, include, InclDirs1++InclDirs) - end - end, - - case lists:keysearch(basic_html, 1, Args) of - {value,_} -> - application:set_env(common_test, basic_html, true); - false -> - application:set_env(common_test, basic_html, false) - end, - - Result = - case lists:keysearch(refresh_logs, 1, Args) of - {value,{refresh_logs,Refresh}} -> - LogDir1 = case Refresh of - [] -> LogDir; - [RefreshDir] -> ?abs(RefreshDir) - end, - {ok,Cwd} = file:get_cwd(), - file:set_cwd(LogDir1), - timer:sleep(500), % give the shell time to print version etc - io:nl(), - case catch ct_logs:make_all_suites_index(refresh) of - {'EXIT',ASReason} -> - file:set_cwd(Cwd), - {error,{all_suites_index,ASReason}}; - _ -> - case catch ct_logs:make_all_runs_index(refresh) of - {'EXIT',ARReason} -> - file:set_cwd(Cwd), - {error,{all_runs_index,ARReason}}; - _ -> - file:set_cwd(Cwd), - io:format("Logs in ~s refreshed!~n~n", [LogDir1]), - timer:sleep(500), % time to flush io before quitting - ok - end + application:set_env(common_test, include, InclDirs), + InclDirs; + CtInclPath -> + AllInclDirs = + string:tokens(CtInclPath,[$:,$ ,$,]) ++ InclDirs, + application:set_env(common_test, include, AllInclDirs), + AllInclDirs end; - false -> - case lists:keysearch(ct_config, 1, Args) of - {value,{ct_config,ConfigFiles}} -> - case lists:keysearch(spec, 1, Args) of - false -> - case get_configfiles(ConfigFiles, [], LogDir, - EvHandlers) of - ok -> - script_start2(VtsOrShell, ConfigFiles, - EvHandlers, Args, LogDir, - Cover); - Error -> - Error - end; - _ -> - script_start2(VtsOrShell, ConfigFiles, - EvHandlers, Args, LogDir, Cover) - end; - false -> - case install([{config,[]}, - {event_handler,EvHandlers}], - LogDir) of - ok -> - script_start2(VtsOrShell, [], EvHandlers, - Args, LogDir, Cover); - Error -> - Error - end - end + _ -> + application:set_env(common_test, auto_compile, false), + [] end, + %% silent connections + SilentConns = + get_start_opt(silent_connections, + fun(["all"]) -> []; + (Conns) -> [list_to_atom(Conn) || Conn <- Conns] + end, Args), + %% stylesheet + Stylesheet = get_start_opt(stylesheet, + fun([SS]) -> ?abs(SS) end, Args), + %% basic_html - used by ct_logs + case proplists:get_value(basic_html, Args) of + undefined -> + application:set_env(common_test, basic_html, false); + _ -> + application:set_env(common_test, basic_html, true) + end, + + StartOpts = #opts{label = Label, vts = Vts, shell = Shell, cover = Cover, + logdir = LogDir, event_handlers = EvHandlers, + include = IncludeDirs, + silent_connections = SilentConns, + stylesheet = Stylesheet, + multiply_timetraps = MultTT, + scale_timetraps = ScaleTT}, + + %% check if log files should be refreshed or go on to run tests... + Result = run_or_refresh(StartOpts, Args), + %% send final results to starting process waiting in script_start/0 Parent ! {self(), Result}. -get_configfiles([File|Files], Acc, LogDir, EvHandlers) -> - case filelib:is_file(File) of - true -> - get_configfiles(Files, [?abs(File)|Acc], - LogDir, EvHandlers); - false -> - {error,{cant_read_config_file,File}} - end; -get_configfiles([], Acc, LogDir, EvHandlers) -> - install([{config,lists:reverse(Acc)}, {event_handler,EvHandlers}], LogDir). +run_or_refresh(StartOpts = #opts{logdir = LogDir}, Args) -> + case proplists:get_value(refresh_logs, Args) of + undefined -> + script_start2(StartOpts, Args); + Refresh -> + LogDir1 = case Refresh of + [] -> which(logdir,LogDir); + [RefreshDir] -> ?abs(RefreshDir) + end, + {ok,Cwd} = file:get_cwd(), + file:set_cwd(LogDir1), + %% give the shell time to print version etc + timer:sleep(500), + io:nl(), + case catch ct_logs:make_all_suites_index(refresh) of + {'EXIT',ASReason} -> + file:set_cwd(Cwd), + {error,{all_suites_index,ASReason}}; + _ -> + case catch ct_logs:make_all_runs_index(refresh) of + {'EXIT',ARReason} -> + file:set_cwd(Cwd), + {error,{all_runs_index,ARReason}}; + _ -> + file:set_cwd(Cwd), + io:format("Logs in ~s refreshed!~n~n", [LogDir1]), + timer:sleep(500), % time to flush io before quitting + ok + end + end + end. -script_start2(false, ConfigFiles, EvHandlers, Args, LogDir, Cover) -> - case lists:keysearch(spec, 1, Args) of - {value,{spec,[]}} -> +script_start2(StartOpts = #opts{vts = undefined, + shell = undefined}, Args) -> + TestSpec = proplists:get_value(spec, Args), + {Terms,Opts} = + case TestSpec of + Specs when Specs =/= [], Specs =/= undefined -> + %% using testspec as input for test + Relaxed = get_start_opt(allow_user_terms, true, false, Args), + case catch ct_testspec:collect_tests_from_file(Specs, Relaxed) of + {E,Reason} when E == error ; E == 'EXIT' -> + {{error,Reason},StartOpts}; + TS -> + SpecStartOpts = get_data_for_node(TS, node()), + + Label = choose_val(StartOpts#opts.label, + SpecStartOpts#opts.label), + + LogDir = choose_val(StartOpts#opts.logdir, + SpecStartOpts#opts.logdir), + + Cover = choose_val(StartOpts#opts.cover, + SpecStartOpts#opts.cover), + MultTT = choose_val(StartOpts#opts.multiply_timetraps, + SpecStartOpts#opts.multiply_timetraps), + ScaleTT = choose_val(StartOpts#opts.scale_timetraps, + SpecStartOpts#opts.scale_timetraps), + AllEvHs = merge_vals([StartOpts#opts.event_handlers, + SpecStartOpts#opts.event_handlers]), + AllInclude = merge_vals([StartOpts#opts.include, + SpecStartOpts#opts.include]), + application:set_env(common_test, include, AllInclude), + + {TS,StartOpts#opts{label = Label, + testspecs = Specs, + cover = Cover, + logdir = LogDir, + config = SpecStartOpts#opts.config, + event_handlers = AllEvHs, + include = AllInclude, + multiply_timetraps = MultTT, + scale_timetraps = ScaleTT}} + end; + _ -> + {undefined,StartOpts} + end, + %% read config/userconfig from start flags + InitConfig = ct_config:prepare_config_list(Args), + TheLogDir = which(logdir, Opts#opts.logdir), + case {TestSpec,Terms} of + {_,{error,_}=Error} -> + Error; + {[],_} -> {error,no_testspec_specified}; - {value,{spec,Specs}} -> - Relaxed = lists:keymember(allow_user_terms, 1, Args), - %% using testspec as input for test - case catch ct_testspec:collect_tests_from_file(Specs, Relaxed) of - {error,Reason} -> - {error,Reason}; - TS -> - {LogDir1,TSCoverFile,ConfigFiles1,EvHandlers1,Include1} = - get_data_for_node(TS,node()), - UserInclude = - case application:get_env(common_test, include) of - {ok,Include} -> Include++Include1; - _ -> Include1 - end, - application:set_env(common_test, include, UserInclude), - LogDir2 = which_logdir(LogDir,LogDir1), - CoverOpt = case {Cover,TSCoverFile} of - {false,undef} -> []; - {_,undef} -> [Cover]; - {false,_} -> [{cover,TSCoverFile}] - end, - case get_configfiles(ConfigFiles++ConfigFiles1, - [], LogDir2, - EvHandlers++EvHandlers1) of - ok -> - {Run,Skip} = ct_testspec:prepare_tests(TS, node()), - do_run(Run, Skip, CoverOpt, Args, LogDir2); - Error -> - Error - end + {undefined,_} -> % no testspec used + case check_and_install_configfiles(InitConfig, TheLogDir, + Opts#opts.event_handlers) of + ok -> % go on read tests from start flags + script_start3(Opts#opts{config=InitConfig, + logdir=TheLogDir}, Args); + Error -> + Error end; - false -> - script_start3(false, ConfigFiles, EvHandlers, Args, LogDir, Cover) + {_,_} -> % testspec used + %% merge config from start flags with config from testspec + AllConfig = merge_vals([InitConfig, Opts#opts.config]), + case check_and_install_configfiles(AllConfig, TheLogDir, + Opts#opts.event_handlers) of + ok -> % read tests from spec + {Run,Skip} = ct_testspec:prepare_tests(Terms, node()), + do_run(Run, Skip, Opts#opts{config=AllConfig, + logdir=TheLogDir}, Args); + Error -> + Error + end end; -script_start2(VtsOrShell, ConfigFiles, EvHandlers, Args, LogDir, Cover) -> - script_start3(VtsOrShell, ConfigFiles, EvHandlers, Args, LogDir, Cover). -script_start3(VtsOrShell, ConfigFiles, EvHandlers, Args, LogDir, Cover) -> - case lists:keysearch(dir, 1, Args) of - {value,{dir,[]}} -> - {error,no_dir_specified}; - {value,{dir,Dirs}} -> - script_start4(VtsOrShell, ConfigFiles, EvHandlers, tests(Dirs), - Cover, Args, LogDir); +script_start2(StartOpts, Args) -> + %% read config/userconfig from start flags + InitConfig = ct_config:prepare_config_list(Args), + LogDir = which(logdir, StartOpts#opts.logdir), + case check_and_install_configfiles(InitConfig, LogDir, + StartOpts#opts.event_handlers) of + ok -> % go on read tests from start flags + script_start3(StartOpts#opts{config=InitConfig, + logdir=LogDir}, Args); + Error -> + Error + end. + +check_and_install_configfiles(Configs, LogDir, EvHandlers) -> + case ct_config:check_config_files(Configs) of false -> - case lists:keysearch(suite, 1, Args) of - {value,{suite,[]}} -> + install([{config,Configs}, + {event_handler,EvHandlers}], LogDir); + {value,{error,{nofile,File}}} -> + {error,{cant_read_config_file,File}}; + {value,{error,{wrong_config,Message}}}-> + {error,{wrong_config,Message}}; + {value,{error,{callback,Info}}} -> + {error,{cant_load_callback_module,Info}} + end. + +script_start3(StartOpts, Args) -> + case proplists:get_value(dir, Args) of + [] -> + {error,no_dir_specified}; + Dirs when is_list(Dirs) -> + script_start4(StartOpts#opts{tests = tests(Dirs)}, Args); + undefined -> + case proplists:get_value(suite, Args) of + [] -> {error,no_suite_specified}; - {value,{suite,Suites}} -> - StepOrCover = - case lists:keysearch(step, 1, Args) of - {value,Step} -> Step; - false -> Cover - end, - S2M = fun(S) -> - {filename:dirname(S), - list_to_atom( - filename:rootname(filename:basename(S)))} - end, - DirMods = lists:map(S2M, Suites), - {Specified,GroupsAndCases} = - case {lists:keysearch(group, 1, Args), - lists:keysearch('case', 1, Args)} of - {{value,{_,Gs}},{value,{_,Cs}}} -> {true,Gs++Cs}; - {{value,{_,Gs}},_} -> {true,Gs}; - {_,{value,{_,Cs}}} -> {true,Cs}; - _ -> {false,[]} - end, - if Specified, length(GroupsAndCases) == 0 -> - {error,no_case_or_group_specified}; - Specified, length(DirMods) > 1 -> + Suites when is_list(Suites) -> + StartOpts1 = + get_start_opt(step, + fun(Step) -> + StartOpts#opts{step = Step, + cover = undefined} + end, StartOpts, Args), + DirMods = [suite_to_test(S) || S <- Suites], + case groups_and_cases(proplists:get_value(group, Args), + proplists:get_value(testcase, Args)) of + Error = {error,_} -> + Error; + [] when DirMods =/= [] -> + Ts = tests(DirMods), + script_start4(StartOpts1#opts{tests = Ts}, Args); + GroupsAndCases when length(DirMods) == 1 -> + Ts = tests(DirMods, GroupsAndCases), + script_start4(StartOpts1#opts{tests = Ts}, Args); + [_,_|_] when length(DirMods) > 1 -> {error,multiple_suites_and_cases}; - length(GroupsAndCases) > 0, length(DirMods) == 1 -> - GsAndCs = lists:map(fun(C) -> list_to_atom(C) end, - GroupsAndCases), - script_start4(VtsOrShell, ConfigFiles, EvHandlers, - tests(DirMods, GsAndCs), - StepOrCover, Args, LogDir); - not Specified, length(DirMods) > 0 -> - script_start4(VtsOrShell, ConfigFiles, EvHandlers, - tests(DirMods), - StepOrCover, Args, LogDir); - true -> - {error,incorrect_suite_and_case_options} + _ -> + {error,incorrect_suite_option} end; - false when VtsOrShell=/=false -> - script_start4(VtsOrShell, ConfigFiles, EvHandlers, - [], Cover, Args, LogDir); - false -> - script_usage(), - {error,incorrect_usage} + undefined -> + if StartOpts#opts.vts ; StartOpts#opts.shell -> + script_start4(StartOpts#opts{tests = []}, Args); + true -> + script_usage(), + {error,incorrect_usage} + end end end. -script_start4(vts, ConfigFiles, EvHandlers, Tests, false, _Args, LogDir) -> +script_start4(#opts{vts = true, config = Config, event_handlers = EvHandlers, + tests = Tests, logdir = LogDir}, _Args) -> + ConfigFiles = + lists:foldl(fun({ct_config_plain,CfgFiles}, AllFiles) when + is_list(hd(CfgFiles)) -> + AllFiles ++ CfgFiles; + ({ct_config_plain,CfgFile}, AllFiles) when + is_integer(hd(CfgFile)) -> + AllFiles ++ [CfgFile]; + (_, AllFiles) -> + AllFiles + end, [], Config), vts:init_data(ConfigFiles, EvHandlers, ?abs(LogDir), Tests); -script_start4(shell, ConfigFiles, EvHandlers, _Tests, false, Args, LogDir) -> - Opts = [{config,ConfigFiles},{event_handler,EvHandlers}], - if ConfigFiles == [] -> + +script_start4(#opts{label = Label, shell = true, config = Config, + event_handlers = EvHandlers, + logdir = LogDir, testspecs = Specs}, _Args) -> + %% label - used by ct_logs + application:set_env(common_test, test_label, Label), + + InstallOpts = [{config,Config},{event_handler,EvHandlers}], + if Config == [] -> ok; true -> - io:format("\nInstalling: ~p\n\n", [ConfigFiles]) + io:format("\nInstalling: ~p\n\n", [Config]) end, - case install(Opts) of + case install(InstallOpts) of ok -> ct_util:start(interactive, LogDir), - log_ts_names(Args), + log_ts_names(Specs), io:nl(), ok; Error -> Error end; -script_start4(vts, _CfgFs, _EvHs, _Tests, _Cover={cover,_}, _Args, _LogDir) -> - %% Add support later (maybe). - script_usage(), - erlang:halt(); -script_start4(shell, _CfgFs, _EvHs, _Tests, _Cover={cover,_}, _Args, _LogDir) -> - %% Add support later (maybe). - script_usage(); -script_start4(false, _CfgFs, _EvHs, Tests, Cover={cover,_}, Args, LogDir) -> - do_run(Tests, [], [Cover], Args, LogDir); -script_start4(false, _ConfigFiles, _EvHandlers, Tests, false, Args, LogDir) -> - do_run(Tests, [], [], Args, LogDir); -script_start4(false, _ConfigFiles, _EvHandlers, Test, Step, Args, LogDir) -> - do_run(Test, [], [Step], Args, LogDir); -script_start4(vts, _ConfigFiles, _EvHandlers, _Test, _Step, _Args, _LogDir) -> - script_usage(), + +script_start4(#opts{vts = true, cover = Cover}, _) -> + case Cover of + undefined -> + script_usage(); + _ -> + %% Add support later (maybe). + io:format("\nCan't run cover in vts mode.\n\n", []) + end, erlang:halt(); -script_start4(shell, _ConfigFiles, _EvHandlers, _Test, _Step, _Args, _LogDir) -> - script_usage(). + +script_start4(#opts{shell = true, cover = Cover}, _) -> + case Cover of + undefined -> + script_usage(); + _ -> + %% Add support later (maybe). + io:format("\nCan't run cover in interactive mode.\n\n", []) + end; + +script_start4(Opts = #opts{tests = Tests}, Args) -> + do_run(Tests, [], Opts, Args). %%%----------------------------------------------------------------- %%% @spec script_usage() -> ok -%%% @doc Print script usage information for <code>run_test</code>. +%%% @doc Print usage information for <code>run_test</code>. script_usage() -> io:format("\n\nUsage:\n\n"), io:format("Run tests in web based GUI:\n\n" @@ -396,24 +491,29 @@ script_usage() -> "\n\t[-decrypt_key Key] | [-decrypt_file KeyFile]" "\n\t[-dir TestDir1 TestDir2 .. TestDirN] |" "\n\t[-suite Suite [-case Case]]" - "\n\t[-include InclDir1 InclDir2 .. InclDirN]" + "\n\t[-include InclDir1 InclDir2 .. InclDirN]" "\n\t[-no_auto_compile]" + "\n\t[-multiply_timetraps N]" + "\n\t[-scale_timetraps]" "\n\t[-basic_html]\n\n"), io:format("Run tests from command line:\n\n" "\trun_test [-dir TestDir1 TestDir2 .. TestDirN] |" "\n\t[-suite Suite1 Suite2 .. SuiteN [-case Case1 Case2 .. CaseN]]" "\n\t[-step [config | keep_inactive]]" "\n\t[-config ConfigFile1 ConfigFile2 .. ConfigFileN]" + "\n\t[-userconfig CallbackModule ConfigFile1 .. ConfigFileN]" "\n\t[-decrypt_key Key] | [-decrypt_file KeyFile]" "\n\t[-logdir LogDir]" "\n\t[-silent_connections [ConnType1 ConnType2 .. ConnTypeN]]" "\n\t[-stylesheet CSSFile]" "\n\t[-cover CoverCfgFile]" "\n\t[-event_handler EvHandler1 EvHandler2 .. EvHandlerN]" - "\n\t[-include InclDir1 InclDir2 .. InclDirN]" + "\n\t[-include InclDir1 InclDir2 .. InclDirN]" "\n\t[-no_auto_compile]" - "\n\t[-basic_html]" - "\n\t[-repeat N [-force_stop]] |" + "\n\t[-multiply_timetraps N]" + "\n\t[-scale_timetraps]" + "\n\t[-basic_html]" + "\n\t[-repeat N [-force_stop]] |" "\n\t[-duration HHMMSS [-force_stop]] |" "\n\t[-until [YYMoMoDD]HHMMSS [-force_stop]]\n\n"), io:format("Run tests using test specification:\n\n" @@ -426,10 +526,12 @@ script_usage() -> "\n\t[-stylesheet CSSFile]" "\n\t[-cover CoverCfgFile]" "\n\t[-event_handler EvHandler1 EvHandler2 .. EvHandlerN]" - "\n\t[-include InclDir1 InclDir2 .. InclDirN]" + "\n\t[-include InclDir1 InclDir2 .. InclDirN]" "\n\t[-no_auto_compile]" - "\n\t[-basic_html]" - "\n\t[-repeat N [-force_stop]] |" + "\n\t[-multiply_timetraps N]" + "\n\t[-scale_timetraps]" + "\n\t[-basic_html]" + "\n\t[-repeat N [-force_stop]] |" "\n\t[-duration HHMMSS [-force_stop]] |" "\n\t[-until [YYMoMoDD]HHMMSS [-force_stop]]\n\n"), io:format("Refresh the HTML index files:\n\n" @@ -440,7 +542,6 @@ script_usage() -> "\trun_test -shell" "\n\t[-config ConfigFile1 ConfigFile2 .. ConfigFileN]" "\n\t[-decrypt_key Key] | [-decrypt_file KeyFile]\n\n"). - %%%----------------------------------------------------------------- %%% @hidden @@ -468,7 +569,7 @@ install(Opts, LogDir) -> [io:format(Fd, "~p.\n", [Opt]) || Opt <- Opts], file:close(Fd), ok; - {error,Reason} -> + {error,Reason} -> io:format("CT failed to install configuration data. Please " "verify that the log directory exists and that " "write permission is set.\n\n", []), @@ -487,69 +588,62 @@ variables_file_name(Dir) -> filename:join(Dir, "variables-"++atom_to_list(node())). %%%----------------------------------------------------------------- -%%% @hidden +%%% @spec run_test(Opts) -> Result +%%% Opts = [tuple()] +%%% Result = [TestResult] | {error,Reason} +%%% +%%% @doc Start tests from the erlang shell or from an erlang program. %%% @equiv ct:run_test/1 +%%%----------------------------------------------------------------- -%% Opts = [OptTuples] -%% OptTuples = {config,CfgFiles} | {dir,TestDirs} | {suite,Suites} | -%% {testcase,Cases} | {spec,TestSpecs} | {allow_user_terms,Bool} | -%% {logdir,LogDir} | {cover,CoverSpecFile} | {step,StepOpts} | -%% {silent_connections,Conns} | {event_handler,EventHandlers} | -%% {include,InclDirs} | {auto_compile,Bool} | -%% {repeat,N} | {duration,DurTime} | {until,StopTime} | {force_stop,Bool} | -%% {decrypt,KeyOrFile} - -run_test(Opt) when is_tuple(Opt) -> - run_test([Opt]); - -run_test(Opts) when is_list(Opts) -> - case lists:keysearch(refresh_logs, 1, Opts) of - {value,{_,RefreshDir}} -> - refresh_logs(?abs(RefreshDir)), - ok; - false -> - Tracing = start_trace(Opts), +run_test(StartOpt) when is_tuple(StartOpt) -> + run_test([StartOpt]); + +run_test(StartOpts) when is_list(StartOpts) -> + case proplists:get_value(refresh_logs, StartOpts) of + undefined -> + Tracing = start_trace(StartOpts), {ok,Cwd} = file:get_cwd(), - io:format("~nCommon Test starting (cwd is ~s)~n~n", [Cwd]), + io:format("~nCommon Test starting (cwd is ~s)~n~n", [Cwd]), Res = - case ct_repeat:loop_test(func, Opts) of + case ct_repeat:loop_test(func, StartOpts) of false -> - case catch run_test1(Opts) of - {'EXIT',Reason} -> + case catch run_test1(StartOpts) of + {'EXIT',Reason} -> file:set_cwd(Cwd), {error,Reason}; - Result -> + Result -> Result end; Result -> Result end, stop_trace(Tracing), - Res + Res; + RefreshDir -> + refresh_logs(?abs(RefreshDir)), + ok end. -run_test1(Opts) -> - LogDir = - case lists:keysearch(logdir, 1, Opts) of - {value,{_,LD}} when is_list(LD) -> LD; - false -> "." - end, - CfgFiles = - case lists:keysearch(config, 1, Opts) of - {value,{_,Files=[File|_]}} when is_list(File) -> - Files; - {value,{_,File=[C|_]}} when is_integer(C) -> - [File]; - {value,{_,[]}} -> - []; - false -> - [] - end, +run_test1(StartOpts) -> + %% label + Label = get_start_opt(label, fun(Lbl) when is_list(Lbl) -> Lbl; + (Lbl) when is_atom(Lbl) -> atom_to_list(Lbl) + end, StartOpts), + %% logdir + LogDir = get_start_opt(logdir, fun(LD) when is_list(LD) -> LD end, + StartOpts), + %% config & userconfig + CfgFiles = ct_config:get_config_file_list(StartOpts), + + %% event handlers EvHandlers = - case lists:keysearch(event_handler, 1, Opts) of - {value,{_,H}} when is_atom(H) -> + case proplists:get_value(event_handler, StartOpts) of + undefined -> + []; + H when is_atom(H) -> [{H,[]}]; - {value,{_,H}} -> + H -> Hs = if is_tuple(H) -> [H]; is_list(H) -> H; @@ -564,41 +658,39 @@ run_test1(Opts) -> {EH,Args}; (_) -> [] - end, Hs)); - _ -> - [] - end, - SilentConns = - case lists:keysearch(silent_connections, 1, Opts) of - {value,{_,all}} -> - []; - {value,{_,Conns}} -> - Conns; - _ -> - undefined - end, - Cover = - case lists:keysearch(cover, 1, Opts) of - {value,{_,CoverFile}} -> - [{cover,?abs(CoverFile)}]; - _ -> - [] + end, Hs)) end, + + %% silent connections + SilentConns = get_start_opt(silent_connections, + fun(all) -> []; + (Conns) -> Conns + end, StartOpts), + %% stylesheet + Stylesheet = get_start_opt(stylesheet, + fun(SS) -> ?abs(SS) end, + StartOpts), + %% code coverage + Cover = get_start_opt(cover, + fun(CoverFile) -> ?abs(CoverFile) end, StartOpts), + + %% timetrap manipulation + MultiplyTT = get_start_opt(multiply_timetraps, value, 1, StartOpts), + ScaleTT = get_start_opt(scale_timetraps, value, false, StartOpts), + + %% auto compile & include files Include = - case lists:keysearch(auto_compile, 1, Opts) of - {value,{auto_compile,ACBool}} -> - application:set_env(common_test, auto_compile, ACBool), - []; - _ -> + case proplists:get_value(auto_compile, StartOpts) of + undefined -> application:set_env(common_test, auto_compile, true), InclDirs = - case lists:keysearch(include, 1, Opts) of - {value,{include,Incl}} when is_list(hd(Incl)) -> - Incl; - {value,{include,Incl}} when is_list(Incl) -> - [Incl]; - false -> - [] + case proplists:get_value(include, StartOpts) of + undefined -> + []; + Incl when is_list(hd(Incl)) -> + Incl; + Incl when is_list(Incl) -> + [Incl] end, case os:getenv("CT_INCLUDE_PATH") of false -> @@ -609,117 +701,173 @@ run_test1(Opts) -> AllInclDirs = InclDirs1++InclDirs, application:set_env(common_test, include, AllInclDirs), AllInclDirs - end + end; + ACBool -> + application:set_env(common_test, auto_compile, ACBool), + [] end, - case lists:keysearch(decrypt, 1, Opts) of - {value,{_,Key={key,_}}} -> + %% decrypt config file + case proplists:get_value(decrypt, StartOpts) of + undefined -> + application:unset_env(common_test, decrypt); + Key={key,_} -> application:set_env(common_test, decrypt, Key); - {value,{_,{file,KeyFile}}} -> - application:set_env(common_test, decrypt, {file,filename:absname(KeyFile)}); - false -> - application:unset_env(common_test, decrypt) + {file,KeyFile} -> + application:set_env(common_test, decrypt, {file,?abs(KeyFile)}) end, - case lists:keysearch(basic_html, 1, Opts) of - {value,{basic_html,BasicHtmlBool}} -> - application:set_env(common_test, basic_html, BasicHtmlBool); - _ -> - application:set_env(common_test, basic_html, false) + %% basic html - used by ct_logs + case proplists:get_value(basic_html, StartOpts) of + undefined -> + application:set_env(common_test, basic_html, false); + BasicHtmlBool -> + application:set_env(common_test, basic_html, BasicHtmlBool) end, - case lists:keysearch(spec, 1, Opts) of - {value,{_,Specs}} -> - Relaxed = - case lists:keysearch(allow_user_terms, 1, Opts) of - {value,{_,true}} -> true; - _ -> false - end, + %% stepped execution + Step = get_start_opt(step, value, StartOpts), + + Opts = #opts{label = Label, + cover = Cover, step = Step, logdir = LogDir, config = CfgFiles, + event_handlers = EvHandlers, include = Include, + silent_connections = SilentConns, + stylesheet = Stylesheet, + multiply_timetraps = MultiplyTT, + scale_timetraps = ScaleTT}, + + %% test specification + case proplists:get_value(spec, StartOpts) of + undefined -> + case proplists:get_value(prepared_tests, StartOpts) of + undefined -> % use dir|suite|case + run_dir(Opts, StartOpts); + {{Run,Skip},Specs} -> % use prepared tests + run_prepared(Run, Skip, Opts#opts{testspecs = Specs}, StartOpts) + end; + Specs -> + Relaxed = get_start_opt(allow_user_terms, value, false, StartOpts), %% using testspec(s) as input for test - run_spec_file(LogDir, CfgFiles, EvHandlers, Include, Specs, Relaxed, Cover, - replace_opt([{silent_connections,SilentConns}], Opts)); - false -> - case lists:keysearch(prepared_tests, 1, Opts) of - {value,{_,{Run,Skip},Specs}} -> % use prepared tests - run_prepared(LogDir, CfgFiles, EvHandlers, - Run, Skip, Cover, - replace_opt([{silent_connections,SilentConns}, - {spec,Specs}],Opts)); - false -> % use dir|suite|case - StepOrCover = - case lists:keysearch(step, 1, Opts) of - {value,Step} -> [Step]; - false -> Cover - end, - run_dir(LogDir, CfgFiles, EvHandlers, StepOrCover, - replace_opt([{silent_connections,SilentConns}], Opts)) - end + run_spec_file(Relaxed, Opts#opts{testspecs = Specs}, StartOpts) end. -replace_opt([O={Key,_Val}|Os], Opts) -> - [O | replace_opt(Os, lists:keydelete(Key, 1, Opts))]; -replace_opt([], Opts) -> - Opts. - -run_spec_file(LogDir, CfgFiles, EvHandlers, Include, Specs, Relaxed, Cover, Opts) -> +run_spec_file(Relaxed, + Opts = #opts{testspecs = Specs, config = CfgFiles}, + StartOpts) -> Specs1 = case Specs of [X|_] when is_integer(X) -> [Specs]; _ -> Specs end, - AbsSpecs = lists:map(fun(SF) -> ?abs(SF) end, Specs1), + AbsSpecs = lists:map(fun(SF) -> ?abs(SF) end, Specs1), log_ts_names(AbsSpecs), case catch ct_testspec:collect_tests_from_file(AbsSpecs, Relaxed) of - {error,CTReason} -> + {Error,CTReason} when Error == error ; Error == 'EXIT' -> exit(CTReason); TS -> - {LogDir1,TSCoverFile,CfgFiles1,EvHandlers1,Include1} = - get_data_for_node(TS, node()), - application:set_env(common_test, include, Include++Include1), - LogDir2 = which_logdir(LogDir, LogDir1), - CoverOpt = case {Cover,TSCoverFile} of - {[],undef} -> []; - {_,undef} -> Cover; - {[],_} -> [{cover,TSCoverFile}] - end, - case get_configfiles(CfgFiles++CfgFiles1, [], LogDir2, - EvHandlers++EvHandlers1) of + SpecOpts = get_data_for_node(TS, node()), + Label = choose_val(Opts#opts.label, + SpecOpts#opts.label), + LogDir = choose_val(Opts#opts.logdir, + SpecOpts#opts.logdir), + AllConfig = merge_vals([CfgFiles, SpecOpts#opts.config]), + Cover = choose_val(Opts#opts.cover, + SpecOpts#opts.cover), + MultTT = choose_val(Opts#opts.multiply_timetraps, + SpecOpts#opts.multiply_timetraps), + ScaleTT = choose_val(Opts#opts.scale_timetraps, + SpecOpts#opts.scale_timetraps), + AllEvHs = merge_vals([Opts#opts.event_handlers, + SpecOpts#opts.event_handlers]), + AllInclude = merge_vals([Opts#opts.include, + SpecOpts#opts.include]), + application:set_env(common_test, include, AllInclude), + + case check_and_install_configfiles(AllConfig, + which(logdir,LogDir), + AllEvHs) of ok -> + Opts1 = Opts#opts{label = Label, + cover = Cover, + logdir = which(logdir, LogDir), + config = AllConfig, + event_handlers = AllEvHs, + include = AllInclude, + testspecs = AbsSpecs, + multiply_timetraps = MultTT, + scale_timetraps = ScaleTT}, {Run,Skip} = ct_testspec:prepare_tests(TS, node()), - do_run(Run, Skip, CoverOpt, - replace_opt([{spec,AbsSpecs}], Opts), - LogDir2); + reformat_result(catch do_run(Run, Skip, Opts1, StartOpts)); {error,GCFReason} -> exit(GCFReason) end end. -run_prepared(LogDir, CfgFiles, EvHandlers, Run, Skip, Cover, Opts) -> - case get_configfiles(CfgFiles, [], LogDir, EvHandlers) of +run_prepared(Run, Skip, Opts = #opts{logdir = LogDir, + config = CfgFiles, + event_handlers = EvHandlers}, + StartOpts) -> + LogDir1 = which(logdir, LogDir), + case check_and_install_configfiles(CfgFiles, LogDir1, EvHandlers) of ok -> - do_run(Run, Skip, Cover, Opts, LogDir); + reformat_result(catch do_run(Run, Skip, Opts#opts{logdir = LogDir1}, + StartOpts)); {error,Reason} -> exit(Reason) - end. - -run_dir(LogDir, CfgFiles, EvHandlers, StepOrCover, Opts) -> - AbsCfgFiles = - lists:map(fun(F) -> - AbsName = ?abs(F), - case filelib:is_file(AbsName) of - true -> AbsName; - false -> exit({no_such_file,AbsName}) - end - end, CfgFiles), - - case install([{config,AbsCfgFiles},{event_handler,EvHandlers}], LogDir) of + end. + +check_config_file(Callback, File)-> + case code:is_loaded(Callback) of + false -> + case code:load_file(Callback) of + {module,_} -> ok; + {error,Why} -> exit({cant_load_callback_module,Why}) + end; + _ -> + ok + end, + case Callback:check_parameter(File) of + {ok,{file,File}}-> + ?abs(File); + {ok,{config,_}}-> + File; + {error,{wrong_config,Message}}-> + exit({wrong_config,{Callback,Message}}); + {error,{nofile,File}}-> + exit({no_such_file,?abs(File)}) + end. + +run_dir(Opts = #opts{logdir = LogDir, + config = CfgFiles, + event_handlers = EvHandlers}, StartOpts) -> + LogDir1 = which(logdir, LogDir), + Opts1 = Opts#opts{logdir = LogDir1}, + AbsCfgFiles = + lists:map(fun({Callback,FileList})-> + case code:is_loaded(Callback) of + {file,_Path}-> + ok; + false -> + case code:load_file(Callback) of + {module,Callback}-> + ok; + {error,_}-> + exit({no_such_module,Callback}) + end + end, + {Callback, + lists:map(fun(File)-> + check_config_file(Callback, File) + end, FileList)} + end, CfgFiles), + case install([{config,AbsCfgFiles},{event_handler,EvHandlers}], LogDir1) of ok -> ok; {error,IReason} -> exit(IReason) end, - case lists:keysearch(dir,1,Opts) of + case lists:keysearch(dir, 1, StartOpts) of {value,{_,Dirs=[Dir|_]}} when not is_integer(Dir), length(Dirs)>1 -> %% multiple dirs (no suite) - do_run(tests(Dirs), [], StepOrCover, Opts, LogDir); + reformat_result(catch do_run(tests(Dirs), [], Opts1, StartOpts)); false -> % no dir %% fun for converting suite name to {Dir,Mod} tuple S2M = fun(S) when is_list(S) -> @@ -728,105 +876,131 @@ run_dir(LogDir, CfgFiles, EvHandlers, StepOrCover, Opts) -> (A) -> {".",A} end, - case lists:keysearch(suite, 1, Opts) of + case lists:keysearch(suite, 1, StartOpts) of {value,{_,Suite}} when is_integer(hd(Suite)) ; is_atom(Suite) -> {Dir,Mod} = S2M(Suite), - case listify(proplists:get_value(group, Opts, [])) ++ - listify(proplists:get_value(testcase, Opts, [])) of + case listify(proplists:get_value(group, StartOpts, [])) ++ + listify(proplists:get_value(testcase, StartOpts, [])) of [] -> - do_run(tests(Dir, listify(Mod)), [], StepOrCover, Opts, LogDir); + reformat_result(catch do_run(tests(Dir, listify(Mod)), + [], Opts1, StartOpts)); GsAndCs -> - do_run(tests(Dir, Mod, GsAndCs), [], StepOrCover, Opts, LogDir) + reformat_result(catch do_run(tests(Dir, Mod, GsAndCs), + [], Opts1, StartOpts)) end; {value,{_,Suites}} -> - do_run(tests(lists:map(S2M, Suites)), [], StepOrCover, Opts, LogDir); + reformat_result(catch do_run(tests(lists:map(S2M, Suites)), + [], Opts1, StartOpts)); _ -> exit(no_tests_specified) - end; + end; {value,{_,Dir}} -> - case lists:keysearch(suite, 1, Opts) of + case lists:keysearch(suite, 1, StartOpts) of {value,{_,Suite}} when is_integer(hd(Suite)) ; is_atom(Suite) -> - Mod = if is_atom(Suite) -> Suite; - true -> list_to_atom(Suite) + Mod = if is_atom(Suite) -> Suite; + true -> list_to_atom(Suite) end, - case listify(proplists:get_value(group, Opts, [])) ++ - listify(proplists:get_value(testcase, Opts, [])) of + case listify(proplists:get_value(group, StartOpts, [])) ++ + listify(proplists:get_value(testcase, StartOpts, [])) of [] -> - do_run(tests(Dir, listify(Mod)), [], StepOrCover, Opts, LogDir); + reformat_result(catch do_run(tests(Dir, listify(Mod)), + [], Opts1, StartOpts)); GsAndCs -> - do_run(tests(Dir, Mod, GsAndCs), [], StepOrCover, Opts, LogDir) + reformat_result(catch do_run(tests(Dir, Mod, GsAndCs), + [], Opts1, StartOpts)) end; {value,{_,Suites=[Suite|_]}} when is_list(Suite) -> Mods = lists:map(fun(Str) -> list_to_atom(Str) end, Suites), - do_run(tests(delistify(Dir), Mods), [], StepOrCover, Opts, LogDir); + reformat_result(catch do_run(tests(delistify(Dir), Mods), + [], Opts1, StartOpts)); {value,{_,Suites}} -> - do_run(tests(delistify(Dir), Suites), [], StepOrCover, Opts, LogDir); + reformat_result(catch do_run(tests(delistify(Dir), Suites), + [], Opts1, StartOpts)); false -> % no suite, only dir - do_run(tests(listify(Dir)), [], StepOrCover, Opts, LogDir) - end + reformat_result(catch do_run(tests(listify(Dir)), + [], Opts1, StartOpts)) + end end. %%%----------------------------------------------------------------- -%%% @hidden +%%% @spec run_testspec(TestSpec) -> Result +%%% TestSpec = [term()] %%% +%%% @doc Run test specified by <code>TestSpec</code>. The terms are +%%% the same as those used in test specification files. +%%% @equiv ct:run_testspec/1 +%%%----------------------------------------------------------------- -%% using testspec(s) as input for test run_testspec(TestSpec) -> {ok,Cwd} = file:get_cwd(), io:format("~nCommon Test starting (cwd is ~s)~n~n", [Cwd]), case catch run_testspec1(TestSpec) of - {'EXIT',Reason} -> + {'EXIT',Reason} -> file:set_cwd(Cwd), {error,Reason}; - Result -> + Result -> Result end. run_testspec1(TestSpec) -> - case ct_testspec:collect_tests_from_list(TestSpec,false) of - {error,CTReason} -> + case catch ct_testspec:collect_tests_from_list(TestSpec, false) of + {E,CTReason} when E == error ; E == 'EXIT' -> exit(CTReason); TS -> - {LogDir,TSCoverFile,CfgFiles,EvHandlers,Include} = - get_data_for_node(TS,node()), - case os:getenv("CT_INCLUDE_PATH") of - false -> - application:set_env(common_test, include, Include); - CtInclPath -> - EnvInclude = string:tokens(CtInclPath, [$:,$ ,$,]), - application:set_env(common_test, include, EnvInclude++Include) - end, - CoverOpt = if TSCoverFile == undef -> []; - true -> [{cover,TSCoverFile}] - end, - case get_configfiles(CfgFiles,[],LogDir,EvHandlers) of + Opts = get_data_for_node(TS, node()), + + AllInclude = + case os:getenv("CT_INCLUDE_PATH") of + false -> + Opts#opts.include; + CtInclPath -> + EnvInclude = string:tokens(CtInclPath, [$:,$ ,$,]), + EnvInclude++Opts#opts.include + end, + application:set_env(common_test, include, AllInclude), + LogDir1 = which(logdir,Opts#opts.logdir), + case check_and_install_configfiles(Opts#opts.config, LogDir1, + Opts#opts.event_handlers) of ok -> - {Run,Skip} = ct_testspec:prepare_tests(TS,node()), - do_run(Run,Skip,CoverOpt,[],LogDir); + Opts1 = Opts#opts{testspecs = [], + logdir = LogDir1, + include = AllInclude}, + {Run,Skip} = ct_testspec:prepare_tests(TS, node()), + reformat_result(catch do_run(Run, Skip, Opts1, [])); {error,GCFReason} -> exit(GCFReason) end end. - -get_data_for_node(#testspec{logdir=LogDirs, - cover=CoverFs, - config=Cfgs, - event_handler=EvHs, - include=Incl}, Node) -> - LogDir = case lists:keysearch(Node,1,LogDirs) of - {value,{Node,Dir}} -> Dir; - false -> "." +get_data_for_node(#testspec{label = Labels, + logdir = LogDirs, + cover = CoverFs, + config = Cfgs, + userconfig = UsrCfgs, + event_handler = EvHs, + include = Incl, + multiply_timetraps = MTs, + scale_timetraps = STs}, Node) -> + Label = proplists:get_value(Node, Labels), + LogDir = case proplists:get_value(Node, LogDirs) of + undefined -> "."; + Dir -> Dir end, - Cover = case lists:keysearch(Node,1,CoverFs) of - {value,{Node,CovFile}} -> CovFile; - false -> undef - end, - ConfigFiles = [F || {N,F} <- Cfgs, N==Node], + Cover = proplists:get_value(Node, CoverFs), + MT = proplists:get_value(Node, MTs), + ST = proplists:get_value(Node, STs), + ConfigFiles = [{?ct_config_txt,F} || {N,F} <- Cfgs, N==Node] ++ + [CBF || {N,CBF} <- UsrCfgs, N==Node], EvHandlers = [{H,A} || {N,H,A} <- EvHs, N==Node], Include = [I || {N,I} <- Incl, N==Node], - {LogDir,Cover,ConfigFiles,EvHandlers,Include}. - + #opts{label = Label, + logdir = LogDir, + cover = Cover, + config = ConfigFiles, + event_handlers = EvHandlers, + include = Include, + multiply_timetraps = MT, + scale_timetraps = ST}. refresh_logs(LogDir) -> {ok,Cwd} = file:get_cwd(), @@ -851,11 +1025,27 @@ refresh_logs(LogDir) -> end end. -which_logdir(".",Dir) -> +which(logdir, undefined) -> + "."; +which(logdir, Dir) -> Dir; -which_logdir(Dir,_) -> - Dir. - +which(multiply_timetraps, undefined) -> + 1; +which(multiply_timetraps, MT) -> + MT; +which(scale_timetraps, undefined) -> + false; +which(scale_timetraps, ST) -> + ST. + +choose_val(undefined, V1) -> + V1; +choose_val(V0, _V1) -> + V0. + +merge_vals(Vs) -> + lists:append(Vs). + listify([C|_]=Str) when is_integer(C) -> [Str]; listify(L) when is_list(L) -> L; listify(E) -> [E]. @@ -869,22 +1059,43 @@ delistify(E) -> E. %%% @equiv ct:run/3 run(TestDir, Suite, Cases) -> install([]), - do_run(tests(TestDir, Suite, Cases), []). + reformat_result(catch do_run(tests(TestDir, Suite, Cases), [])). %%%----------------------------------------------------------------- %%% @hidden %%% @equiv ct:run/2 run(TestDir, Suite) when is_list(TestDir), is_integer(hd(TestDir)) -> install([]), - do_run(tests(TestDir, Suite), []). + reformat_result(catch do_run(tests(TestDir, Suite), [])). %%%----------------------------------------------------------------- %%% @hidden %%% @equiv ct:run/1 run(TestDirs) -> install([]), - do_run(tests(TestDirs), []). + reformat_result(catch do_run(tests(TestDirs), [])). + +reformat_result({user_error,Reason}) -> + {error,Reason}; +reformat_result(Result) -> + Result. +suite_to_test(Suite) -> + {filename:dirname(Suite),list_to_atom(filename:rootname(filename:basename(Suite)))}. + +groups_and_cases(Gs, Cs) when ((Gs == undefined) or (Gs == [])) and + ((Cs == undefined) or (Cs == [])) -> + []; +groups_and_cases(Gs, Cs) when Gs == undefined ; Gs == [] -> + [list_to_atom(C) || C <- Cs]; +groups_and_cases(Gs, Cs) when Cs == undefined ; Cs == [] -> + [{list_to_atom(G),all} || G <- Gs]; +groups_and_cases([G], Cs) -> + [{list_to_atom(G),[list_to_atom(C) || C <- Cs]}]; +groups_and_cases([_,_|_] , Cs) when Cs =/= [] -> + {error,multiple_groups_and_cases}; +groups_and_cases(_Gs, _Cs) -> + {error,incorrect_group_or_case_option}. tests(TestDir, Suites, []) when is_list(TestDir), is_integer(hd(TestDir)) -> [{?testdir(TestDir,Suites),ensure_atom(Suites),all}]; @@ -901,30 +1112,52 @@ tests(TestDir) when is_list(TestDir), is_integer(hd(TestDir)) -> tests(TestDirs) when is_list(TestDirs), is_list(hd(TestDirs)) -> [{?testdir(TestDir,all),all,all} || TestDir <- TestDirs]. -do_run(Tests, Opt) -> - do_run(Tests, [], Opt, [], "."). +do_run(Tests, Misc) when is_list(Misc) -> + do_run(Tests, Misc, "."). -do_run(Tests, Opt, LogDir) -> - do_run(Tests, [], Opt, [], LogDir). +do_run(Tests, Misc, LogDir) when is_list(Misc) -> + Opts = + case proplists:get_value(step, Misc) of + undefined -> + #opts{}; + StepOpts -> + #opts{step = StepOpts} + end, + Opts1 = + case proplists:get_value(cover, Misc) of + undefined -> + Opts; + CoverFile -> + Opts#opts{cover = CoverFile} + end, + do_run(Tests, [], Opts1#opts{logdir = LogDir}, []). + +do_run(Tests, Skip, Opts, Args) -> + #opts{label = Label, cover = Cover} = Opts, + + %% label - used by ct_logs + TestLabel = + if Label == undefined -> undefined; + is_atom(Label) -> atom_to_list(Label); + is_list(Label) -> Label; + true -> undefined + end, + application:set_env(common_test, test_label, TestLabel), -do_run(Tests, Skip, Opt, Args, LogDir) -> case code:which(test_server) of non_existing -> exit({error,no_path_to_test_server}); _ -> - Opt1 = - case lists:keysearch(cover, 1, Opt) of - {value,{_,CoverFile}} -> - case ct_cover:get_spec(CoverFile) of - {error,Reason} -> - exit({error,Reason}); - Spec -> - [{cover_spec,Spec} | - lists:keydelete(cover, 1, Opt)] - end; - _ -> - Opt - end, + Opts1 = if Cover == undefined -> + Opts; + true -> + case ct_cover:get_spec(Cover) of + {error,Reason} -> + exit({error,Reason}); + CoverSpec -> + Opts#opts{coverspec = CoverSpec} + end + end, %% This env variable is used by test_server to determine %% which framework it runs under. case os:getenv("TEST_SERVER_FRAMEWORK") of @@ -935,60 +1168,58 @@ do_run(Tests, Skip, Opt, Args, LogDir) -> Other -> erlang:display(list_to_atom("Note: TEST_SERVER_FRAMEWORK = " ++ Other)) end, - case ct_util:start(LogDir) of + case ct_util:start(Opts#opts.logdir) of {error,interactive_mode} -> io:format("CT is started in interactive mode. " "To exit this mode, run ct:stop_interactive().\n" "To enter the interactive mode again, " "run ct:start_interactive()\n\n",[]), {error,interactive_mode}; - _Pid -> - %% save style sheet info - case lists:keysearch(stylesheet, 1, Args) of - {value,{_,SSFile}} -> - ct_util:set_testdata({stylesheet,SSFile}); - _ -> - ct_util:set_testdata({stylesheet,undefined}) - end, - - case lists:keysearch(silent_connections, 1, Args) of - {value,{silent_connections,undefined}} -> - ok; - {value,{silent_connections,[]}} -> + %% save stylesheet info + ct_util:set_testdata({stylesheet,Opts#opts.stylesheet}), + %% enable silent connections + case Opts#opts.silent_connections of + [] -> Conns = ct_util:override_silence_all_connections(), ct_logs:log("Silent connections", "~p", [Conns]); - {value,{silent_connections,Cs}} -> - Conns = lists:map(fun(S) when is_list(S) -> - list_to_atom(S); - (A) -> A - end, Cs), + Conns when is_list(Conns) -> ct_util:override_silence_connections(Conns), ct_logs:log("Silent connections", "~p", [Conns]); _ -> ok end, - log_ts_names(Args), + log_ts_names(Opts1#opts.testspecs), TestSuites = suite_tuples(Tests), - {SuiteMakeErrors,AllMakeErrors} = + {_TestSuites1,SuiteMakeErrors,AllMakeErrors} = case application:get_env(common_test, auto_compile) of {ok,false} -> - SuitesNotFound = verify_suites(TestSuites), - {SuitesNotFound,SuitesNotFound}; + {TestSuites1,SuitesNotFound} = + verify_suites(TestSuites), + {TestSuites1,SuitesNotFound,SuitesNotFound}; _ -> {SuiteErrs,HelpErrs} = auto_compile(TestSuites), - {SuiteErrs,SuiteErrs++HelpErrs} + {TestSuites,SuiteErrs,SuiteErrs++HelpErrs} end, case continue(AllMakeErrors) of true -> SavedErrors = save_make_errors(SuiteMakeErrors), ct_repeat:log_loop_info(Args), - {Tests1,Skip1} = final_tests(Tests,[],Skip,SavedErrors), - R = do_run_test(Tests1, Skip1, Opt1), - ct_util:stop(normal), - R; + + {Tests1,Skip1} = final_tests(Tests,Skip,SavedErrors), + + R = (catch do_run_test(Tests1, Skip1, Opts1)), + case R of + {EType,_} = Error when EType == user_error ; + EType == error -> + ct_util:stop(clean), + exit(Error); + _ -> + ct_util:stop(normal), + R + end; false -> io:nl(), ct_util:stop(clean), @@ -1012,7 +1243,7 @@ auto_compile(TestSuites) -> case application:get_env(common_test, include) of {ok,UserInclDirs} when length(UserInclDirs) > 0 -> io:format("Including the following directories:~n"), - [begin io:format("~p~n",[UserInclDir]), {i,UserInclDir} end || + [begin io:format("~p~n",[UserInclDir]), {i,UserInclDir} end || UserInclDir <- UserInclDirs]; _ -> [] @@ -1020,11 +1251,11 @@ auto_compile(TestSuites) -> SuiteMakeErrors = lists:flatmap(fun({TestDir,Suite} = TS) -> case run_make(suites, TestDir, Suite, UserInclude) of - {error,{make_failed,Bad}} -> + {error,{make_failed,Bad}} -> [{TS,Bad}]; - {error,_} -> + {error,_} -> [{TS,[filename:join(TestDir,"*_SUITE")]}]; - _ -> + _ -> [] end end, TestSuites), @@ -1048,39 +1279,63 @@ auto_compile(TestSuites) -> true -> % already visited {Done,Failed} end - end, {[],[]}, TestSuites), + end, {[],[]}, TestSuites), {SuiteMakeErrors,lists:reverse(HelpMakeErrors)}. %% verify that specified test suites exist (if auto compile is disabled) verify_suites(TestSuites) -> io:nl(), - Verify = - fun({Dir,Suite},NotFound) -> + Verify = + fun({Dir,Suite}=DS,{Found,NotFound}) -> case locate_test_dir(Dir, Suite) of {ok,TestDir} -> if Suite == all -> - NotFound; + {[DS|Found],NotFound}; true -> - Beam = filename:join(TestDir, atom_to_list(Suite)++".beam"), + Beam = filename:join(TestDir, + atom_to_list(Suite)++".beam"), case filelib:is_regular(Beam) of - true -> - NotFound; - false -> - Name = filename:join(TestDir, atom_to_list(Suite)), - io:format("Suite ~w not found in directory ~s~n", - [Suite,TestDir]), - [{{Dir,Suite},[Name]} | NotFound] + true -> + {[DS|Found],NotFound}; + false -> + case code:is_loaded(Suite) of + {file,SuiteFile} -> + %% test suite is already loaded and + %% since auto_compile == false, + %% let's assume the user has + %% loaded the beam file explicitly + ActualDir = filename:dirname(SuiteFile), + {[{ActualDir,Suite}|Found],NotFound}; + false -> + Name = + filename:join(TestDir, + atom_to_list(Suite)), + io:format(user, + "Suite ~w not found" + "in directory ~s~n", + [Suite,TestDir]), + {Found,[{DS,[Name]}|NotFound]} + end end end; {error,_Reason} -> - io:format("Directory ~s is invalid~n", [Dir]), - Name = filename:join(Dir, atom_to_list(Suite)), - [{{Dir,Suite},[Name]} | NotFound] + case code:is_loaded(Suite) of + {file,SuiteFile} -> + %% test suite is already loaded and since + %% auto_compile == false, let's assume the + %% user has loaded the beam file explicitly + ActualDir = filename:dirname(SuiteFile), + {[{ActualDir,Suite}|Found],NotFound}; + false -> + io:format(user, "Directory ~s is invalid~n", [Dir]), + Name = filename:join(Dir, atom_to_list(Suite)), + {Found,[{DS,[Name]}|NotFound]} + end end end, - lists:reverse(lists:foldl(Verify, [], TestSuites)). - - + {ActualFound,Missing} = lists:foldl(Verify, {[],[]}, TestSuites), + {lists:reverse(ActualFound),lists:reverse(Missing)}. + save_make_errors([]) -> []; save_make_errors(Errors) -> @@ -1096,7 +1351,7 @@ get_bad_suites([{{_TestDir,_Suite},Failed}|Errors], BadSuites) -> get_bad_suites([], BadSuites) -> BadSuites. - + %%%----------------------------------------------------------------- %%% @hidden @@ -1107,7 +1362,7 @@ step(TestDir, Suite, Case) -> %%%----------------------------------------------------------------- %%% @hidden %%% @equiv ct:step/4 -step(TestDir, Suite, Case, Opts) when is_list(TestDir), is_atom(Suite), is_atom(Case), +step(TestDir, Suite, Case, Opts) when is_list(TestDir), is_atom(Suite), is_atom(Case), Suite =/= all, Case =/= all -> do_run([{TestDir,Suite,Case}], [{step,Opts}]). @@ -1121,8 +1376,22 @@ suite_tuples([{TestDir,Suite,_} | Tests]) when is_atom(Suite) -> suite_tuples([]) -> []. -final_tests([{TestDir,Suites,_}|Tests], - Final, Skip, Bad) when is_list(Suites), is_atom(hd(Suites)) -> +final_tests(Tests, Skip, Bad) -> + + %%! --- Thu Jun 24 15:47:27 2010 --- peppe was here! + %%! io:format(user, "FINAL0 = ~p~nSKIP0 = ~p~n", [Tests, Skip]), + + {Tests1,Skip1} = final_tests1(Tests, [], Skip, Bad), + Skip2 = final_skip(Skip1, []), + + + %%! --- Thu Jun 24 15:47:27 2010 --- peppe was here! + %%! io:format(user, "FINAL1 = ~p~nSKIP1 = ~p~n", [Tests1, Skip2]), + + {Tests1,Skip2}. + +final_tests1([{TestDir,Suites,_}|Tests], Final, Skip, Bad) when + is_list(Suites), is_atom(hd(Suites)) -> % Separate = % fun(S,{DoSuite,Dont}) -> % case lists:keymember({TestDir,S},1,Bad) of @@ -1140,10 +1409,10 @@ final_tests([{TestDir,Suites,_}|Tests], Skip1 = [{TD,S,"Make failed"} || {{TD,S},_} <- Bad, S1 <- Suites, S == S1, TD == TestDir], - Final1 = [{TestDir,S,all} || S <- Suites], - final_tests(Tests, lists:reverse(Final1)++Final, Skip++Skip1, Bad); + Final1 = [{TestDir,S,all} || S <- Suites], + final_tests1(Tests, lists:reverse(Final1)++Final, Skip++Skip1, Bad); -final_tests([{TestDir,all,all}|Tests], Final, Skip, Bad) -> +final_tests1([{TestDir,all,all}|Tests], Final, Skip, Bad) -> MissingSuites = case lists:keysearch({TestDir,all}, 1, Bad) of {value,{_,Failed}} -> @@ -1153,27 +1422,59 @@ final_tests([{TestDir,all,all}|Tests], Final, Skip, Bad) -> end, Missing = [{TestDir,S,"Make failed"} || S <- MissingSuites], Final1 = [{TestDir,all,all}|Final], - final_tests(Tests, Final1, Skip++Missing, Bad); + final_tests1(Tests, Final1, Skip++Missing, Bad); -final_tests([{TestDir,Suite,Cases}|Tests], - Final, Skip, Bad) when Cases==[]; Cases==all -> - final_tests([{TestDir,[Suite],all}|Tests], Final, Skip, Bad); +final_tests1([{TestDir,Suite,Cases}|Tests], Final, Skip, Bad) when + Cases==[]; Cases==all -> + final_tests1([{TestDir,[Suite],all}|Tests], Final, Skip, Bad); -final_tests([{TestDir,Suite,Cases}|Tests], Final, Skip, Bad) -> +final_tests1([{TestDir,Suite,GrsOrCs}|Tests], Final, Skip, Bad) when + is_list(GrsOrCs) -> case lists:keymember({TestDir,Suite}, 1, Bad) of - false -> - Do = {TestDir,Suite,Cases}, - final_tests(Tests, [Do|Final], Skip, Bad); true -> - Do = {TestDir,Suite,Cases}, - Skip1 = Skip ++ [{TestDir,Suite,Cases,"Make failed"}], - final_tests(Tests, [Do|Final], Skip1, Bad) + Skip1 = Skip ++ [{TestDir,Suite,all,"Make failed"}], + final_tests1(Tests, [{TestDir,Suite,all}|Final], Skip1, Bad); + false -> + GrsOrCs1 = + lists:flatmap( + %% for now, only flat group defs are allowed as + %% start options and test spec terms + fun({all,all}) -> + ct_framework:make_all_conf(TestDir, + Suite, []); + ({skipped,Group,TCs}) -> + [ct_framework:make_conf(TestDir, Suite, + Group, [skipped], TCs)]; + ({Group,TCs}) -> + [ct_framework:make_conf(TestDir, Suite, + Group, [], TCs)]; + (TC) -> + [TC] + end, GrsOrCs), + Do = {TestDir,Suite,GrsOrCs1}, + final_tests1(Tests, [Do|Final], Skip, Bad) end; -final_tests([], Final, Skip, _Bad) -> +final_tests1([], Final, Skip, _Bad) -> {lists:reverse(Final),Skip}. -continue([]) -> +final_skip([{TestDir,Suite,{all,all},Reason}|Skips], Final) -> + SkipConf = ct_framework:make_conf(TestDir, Suite, all, [], all), + Skip = {TestDir,Suite,SkipConf,Reason}, + final_skip(Skips, [Skip|Final]); + +final_skip([{TestDir,Suite,{Group,TCs},Reason}|Skips], Final) -> + Conf = ct_framework:make_conf(TestDir, Suite, Group, [], TCs), + Skip = {TestDir,Suite,Conf,Reason}, + final_skip(Skips, [Skip|Final]); + +final_skip([Skip|Skips], Final) -> + final_skip(Skips, [Skip|Final]); + +final_skip([], Final) -> + lists:reverse(Final). + +continue([]) -> true; continue(_MakeErrors) -> io:nl(), @@ -1214,7 +1515,7 @@ set_group_leader_same_as_shell() -> false end end, - case [P || P <- processes(), GS2or3(P), + case [P || P <- processes(), GS2or3(P), true == lists:keymember(shell,1,element(2,process_info(P,dictionary)))] of [GL|_] -> group_leader(GL, self()); @@ -1238,29 +1539,29 @@ check_and_add([{TestDir0,M,_} | Tests], Added) -> check_and_add([], _) -> ok. -do_run_test(Tests, Skip, Opt) -> +do_run_test(Tests, Skip, Opts) -> case check_and_add(Tests, []) of ok -> ct_util:set_testdata({stats,{0,0,{0,0}}}), ct_util:set_testdata({cover,undefined}), test_server_ctrl:start_link(local), - case lists:keysearch(cover_spec, 1, Opt) of - {value,{_,CovData={CovFile, - CovNodes, - _CovImport, - CovExport, - #cover{app = CovApp, - level = CovLevel, - excl_mods = CovExcl, - incl_mods = CovIncl, - cross = CovCross, - src = _CovSrc}}}} -> + case Opts#opts.coverspec of + CovData={CovFile, + CovNodes, + _CovImport, + CovExport, + #cover{app = CovApp, + level = CovLevel, + excl_mods = CovExcl, + incl_mods = CovIncl, + cross = CovCross, + src = _CovSrc}} -> ct_logs:log("COVER INFO","Using cover specification file: ~s~n" "App: ~w~n" "Cross cover: ~w~n" "Including ~w modules~n" "Excluding ~w modules", - [CovFile,CovApp,CovCross,length(CovIncl),length(CovExcl)]), + [CovFile,CovApp,CovCross,length(CovIncl),length(CovExcl)]), %% cover export file will be used for export and import %% between tests so make sure it doesn't exist initially @@ -1293,33 +1594,38 @@ do_run_test(Tests, Skip, Opt) -> true; _ -> false - end, + end, + %% let test_server expand the test tuples and count no of cases {Suites,NoOfCases} = count_test_cases(Tests, Skip), Suites1 = delete_dups(Suites), NoOfTests = length(Tests), NoOfSuites = length(Suites1), - ct_util:warn_duplicates(Suites1), + ct_util:warn_duplicates(Suites1), {ok,Cwd} = file:get_cwd(), io:format("~nCWD set to: ~p~n", [Cwd]), if NoOfCases == unknown -> - io:format("~nTEST INFO: ~w test(s), ~w suite(s)~n~n", + io:format("~nTEST INFO: ~w test(s), ~w suite(s)~n~n", [NoOfTests,NoOfSuites]), - ct_logs:log("TEST INFO","~w test(s), ~w suite(s)", + ct_logs:log("TEST INFO","~w test(s), ~w suite(s)", [NoOfTests,NoOfSuites]); true -> - io:format("~nTEST INFO: ~w test(s), ~w case(s) in ~w suite(s)~n~n", + io:format("~nTEST INFO: ~w test(s), ~w case(s) in ~w suite(s)~n~n", [NoOfTests,NoOfCases,NoOfSuites]), - ct_logs:log("TEST INFO","~w test(s), ~w case(s) in ~w suite(s)", + ct_logs:log("TEST INFO","~w test(s), ~w case(s) in ~w suite(s)", [NoOfTests,NoOfCases,NoOfSuites]) end, + + test_server_ctrl:multiply_timetraps(Opts#opts.multiply_timetraps), + test_server_ctrl:scale_timetraps(Opts#opts.scale_timetraps), + ct_event:notify(#event{name=start_info, node=node(), data={NoOfTests,NoOfSuites,NoOfCases}}), - CleanUp = add_jobs(Tests, Skip, Opt, []), + CleanUp = add_jobs(Tests, Skip, Opts, []), unlink(whereis(test_server_ctrl)), - catch test_server_ctrl:wait_finish(), - %% check if last testcase has left a "dead" trace window + catch test_server_ctrl:wait_finish(), + %% check if last testcase has left a "dead" trace window %% behind, and if so, kill it case ct_util:get_testdata(interpret) of {_What,kill,{TCPid,AttPid}} -> @@ -1327,8 +1633,8 @@ do_run_test(Tests, Skip, Opt) -> _ -> ok end, - lists:foreach(fun(Suite) -> - maybe_cleanup_interpret(Suite, Opt) + lists:foreach(fun(Suite) -> + maybe_cleanup_interpret(Suite, Opts#opts.step) end, CleanUp); Error -> Error @@ -1344,21 +1650,32 @@ count_test_cases(Tests, Skip) -> SendResult = fun(Me, Result) -> Me ! {no_of_cases,Result} end, TSPid = test_server_ctrl:start_get_totals(SendResult), Ref = erlang:monitor(process, TSPid), - add_jobs(Tests, Skip, [], []), - {Suites,NoOfCases} = count_test_cases1(length(Tests), 0, [], Ref), - erlang:demonitor(Ref), - test_server_ctrl:stop_get_totals(), - {Suites,NoOfCases}. + add_jobs(Tests, Skip, #opts{}, []), + Counted = (catch count_test_cases1(length(Tests), 0, [], Ref)), + erlang:demonitor(Ref, [flush]), + case Counted of + {error,{test_server_died}} = Error -> + throw(Error); + {error,Reason} -> + unlink(whereis(test_server_ctrl)), + test_server_ctrl:stop(), + throw({user_error,Reason}); + Result -> + test_server_ctrl:stop_get_totals(), + Result + end. count_test_cases1(0, N, Suites, _) -> {lists:flatten(Suites), N}; count_test_cases1(Jobs, N, Suites, Ref) -> receive - {no_of_cases,{Ss,N1}} -> + {_,{error,_Reason} = Error} -> + throw(Error); + {no_of_cases,{Ss,N1}} -> count_test_cases1(Jobs-1, add_known(N,N1), [Ss|Suites], Ref); - {'DOWN', Ref, _, _, _} -> - {[],0} - end. + {'DOWN', Ref, _, _, Info} -> + throw({error,{test_server_died,Info}}) + end. add_known(unknown, _) -> unknown; @@ -1367,72 +1684,109 @@ add_known(_, unknown) -> add_known(N, N1) -> N+N1. -add_jobs([{TestDir,all,_}|Tests], Skip, Opt, CleanUp) -> +add_jobs([{TestDir,all,_}|Tests], Skip, Opts, CleanUp) -> Name = get_name(TestDir), case catch test_server_ctrl:add_dir_with_skip(Name, TestDir, skiplist(TestDir,Skip)) of - {'EXIT',_} -> + {'EXIT',_} -> CleanUp; _ -> wait_for_idle(), - add_jobs(Tests, Skip, Opt, CleanUp) + add_jobs(Tests, Skip, Opts, CleanUp) end; -add_jobs([{TestDir,[Suite],all}|Tests], Skip, Opt, CleanUp) when is_atom(Suite) -> - add_jobs([{TestDir,Suite,all}|Tests], Skip, Opt, CleanUp); -add_jobs([{TestDir,Suites,all}|Tests], Skip, Opt, CleanUp) when is_list(Suites) -> +add_jobs([{TestDir,[Suite],all}|Tests], Skip, Opts, CleanUp) when is_atom(Suite) -> + add_jobs([{TestDir,Suite,all}|Tests], Skip, Opts, CleanUp); +add_jobs([{TestDir,Suites,all}|Tests], Skip, Opts, CleanUp) when is_list(Suites) -> Name = get_name(TestDir) ++ ".suites", case catch test_server_ctrl:add_module_with_skip(Name, Suites, skiplist(TestDir,Skip)) of - {'EXIT',_} -> + {'EXIT',_} -> CleanUp; _ -> wait_for_idle(), - add_jobs(Tests, Skip, Opt, CleanUp) + add_jobs(Tests, Skip, Opts, CleanUp) end; -add_jobs([{TestDir,Suite,all}|Tests], Skip, Opt, CleanUp) -> - case maybe_interpret(Suite, all, Opt) of +add_jobs([{TestDir,Suite,all}|Tests], Skip, Opts, CleanUp) -> + case maybe_interpret(Suite, all, Opts) of ok -> Name = get_name(TestDir) ++ "." ++ atom_to_list(Suite), case catch test_server_ctrl:add_module_with_skip(Name, [Suite], skiplist(TestDir,Skip)) of - {'EXIT',_} -> + {'EXIT',_} -> CleanUp; _ -> wait_for_idle(), - add_jobs(Tests, Skip, Opt, [Suite|CleanUp]) + add_jobs(Tests, Skip, Opts, [Suite|CleanUp]) end; Error -> Error end; -add_jobs([{TestDir,Suite,[Case]}|Tests], Skip, Opt, CleanUp) when is_atom(Case) -> - add_jobs([{TestDir,Suite,Case}|Tests], Skip, Opt, CleanUp); -add_jobs([{TestDir,Suite,Cases}|Tests], Skip, Opt, CleanUp) when is_list(Cases) -> - case maybe_interpret(Suite, Cases, Opt) of + +%% group (= conf case in test_server) +add_jobs([{TestDir,Suite,Confs}|Tests], Skip, Opts, CleanUp) when + element(1, hd(Confs)) == conf -> + Group = fun(Conf) -> proplists:get_value(name, element(2, Conf)) end, + TestCases = fun(Conf) -> element(4, Conf) end, + TCTestName = fun(all) -> ""; + ([C]) when is_atom(C) -> "." ++ atom_to_list(C); + (Cs) when is_list(Cs) -> ".cases" + end, + GrTestName = + case Confs of + [Conf] -> + "." ++ atom_to_list(Group(Conf)) ++ TCTestName(TestCases(Conf)); + _ -> + ".groups" + end, + TestName = get_name(TestDir) ++ "." ++ atom_to_list(Suite) ++ GrTestName, + case maybe_interpret(Suite, init_per_group, Opts) of + ok -> + case catch test_server_ctrl:add_conf_with_skip(TestName, Suite, Confs, + skiplist(TestDir,Skip)) of + {'EXIT',_} -> + CleanUp; + _ -> + wait_for_idle(), + add_jobs(Tests, Skip, Opts, [Suite|CleanUp]) + end; + Error -> + Error + end; + +%% test case +add_jobs([{TestDir,Suite,[Case]}|Tests], Skip, Opts, CleanUp) when is_atom(Case) -> + add_jobs([{TestDir,Suite,Case}|Tests], Skip, Opts, CleanUp); + +add_jobs([{TestDir,Suite,Cases}|Tests], Skip, Opts, CleanUp) when is_list(Cases) -> + Cases1 = lists:map(fun({GroupName,_}) when is_atom(GroupName) -> GroupName; + (Case) -> Case + end, Cases), + case maybe_interpret(Suite, Cases1, Opts) of ok -> Name = get_name(TestDir) ++ "." ++ atom_to_list(Suite) ++ ".cases", - case catch test_server_ctrl:add_cases_with_skip(Name, Suite, Cases, + case catch test_server_ctrl:add_cases_with_skip(Name, Suite, Cases1, skiplist(TestDir,Skip)) of - {'EXIT',_} -> + {'EXIT',_} -> CleanUp; _ -> wait_for_idle(), - add_jobs(Tests, Skip, Opt, [Suite|CleanUp]) + add_jobs(Tests, Skip, Opts, [Suite|CleanUp]) end; Error -> Error end; -add_jobs([{TestDir,Suite,Case}|Tests], Skip, Opt, CleanUp) when is_atom(Case) -> - case maybe_interpret(Suite, Case, Opt) of +add_jobs([{TestDir,Suite,Case}|Tests], Skip, Opts, CleanUp) when is_atom(Case) -> + case maybe_interpret(Suite, Case, Opts) of ok -> - Name = get_name(TestDir) ++ "." ++ atom_to_list(Suite) ++ "." ++ + Name = get_name(TestDir) ++ "." ++ atom_to_list(Suite) ++ "." ++ atom_to_list(Case), case catch test_server_ctrl:add_case_with_skip(Name, Suite, Case, skiplist(TestDir,Skip)) of - {'EXIT',_} -> + {'EXIT',_} -> CleanUp; _ -> wait_for_idle(), - add_jobs(Tests, Skip, Opt, [Suite|CleanUp]) + add_jobs(Tests, Skip, Opts, [Suite|CleanUp]) end; Error -> Error @@ -1453,7 +1807,7 @@ wait_for_idle() -> idle -> ok; {'DOWN', Ref, _, _, _} -> error end, - erlang:demonitor(Ref), + erlang:demonitor(Ref, [flush]), ct_util:update_last_run_index(), Result end. @@ -1482,7 +1836,7 @@ get_name(Dir) -> end, Base = filename:basename(TestDir), case filename:basename(filename:dirname(TestDir)) of - "" -> + "" -> Base; TopDir -> TopDir ++ "." ++ Base @@ -1513,15 +1867,15 @@ run_make(Targets, TestDir0, Mod, UserInclude) -> {i,CtInclude}, {i,XmerlInclude}, debug_info], - Result = + Result = if Mod == all ; Targets == helpmods -> case (catch ct_make:all([noexec|ErlFlags])) of - {'EXIT',_} = Failure -> + {'EXIT',_} = Failure -> Failure; MakeInfo -> FileTest = fun(F, suites) -> is_suite(F); - (F, helpmods) -> not is_suite(F); - (_, _) -> true end, + (F, helpmods) -> not is_suite(F) + end, Files = lists:flatmap(fun({F,out_of_date}) -> case FileTest(F, Targets) of true -> [F]; @@ -1535,7 +1889,7 @@ run_make(Targets, TestDir0, Mod, UserInclude) -> true -> (catch ct_make:files([Mod], [load|ErlFlags])) end, - + ok = file:set_cwd(Cwd), %% send finished_make notification ct_event:notify(#event{name=finished_make, @@ -1549,7 +1903,7 @@ run_make(Targets, TestDir0, Mod, UserInclude) -> {error,{make_crashed,TestDir,Reason}}; {error,ModInfo} -> io:format("{error,make_failed}\n", []), - Bad = [filename:join(TestDir, M) || {M,R} <- ModInfo, + Bad = [filename:join(TestDir, M) || {M,R} <- ModInfo, R == error], {error,{make_failed,Bad}} end; @@ -1561,8 +1915,8 @@ run_make(Targets, TestDir0, Mod, UserInclude) -> get_dir(App, Dir) -> filename:join(code:lib_dir(App), Dir). -maybe_interpret(Suite, Cases, [{step,StepOpts}]) -> - %% if other suite has run before this one, check if last testcase +maybe_interpret(Suite, Cases, #opts{step = StepOpts}) when StepOpts =/= undefined -> + %% if other suite has run before this one, check if last testcase %% has left a "dead" trace window behind, and if so, kill it case ct_util:get_testdata(interpret) of {_What,kill,{TCPid,AttPid}} -> @@ -1605,7 +1959,7 @@ maybe_interpret2(Suite, Cases, StepOpts) -> WinOp = case lists:member(keep_inactive, ensure_atom(StepOpts)) of true -> no_kill; false -> kill - end, + end, ct_util:set_testdata({interpret,{{Suite,Cases},WinOp, {undefined,undefined}}}), ok. @@ -1621,37 +1975,44 @@ set_break_on_config(Suite, StepOpts) -> ok end. -maybe_cleanup_interpret(Suite, [{step,_}]) -> - i:iq(Suite); -maybe_cleanup_interpret(_, _) -> - ok. +maybe_cleanup_interpret(_, undefined) -> + ok; +maybe_cleanup_interpret(Suite, _) -> + i:iq(Suite). + +log_ts_names([]) -> + ok; +log_ts_names(Specs) -> + List = lists:map(fun(Name) -> + Name ++ " " + end, Specs), + ct_logs:log("Test Specification file(s)", "~s", + [lists:flatten(List)]). -log_ts_names(Args) -> - case lists:keysearch(spec, 1, Args) of - {value,{_,Specs}} -> - List = lists:map(fun(Name) -> - Name ++ " " - end, Specs), - ct_logs:log("Test Specification file(s)", "~s", - [lists:flatten(List)]); - _ -> - ok - end. - merge_arguments(Args) -> merge_arguments(Args, []). merge_arguments([LogDir={logdir,_}|Args], Merged) -> merge_arguments(Args, handle_arg(replace, LogDir, Merged)); + merge_arguments([CoverFile={cover,_}|Args], Merged) -> merge_arguments(Args, handle_arg(replace, CoverFile, Merged)); -merge_arguments([Arg={_,_}|Args], Merged) -> + +merge_arguments([{'case',TC}|Args], Merged) -> + merge_arguments(Args, handle_arg(merge, {testcase,TC}, Merged)); + +merge_arguments([Arg|Args], Merged) -> merge_arguments(Args, handle_arg(merge, Arg, Merged)); + merge_arguments([], Merged) -> Merged. handle_arg(replace, {Key,Elems}, [{Key,_}|Merged]) -> [{Key,Elems}|Merged]; +handle_arg(merge, {event_handler_init,Elems}, [{event_handler_init,PrevElems}|Merged]) -> + [{event_handler_init,PrevElems++["add"|Elems]}|Merged]; +handle_arg(merge, {userconfig,Elems}, [{userconfig,PrevElems}|Merged]) -> + [{userconfig,PrevElems++["add"|Elems]}|Merged]; handle_arg(merge, {Key,Elems}, [{Key,PrevElems}|Merged]) -> [{Key,PrevElems++Elems}|Merged]; handle_arg(Op, Arg, [Other|Merged]) -> @@ -1659,6 +2020,164 @@ handle_arg(Op, Arg, [Other|Merged]) -> handle_arg(_,Arg,[]) -> [Arg]. +get_start_opt(Key, IfExists, Args) -> + get_start_opt(Key, IfExists, undefined, Args). + +get_start_opt(Key, IfExists, IfNotExists, Args) -> + case lists:keysearch(Key, 1, Args) of + {value,{Key,Val}} when is_function(IfExists) -> + IfExists(Val); + {value,{Key,Val}} when IfExists == value -> + Val; + {value,{Key,_Val}} -> + IfExists; + _ when is_function(IfNotExists) -> + IfNotExists(); + _ -> + IfNotExists + end. + +event_handler_args2opts(Args) -> + case proplists:get_value(event_handler, Args) of + undefined -> + event_handler_args2opts([], Args); + EHs -> + event_handler_args2opts([{list_to_atom(EH),[]} || EH <- EHs], Args) + end. +event_handler_args2opts(Default, Args) -> + case proplists:get_value(event_handler_init, Args) of + undefined -> + Default; + EHs -> + event_handler_init_args2opts(EHs) + end. +event_handler_init_args2opts([EH, Arg, "and" | EHs]) -> + [{list_to_atom(EH),lists:flatten(io_lib:format("~s",[Arg]))} | + event_handler_init_args2opts(EHs)]; +event_handler_init_args2opts([EH, Arg]) -> + [{list_to_atom(EH),lists:flatten(io_lib:format("~s",[Arg]))}]; +event_handler_init_args2opts([]) -> + []. + +%% This function reads pa and pz arguments, converts dirs from relative +%% to absolute, and re-inserts them in the code path. The order of the +%% dirs in the code path remain the same. Note however that since this +%% function is only used for arguments "pre run_test erl_args", the order +%% relative dirs "post run_test erl_args" is not kept! +rel_to_abs(CtArgs) -> + {PA,PZ} = get_pa_pz(CtArgs, [], []), + io:format(user, "~n", []), + [begin + code:del_path(filename:basename(D)), + Abs = filename:absname(D), + code:add_pathz(Abs), + if D /= Abs -> + io:format(user, "Converting ~p to ~p and re-inserting " + "with add_pathz/1~n", + [D, Abs]); + true -> + ok + end + end || D <- PZ], + [begin + code:del_path(filename:basename(D)), + Abs = filename:absname(D), + code:add_patha(Abs), + if D /= Abs -> + io:format(user, "Converting ~p to ~p and re-inserting " + "with add_patha/1~n", + [D, Abs]); + true ->ok + end + end || D <- PA], + io:format(user, "~n", []). + +get_pa_pz([{pa,Dirs} | Args], PA, PZ) -> + get_pa_pz(Args, PA ++ Dirs, PZ); +get_pa_pz([{pz,Dirs} | Args], PA, PZ) -> + get_pa_pz(Args, PA, PZ ++ Dirs); +get_pa_pz([_ | Args], PA, PZ) -> + get_pa_pz(Args, PA, PZ); +get_pa_pz([], PA, PZ) -> + {PA,PZ}. + +%% This function translates ct:run_test/1 start options +%% to run_test start arguments (on the init arguments format) - +%% this is useful mainly for testing the ct_run start functions. +opts2args(EnvStartOpts) -> + lists:flatmap(fun({config,CfgFiles}) -> + [{ct_config,[CfgFiles]}]; + ({userconfig,{CBM,CfgStr=[X|_]}}) when is_integer(X) -> + [{userconfig,[atom_to_list(CBM),CfgStr]}]; + ({userconfig,{CBM,CfgStrs}}) when is_list(CfgStrs) -> + [{userconfig,[atom_to_list(CBM) | CfgStrs]}]; + ({userconfig,UserCfg}) when is_list(UserCfg) -> + Strs = + lists:map(fun({CBM,CfgStr=[X|_]}) when is_integer(X) -> + [atom_to_list(CBM),CfgStr,"and"]; + ({CBM,CfgStrs}) when is_list(CfgStrs) -> + [atom_to_list(CBM) | CfgStrs] ++ ["and"] + end, UserCfg), + [_LastAnd|StrsR] = lists:reverse(lists:flatten(Strs)), + [{userconfig,lists:reverse(StrsR)}]; + ({testcase,Case}) when is_atom(Case) -> + [{'case',[atom_to_list(Case)]}]; + ({testcase,Cases}) -> + [{'case',[atom_to_list(C) || C <- Cases]}]; + ({'case',Cases}) -> + [{'case',[atom_to_list(C) || C <- Cases]}]; + ({allow_user_terms,true}) -> + [{allow_user_terms,[]}]; + ({allow_user_terms,false}) -> + []; + ({auto_compile,false}) -> + [{no_auto_compile,[]}]; + ({auto_compile,true}) -> + []; + ({scale_timetraps,true}) -> + [{scale_timetraps,[]}]; + ({scale_timetraps,false}) -> + []; + ({force_stop,true}) -> + [{force_stop,[]}]; + ({force_stop,false}) -> + []; + ({decrypt,{key,Key}}) -> + [{ct_decrypt_key,[Key]}]; + ({decrypt,{file,File}}) -> + [{ct_decrypt_file,[File]}]; + ({basic_html,true}) -> + ({basic_html,[]}); + ({basic_html,false}) -> + []; + ({event_handler,EH}) when is_atom(EH) -> + [{event_handler,[atom_to_list(EH)]}]; + ({event_handler,EHs}) when is_list(EHs) -> + [{event_handler,[atom_to_list(EH) || EH <- EHs]}]; + ({event_handler,{EH,Arg}}) when is_atom(EH) -> + ArgStr = lists:flatten(io_lib:format("~p", [Arg])), + [{event_handler_init,[atom_to_list(EH),ArgStr]}]; + ({event_handler,{EHs,Arg}}) when is_list(EHs) -> + ArgStr = lists:flatten(io_lib:format("~p", [Arg])), + Strs = lists:map(fun(EH) -> + [atom_to_list(EH),ArgStr,"and"] + end, EHs), + [_LastAnd|StrsR] = lists:reverse(lists:flatten(Strs)), + [{event_handler_init,lists:reverse(StrsR)}]; + ({Opt,As=[A|_]}) when is_atom(A) -> + [{Opt,[atom_to_list(Atom) || Atom <- As]}]; + ({Opt,Strs=[S|_]}) when is_list(S) -> + [{Opt,Strs}]; + ({Opt,A}) when is_atom(A) -> + [{Opt,[atom_to_list(A)]}]; + ({Opt,I}) when is_integer(I) -> + [{Opt,[integer_to_list(I)]}]; + ({Opt,S}) when is_list(S) -> + [{Opt,[S]}]; + (Opt) -> + Opt + end, EnvStartOpts). + locate_test_dir(Dir, Suite) -> TestDir = case ct_util:is_test_dir(Dir) of true -> Dir; @@ -1723,18 +2242,18 @@ start_trace(Args) -> case file:consult(TraceSpec) of {ok,Terms} -> case catch do_trace(Terms) of - ok -> + ok -> true; {_,Error} -> io:format("Warning! Tracing not started. Reason: ~p~n~n", [Error]), false - end; + end; {_,Error} -> io:format("Warning! Tracing not started. Reason: ~p~n~n", [Error]), false - end; + end; false -> false end. @@ -1746,61 +2265,22 @@ do_trace(Terms) -> case dbg:tpl(M,[{'_',[],[{return_trace}]}]) of {error,What} -> exit({error,{tracing_failed,What}}); _ -> ok - end; + end; ({f,M,F}) -> case dbg:tpl(M,F,[{'_',[],[{return_trace}]}]) of {error,What} -> exit({error,{tracing_failed,What}}); _ -> ok - end; + end; (Huh) -> exit({error,{unrecognized_trace_term,Huh}}) end, Terms), ok. - + stop_trace(true) -> dbg:stop_clear(); stop_trace(false) -> ok. -preload() -> - io:format("~nLoading Common Test and Test Server modules...~n~n"), - preload_mod([ct_logs, - ct_make, - ct_telnet, - ct, - ct_master, - ct_testspec, - ct_cover, - ct_master_event, - ct_util, - ct_event, - ct_master_logs, - ct_framework, - teln, - ct_ftp, - ct_rpc, - unix_telnet, - ct_gen_conn, - ct_line, - ct_snmp, - test_server_sup, - test_server, - test_server_ctrl, - test_server_h, - test_server_line, - test_server_node]). - -preload_mod([M|Ms]) -> - case code:is_loaded(M) of - false -> - {module,M} = code:load_file(M), - preload_mod(Ms); - _ -> - ok - end; -preload_mod([]) -> - ok. - ensure_atom(Atom) when is_atom(Atom) -> Atom; ensure_atom(String) when is_list(String), is_integer(hd(String)) -> @@ -1809,4 +2289,3 @@ ensure_atom(List) when is_list(List) -> [ensure_atom(Item) || Item <- List]; ensure_atom(Other) -> Other. - diff --git a/lib/common_test/src/ct_slave.erl b/lib/common_test/src/ct_slave.erl new file mode 100644 index 0000000000..d2a491e079 --- /dev/null +++ b/lib/common_test/src/ct_slave.erl @@ -0,0 +1,439 @@ +%%-------------------------------------------------------------------- +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% + +%%% @doc Common Test Framework functions for starting and stopping nodes for +%%% Large Scale Testing. +%%% +%%% <p>This module exports functions which are used by the Common Test Master +%%% to start and stop "slave" nodes. It is the default callback module for the +%%% <code>{init, node_start}</code> term of the Test Specification.</p> + +%%---------------------------------------------------------------------- +%% File : ct_slave.erl +%% Description : CT module for starting nodes for large-scale testing. +%% +%% Created : 7 April 2010 +%%---------------------------------------------------------------------- +-module(ct_slave). + +-export([start/1, start/2, start/3, stop/1, stop/2]). + +-export([slave_started/2, slave_ready/2, monitor_master/1]). + +-record(options, {username, password, boot_timeout, init_timeout, + startup_timeout, startup_functions, monitor_master, + kill_if_fail, erl_flags}). + +%%%----------------------------------------------------------------- +%%% @spec start(Node) -> Result +%%% Node = atom() +%%% Result = {ok, NodeName} | +%%% {error, already_started, NodeName} | +%%% {error, started_not_connected, NodeName} | +%%% {error, boot_timeout, NodeName} | +%%% {error, init_timeout, NodeName} | +%%% {error, startup_timeout, NodeName} | +%%% {error, not_alive, NodeName} +%%% NodeName = atom() +%%% @doc Starts an Erlang node with name <code>Node</code> on the local host. +%%% @see start/3 +start(Node)-> + start(gethostname(), Node). + +%%%----------------------------------------------------------------- +%%% @spec start(Host, Node) -> Result +%%% Node = atom() +%%% Host = atom() +%%% Result = {ok, NodeName} | +%%% {error, already_started, NodeName} | +%%% {error, started_not_connected, NodeName} | +%%% {error, boot_timeout, NodeName} | +%%% {error, init_timeout, NodeName} | +%%% {error, startup_timeout, NodeName} | +%%% {error, not_alive, NodeName} +%%% NodeName = atom() +%%% @doc Starts an Erlang node with name <code>Node</code> on host +%%% <code>Host</code> with the default options. +%%% @see start/3 +start(Host, Node)-> + start(Host, Node, []). + +%%%----------------------------------------------------------------- +%%% @spec start(Host, Node, Opts) -> Result +%%% Node = atom() +%%% Host = atom() +%%% Opts = [OptTuples] +%%% OptTuples = {username, Username} | +%%% {password, Password} | +%%% {boot_timeout, BootTimeout} | {init_timeout, InitTimeout} | +%%% {startup_timeout, StartupTimeout} | +%%% {startup_functions, StartupFunctions} | +%%% {monitor_master, Monitor} | +%%% {kill_if_fail, KillIfFail} | +%%% {erl_flags, ErlangFlags} +%%% Username = string() +%%% Password = string() +%%% BootTimeout = integer() +%%% InitTimeout = integer() +%%% StartupTimeout = integer() +%%% StartupFunctions = [StartupFunctionSpec] +%%% StartupFunctionSpec = {Module, Function, Arguments} +%%% Module = atom() +%%% Function = atom() +%%% Arguments = [term] +%%% Monitor = bool() +%%% KillIfFail = bool() +%%% ErlangFlags = string() +%%% Result = {ok, NodeName} | {error, already_started, NodeName} | +%%% {error, started_not_connected, NodeName} | +%%% {error, boot_timeout, NodeName} | +%%% {error, init_timeout, NodeName} | +%%% {error, startup_timeout, NodeName} | +%%% {error, not_alive, NodeName} +%%% NodeName = atom() +%%% @doc Starts an Erlang node with name <code>Node</code> on host +%%% <code>Host</code> as specified by the combination of options in +%%% <code>Opts</code>. +%%% +%%% <p>Options <code>Username</code> and <code>Password</code> will be used +%%% to log in onto the remote host <code>Host</code>. +%%% Username, if omitted, defaults to the current user name, +%%% and password is empty by default.</p> +%%% +%%% <p>A list of functions specified in the <code>Startup</code> option will be +%%% executed after startup of the node. Note that all used modules should be +%%% present in the code path on the <code>Host</code>.</p> +%%% +%%% <p>The timeouts are applied as follows: +%%% <list> +%%% <item> +%%% <code>BootTimeout</code> - time to start the Erlang node, in seconds. +%%% Defaults to 3 seconds. If node does not become pingable within this time, +%%% the result <code>{error, boot_timeout, NodeName}</code> is returned; +%%% </item> +%%% <item> +%%% <code>InitTimeout</code> - time to wait for the node until it calls the +%%% internal callback function informing master about successfull startup. +%%% Defaults to one second. +%%% In case of timed out message the result +%%% <code>{error, init_timeout, NodeName}</code> is returned; +%%% </item> +%%% <item> +%%% <code>StartupTimeout</code> - time to wait intil the node finishes to run +%%% the <code>StartupFunctions</code>. Defaults to one second. +%%% If this timeout occurs, the result +%%% <code>{error, startup_timeout, NodeName}</code> is returned. +%%% </item> +%%% </list></p> +%%% +%%% <p>Option <code>monitor_master</code> specifies, if the slave node should be +%%% stopped in case of master node stop. Defaults to false.</p> +%%% +%%% <p>Option <code>kill_if_fail</code> specifies, if the slave node should be +%%% killed in case of a timeout during initialization or startup. +%%% Defaults to true. Note that node also may be still alive it the boot +%%% timeout occurred, but it will not be killed in this case.</p> +%%% +%%% <p>Option <code>erlang_flags</code> specifies, which flags will be added +%%% to the parameters of the <code>erl</code> executable.</p> +%%% +%%% <p>Special return values are: +%%% <list> +%%% <item><code>{error, already_started, NodeName}</code> - if the node with +%%% the given name is already started on a given host;</item> +%%% <item><code>{error, started_not_connected, NodeName}</code> - if node is +%%% started, but not connected to the master node.</item> +%%% <item><code>{error, not_alive, NodeName}</code> - if node on which the +%%% <code>ct_slave:start/3</code> is called, is not alive. Note that +%%% <code>NodeName</code> is the name of current node in this case.</item> +%%% </list></p> +%%% +start(Host, Node, Options)-> + ENode = enodename(Host, Node), + case erlang:is_alive() of + false-> + {error, not_alive, node()}; + true-> + case is_started(ENode) of + false-> + OptionsRec = fetch_options(Options), + do_start(Host, Node, OptionsRec); + {true, not_connected}-> + {error, started_not_connected, ENode}; + {true, connected}-> + {error, already_started, ENode} + end + end. + +%%% @spec stop(Node) -> Result +%%% Node = atom() +%%% Result = {ok, NodeName} | +%%% {error, not_started, NodeName} | +%%% {error, not_connected, NodeName} | +%%% {error, stop_timeout, NodeName} +%%% NodeName = atom() +%%% @doc Stops the running Erlang node with name <code>Node</code> on +%%% the localhost. +stop(Node)-> + stop(gethostname(), Node). + +%%% @spec stop(Host, Node) -> Result +%%% Host = atom() +%%% Node = atom() +%%% Result = {ok, NodeName} | +%%% {error, not_started, NodeName} | +%%% {error, not_connected, NodeName} | +%%% {error, stop_timeout, NodeName} +%%% NodeName = atom() +%%% @doc Stops the running Erlang node with name <code>Node</code> on +%%% host <code>Host</code>. +stop(Host, Node)-> + ENode = enodename(Host, Node), + case is_started(ENode) of + {true, connected}-> + do_stop(ENode); + {true, not_connected}-> + {error, not_connected, ENode}; + false-> + {error, not_started, ENode} + end. + +%%% fetch an option value from the tagged tuple list with default +get_option_value(Key, OptionList, Default)-> + case lists:keyfind(Key, 1, OptionList) of + false-> + Default; + {Key, Value}-> + Value + end. + +%%% convert option list to the option record, fill all defaults +fetch_options(Options)-> + UserName = get_option_value(username, Options, []), + Password = get_option_value(password, Options, []), + BootTimeout = get_option_value(boot_timeout, Options, 3), + InitTimeout = get_option_value(init_timeout, Options, 1), + StartupTimeout = get_option_value(startup_timeout, Options, 1), + StartupFunctions = get_option_value(startup_functions, Options, []), + Monitor = get_option_value(monitor_master, Options, false), + KillIfFail = get_option_value(kill_if_fail, Options, true), + ErlFlags = get_option_value(erl_flags, Options, []), + #options{username=UserName, password=Password, + boot_timeout=BootTimeout, init_timeout=InitTimeout, + startup_timeout=StartupTimeout, startup_functions=StartupFunctions, + monitor_master=Monitor, kill_if_fail=KillIfFail, erl_flags=ErlFlags}. + +% send a message when slave node is started +% @hidden +slave_started(ENode, MasterPid)-> + MasterPid ! {node_started, ENode}, + ok. + +% send a message when slave node has finished startup +% @hidden +slave_ready(ENode, MasterPid)-> + MasterPid ! {node_ready, ENode}, + ok. + +% start monitoring of the master node +% @hidden +monitor_master(MasterNode)-> + spawn(fun()->monitor_master_int(MasterNode) end). + +% code of the masterdeath-waiter process +monitor_master_int(MasterNode)-> + erlang:monitor_node(MasterNode, true), + receive + {nodedown, MasterNode}-> + init:stop() + end. + +% check if node is listed in the nodes() +is_connected(ENode)-> + [N||N<-nodes(), N==ENode] == [ENode]. + +% check if node is alive (ping and disconnect if pingable) +is_started(ENode)-> + case is_connected(ENode) of + true-> + {true, connected}; + false-> + case net_adm:ping(ENode) of + pang-> + false; + pong-> + erlang:disconnect_node(ENode), + {true, not_connected} + end + end. + +% make a Erlang node name from name and hostname +enodename(Host, Node)-> + list_to_atom(atom_to_list(Node)++"@"++atom_to_list(Host)). + +% performs actual start of the "slave" node +do_start(Host, Node, Options)-> + ENode = enodename(Host, Node), + Functions = + lists:concat([[{ct_slave, slave_started, [ENode, self()]}], + Options#options.startup_functions, + [{ct_slave, slave_ready, [ENode, self()]}]]), + Functions2 = if + Options#options.monitor_master-> + [{ct_slave, monitor_master, [node()]}|Functions]; + true-> + Functions + end, + MasterHost = gethostname(), + if + MasterHost == Host -> + spawn_local_node(Node, Options); + true-> + spawn_remote_node(Host, Node, Options) + end, + BootTimeout = Options#options.boot_timeout, + InitTimeout = Options#options.init_timeout, + StartupTimeout = Options#options.startup_timeout, + Result = case wait_for_node_alive(ENode, BootTimeout) of + pong-> + call_functions(ENode, Functions2), + receive + {node_started, ENode}-> + receive + {node_ready, ENode}-> + {ok, ENode} + after StartupTimeout*1000-> + {error, startup_timeout, ENode} + end + after InitTimeout*1000 -> + {error, init_timeout, ENode} + end; + pang-> + {error, boot_timeout, ENode} + end, + case Result of + {ok, ENode}-> + ok; + {error, Timeout, ENode} + when ((Timeout==init_timeout) or (Timeout==startup_timeout)) and + Options#options.kill_if_fail-> + do_stop(ENode); + _-> ok + end, + Result. + +% are we using fully qualified hostnames +long_or_short()-> + case net_kernel:longnames() of + true-> + " -name "; + false-> + " -sname " + end. + +% get the localhost's name, depending on the using name policy +gethostname()-> + Hostname = case net_kernel:longnames() of + true-> + net_adm:localhost(); + _-> + {ok, Name}=inet:gethostname(), + Name + end, + list_to_atom(Hostname). + +% get cmd for starting Erlang +get_cmd(Node, Flags)-> + Cookie = erlang:get_cookie(), + "erl -detached -noinput -setcookie "++ atom_to_list(Cookie) ++ + long_or_short() ++ atom_to_list(Node) ++ " " ++ Flags. + +% spawn node locally +spawn_local_node(Node, Options)-> + ErlFlags = Options#options.erl_flags, + Cmd = get_cmd(Node, ErlFlags), + open_port({spawn, Cmd}, [stream]). + +% start crypto and ssh if not yet started +check_for_ssh_running()-> + case application:get_application(crypto) of + undefined-> + application:start(crypto), + case application:get_application(ssh) of + undefined-> + application:start(ssh); + {ok, ssh}-> + ok + end; + {ok, crypto}-> + ok + end. + +% spawn node remotely +spawn_remote_node(Host, Node, Options)-> + Username = Options#options.username, + Password = Options#options.password, + ErlFlags = Options#options.erl_flags, + SSHOptions = case {Username, Password} of + {[], []}-> + []; + {_, []}-> + [{user, Username}]; + {_, _}-> + [{user, Username}, {password, Password}] + end ++ [{silently_accept_hosts, true}], + check_for_ssh_running(), + {ok, SSHConnRef} = ssh:connect(atom_to_list(Host), 22, SSHOptions), + {ok, SSHChannelId} = ssh_connection:session_channel(SSHConnRef, infinity), + ssh_connection:exec(SSHConnRef, SSHChannelId, get_cmd(Node, ErlFlags), infinity). + +% call functions on a remote Erlang node +call_functions(_Node, [])-> + ok; +call_functions(Node, [{M, F, A}|Functions])-> + rpc:call(Node, M, F, A), + call_functions(Node, Functions). + +% wait N seconds until node is pingable +wait_for_node_alive(_Node, 0)-> + pang; +wait_for_node_alive(Node, N)-> + timer:sleep(1000), + case net_adm:ping(Node) of + pong-> + pong; + pang-> + wait_for_node_alive(Node, N-1) + end. + +% call init:stop on a remote node +do_stop(ENode)-> + spawn(ENode, init, stop, []), + wait_for_node_dead(ENode, 5). + +% wait N seconds until node is disconnected +wait_for_node_dead(Node, 0)-> + {error, stop_timeout, Node}; +wait_for_node_dead(Node, N)-> + timer:sleep(1000), + case lists:member(Node, nodes()) of + true-> + wait_for_node_dead(Node, N-1); + false-> + {ok, Node} + end. diff --git a/lib/common_test/src/ct_snmp.erl b/lib/common_test/src/ct_snmp.erl index 7ff88ad7d3..8fe63e8ed1 100644 --- a/lib/common_test/src/ct_snmp.erl +++ b/lib/common_test/src/ct_snmp.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2004-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2004-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -332,7 +332,7 @@ set_info(Config) -> register_users(MgrAgentConfName, Users) -> {snmp, SnmpVals} = ct:get_config(MgrAgentConfName), NewSnmpVals = lists:keyreplace(users, 1, SnmpVals, {users, Users}), - ct_util:update_config(MgrAgentConfName, {snmp, NewSnmpVals}), + ct_config:update_config(MgrAgentConfName, {snmp, NewSnmpVals}), setup_users(Users). %%% @spec register_agents(MgrAgentConfName, ManagedAgents) -> ok | {error, Reason} @@ -347,7 +347,7 @@ register_agents(MgrAgentConfName, ManagedAgents) -> {snmp, SnmpVals} = ct:get_config(MgrAgentConfName), NewSnmpVals = lists:keyreplace(managed_agents, 1, SnmpVals, {managed_agents, ManagedAgents}), - ct_util:update_config(MgrAgentConfName, {snmp, NewSnmpVals}), + ct_config:update_config(MgrAgentConfName, {snmp, NewSnmpVals}), setup_managed_agents(ManagedAgents). %%% @spec register_usm_users(MgrAgentConfName, UsmUsers) -> ok | {error, Reason} @@ -361,7 +361,7 @@ register_agents(MgrAgentConfName, ManagedAgents) -> register_usm_users(MgrAgentConfName, UsmUsers) -> {snmp, SnmpVals} = ct:get_config(MgrAgentConfName), NewSnmpVals = lists:keyreplace(users, 1, SnmpVals, {usm_users, UsmUsers}), - ct_util:update_config(MgrAgentConfName, {snmp, NewSnmpVals}), + ct_config:update_config(MgrAgentConfName, {snmp, NewSnmpVals}), EngineID = ct:get_config({MgrAgentConfName, engine_id}, ?ENGINE_ID), setup_usm_users(UsmUsers, EngineID). @@ -376,7 +376,7 @@ unregister_users(MgrAgentConfName) -> ct:get_config({MgrAgentConfName, users})), {snmp, SnmpVals} = ct:get_config(MgrAgentConfName), NewSnmpVals = lists:keyreplace(users, 1, SnmpVals, {users, []}), - ct_util:update_config(MgrAgentConfName, {snmp, NewSnmpVals}), + ct_config:update_config(MgrAgentConfName, {snmp, NewSnmpVals}), takedown_users(Users). %%% @spec unregister_agents(MgrAgentConfName) -> ok | {error, Reason} @@ -393,7 +393,7 @@ unregister_agents(MgrAgentConfName) -> {snmp, SnmpVals} = ct:get_config(MgrAgentConfName), NewSnmpVals = lists:keyreplace(managed_agents, 1, SnmpVals, {managed_agents, []}), - ct_util:update_config(MgrAgentConfName, {snmp, NewSnmpVals}), + ct_config:update_config(MgrAgentConfName, {snmp, NewSnmpVals}), takedown_managed_agents(ManagedAgents). @@ -409,7 +409,7 @@ update_usm_users(MgrAgentConfName, UsmUsers) -> {snmp, SnmpVals} = ct:get_config(MgrAgentConfName), NewSnmpVals = lists:keyreplace(usm_users, 1, SnmpVals, {usm_users, UsmUsers}), - ct_util:update_config(MgrAgentConfName, {snmp, NewSnmpVals}), + ct_config:update_config(MgrAgentConfName, {snmp, NewSnmpVals}), EngineID = ct:get_config({MgrAgentConfName, engine_id}, ?ENGINE_ID), do_update_usm_users(UsmUsers, EngineID). diff --git a/lib/common_test/src/ct_telnet_client.erl b/lib/common_test/src/ct_telnet_client.erl index 1a12c5e343..d703b39ac5 100644 --- a/lib/common_test/src/ct_telnet_client.erl +++ b/lib/common_test/src/ct_telnet_client.erl @@ -35,8 +35,6 @@ -export([open/1, open/2, open/3, open/4, close/1]). -export([send_data/2, get_data/1]). --define(DBG, false). - -define(TELNET_PORT, 23). -define(OPEN_TIMEOUT,10000). -define(IDLE_TIMEOUT,10000). @@ -287,35 +285,38 @@ get_subcmd([?SE | Rest], Acc) -> get_subcmd([Opt | Rest], Acc) -> get_subcmd(Rest, [Opt | Acc]). - +-ifdef(debug). dbg(_Str,_Args) -> - if ?DBG -> io:format(_Str,_Args); - true -> ok + io:format(_Str,_Args). + +cmd_dbg(_Cmd) -> + case _Cmd of + [?IAC|Cmd1] -> + cmd_dbg(Cmd1); + [Ctrl|Opts] -> + CtrlStr = + case Ctrl of + ?DO -> "DO"; + ?DONT -> "DONT"; + ?WILL -> "WILL"; + ?WONT -> "WONT"; + ?NOP -> "NOP"; + _ -> "CMD" + end, + Opts1 = + case Opts of + [Opt] -> Opt; + _ -> Opts + end, + io:format("~s(~w): ~w\n", [CtrlStr,Ctrl,Opts1]); + Any -> + io:format("Unexpected in cmd_dbg:~n~w~n",[Any]) end. +-else. +dbg(_Str,_Args) -> + ok. + cmd_dbg(_Cmd) -> - if ?DBG -> - case _Cmd of - [?IAC|Cmd1] -> - cmd_dbg(Cmd1); - [Ctrl|Opts] -> - CtrlStr = - case Ctrl of - ?DO -> "DO"; - ?DONT -> "DONT"; - ?WILL -> "WILL"; - ?WONT -> "WONT"; - ?NOP -> "NOP"; - _ -> "CMD" - end, - Opts1 = - case Opts of - [Opt] -> Opt; - _ -> Opts - end, - io:format("~s(~w): ~w\n", [CtrlStr,Ctrl,Opts1]); - Any -> - io:format("Unexpected in cmd_dbg:~n~w~n",[Any]) - end; - true -> ok - end. + ok. +-endif. diff --git a/lib/common_test/src/ct_testspec.erl b/lib/common_test/src/ct_testspec.erl index 4378ec5a52..f5069427a2 100644 --- a/lib/common_test/src/ct_testspec.erl +++ b/lib/common_test/src/ct_testspec.erl @@ -17,7 +17,7 @@ %% %CopyrightEnd% %% -%%% @doc Common Test Framework functions handlig test specifikations. +%%% @doc Common Test Framework functions handling test specifications. %%% %%% <p>This module exports functions that are used within CT to %%% scan and parse test specifikations.</p> @@ -185,7 +185,15 @@ prepare_cases(Node,Dir,Suite,Cases) -> {[{{Node,Dir},{Suite,all}}],SkipAll}; Skipped -> %% note: this adds a test even if only skip is specified - PrepC = lists:foldr(fun({C,{skip,_Cmt}},Acc) -> + PrepC = lists:foldr(fun({{G,Cs},{skip,_Cmt}}, Acc) when + is_atom(G) -> + case lists:keymember(G, 1, Cases) of + true -> + Acc; + false -> + [{skipped,G,Cs}|Acc] + end; + ({C,{skip,_Cmt}},Acc) -> case lists:member(C,Cases) of true -> Acc; @@ -194,7 +202,7 @@ prepare_cases(Node,Dir,Suite,Cases) -> end; (C,Acc) -> [C|Acc] end, [], Cases), - {{{Node,Dir},{Suite,PrepC}},Skipped} + {{{Node,Dir},{Suite,PrepC}},Skipped} end. get_skipped_suites(Node,Dir,Suites) -> @@ -210,7 +218,7 @@ get_skipped_cases(Node,Dir,Suite,Cases) -> case lists:keysearch(all,1,Cases) of {value,{all,{skip,Cmt}}} -> [{{Node,Dir},{Suite,Cmt}}]; - false -> + _ -> get_skipped_cases1(Node,Dir,Suite,Cases) end. @@ -270,33 +278,8 @@ collect_tests(Terms,TestSpec,Relaxed) -> put(relaxed,Relaxed), TestSpec1 = get_global(Terms,TestSpec), TestSpec2 = get_all_nodes(Terms,TestSpec1), - case catch evaluate(Terms,TestSpec2) of - {error,{Node,{M,F,A},Reason}} -> - io:format("Error! Common Test failed to evaluate ~w:~w/~w on ~w. " - "Reason: ~p~n~n", [M,F,A,Node,Reason]); - _ -> ok - end, - add_tests(Terms,TestSpec2). - -evaluate([{eval,NodeRef,{M,F,Args}}|Ts],Spec) -> - Node = ref2node(NodeRef,Spec#testspec.nodes), - case rpc:call(Node,M,F,Args) of - {badrpc,Reason} -> - throw({error,{Node,{M,F,length(Args)},Reason}}); - _ -> - ok - end, - evaluate(Ts,Spec); -evaluate([{eval,{M,F,Args}}|Ts],Spec) -> - case catch apply(M,F,Args) of - {'EXIT',Reason} -> - throw({error,{node(),{M,F,length(Args)},Reason}}); - _ -> - ok - end, - evaluate(Ts,Spec); -evaluate([],_Spec) -> - ok. + {Terms2, TestSpec3} = filter_init_terms(Terms, [], TestSpec2), + add_tests(Terms2,TestSpec3). get_global([{alias,Ref,Dir}|Ts],Spec=#testspec{alias=Refs}) -> get_global(Ts,Spec#testspec{alias=[{Ref,get_absdir(Dir,Spec)}|Refs]}); @@ -305,6 +288,26 @@ get_global([{node,Ref,Node}|Ts],Spec=#testspec{nodes=Refs}) -> get_global([_|Ts],Spec) -> get_global(Ts,Spec); get_global([],Spec) -> Spec. +get_absfile(Callback, FullName,#testspec{spec_dir=SpecDir}) -> + % we need to temporary switch to new cwd here, because + % otherwise config files cannot be found + {ok, OldWd} = file:get_cwd(), + ok = file:set_cwd(SpecDir), + R = Callback:check_parameter(FullName), + ok = file:set_cwd(OldWd), + case R of + {ok, {file, FullName}}-> + File = filename:basename(FullName), + Dir = get_absname(filename:dirname(FullName),SpecDir), + filename:join(Dir,File); + {ok, {config, FullName}}-> + FullName; + {error, {nofile, FullName}}-> + FullName; + {error, {wrong_config, FullName}}-> + FullName + end. + get_absfile(FullName,#testspec{spec_dir=SpecDir}) -> File = filename:basename(FullName), Dir = get_absname(filename:dirname(FullName),SpecDir), @@ -353,6 +356,68 @@ get_all_nodes([_|Ts],Spec) -> get_all_nodes([],Spec) -> Spec. +filter_init_terms([{init, InitOptions}|Ts], NewTerms, Spec)-> + filter_init_terms([{init, list_nodes(Spec), InitOptions}|Ts], NewTerms, Spec); +filter_init_terms([{init, NodeRef, InitOptions}|Ts], NewTerms, Spec) + when is_atom(NodeRef)-> + filter_init_terms([{init, [NodeRef], InitOptions}|Ts], NewTerms, Spec); +filter_init_terms([{init, NodeRefs, InitOption}|Ts], NewTerms, Spec) when is_tuple(InitOption) -> + filter_init_terms([{init, NodeRefs, [InitOption]}|Ts], NewTerms, Spec); +filter_init_terms([{init, [NodeRef|NodeRefs], InitOptions}|Ts], NewTerms, Spec=#testspec{init=InitData})-> + NodeStartOptions = case lists:keyfind(node_start, 1, InitOptions) of + {node_start, NSOptions}-> + case lists:keyfind(callback_module, 1, NSOptions) of + {callback_module, _Callback}-> + NSOptions; + false-> + [{callback_module, ct_slave}|NSOptions] + end; + false-> + [] + end, + EvalTerms = case lists:keyfind(eval, 1, InitOptions) of + {eval, MFA} when is_tuple(MFA)-> + [MFA]; + {eval, MFAs} when is_list(MFAs)-> + MFAs; + false-> + [] + end, + Node = ref2node(NodeRef,Spec#testspec.nodes), + InitData2 = add_option({node_start, NodeStartOptions}, Node, InitData, true), + InitData3 = add_option({eval, EvalTerms}, Node, InitData2, false), + filter_init_terms([{init, NodeRefs, InitOptions}|Ts], NewTerms, Spec#testspec{init=InitData3}); +filter_init_terms([{init, [], _}|Ts], NewTerms, Spec)-> + filter_init_terms(Ts, NewTerms, Spec); +filter_init_terms([Term|Ts], NewTerms, Spec)-> + filter_init_terms(Ts, [Term|NewTerms], Spec); +filter_init_terms([], NewTerms, Spec)-> + {lists:reverse(NewTerms), Spec}. + +add_option([], _, List, _)-> + List; +add_option({Key, Value}, Node, List, WarnIfExists) when is_list(Value)-> + OldOptions = case lists:keyfind(Node, 1, List) of + {Node, Options}-> + Options; + false-> + [] + end, + NewOption = case lists:keyfind(Key, 1, OldOptions) of + {Key, OldOption} when WarnIfExists, OldOption/=[]-> + io:format("There is an option ~w=~w already defined for node ~p, skipping new ~w~n", + [Key, OldOption, Node, Value]), + OldOption; + {Key, OldOption}-> + OldOption ++ Value; + false-> + Value + end, + lists:keystore(Node, 1, List, + {Node, lists:keystore(Key, 1, OldOptions, {Key, NewOption})}); +add_option({Key, Value}, Node, List, WarnIfExists)-> + add_option({Key, [Value]}, Node, List, WarnIfExists). + save_nodes(Nodes,Spec=#testspec{nodes=NodeRefs}) -> NodeRefs1 = lists:foldr(fun(all_nodes,NR) -> @@ -375,6 +440,15 @@ save_nodes(Nodes,Spec=#testspec{nodes=NodeRefs}) -> list_nodes(#testspec{nodes=NodeRefs}) -> lists:map(fun({_Ref,Node}) -> Node end, NodeRefs). + + +%% --------------------------------------------------------- +%% / \ +%% | When adding tests, remember to update valid_terms/0 also! | +%% \ / +%% --------------------------------------------------------- + + %% Associate a "global" logdir with all nodes %% except those with specific logdir, e.g: %% ["/tmp/logdir",{ct1@finwe,"/tmp/logdir2"}] @@ -400,6 +474,24 @@ add_tests([{logdir,Node,Dir}|Ts],Spec) -> add_tests([{logdir,Dir}|Ts],Spec) -> add_tests([{logdir,all_nodes,Dir}|Ts],Spec); +%% --- label --- +add_tests([{label,all_nodes,Lbl}|Ts],Spec) -> + Labels = Spec#testspec.label, + Tests = [{label,N,Lbl} || N <- list_nodes(Spec), + lists:keymember(ref2node(N,Spec#testspec.nodes), + 1,Labels) == false], + add_tests(Tests++Ts,Spec); +add_tests([{label,Nodes,Lbl}|Ts],Spec) when is_list(Nodes) -> + Ts1 = separate(Nodes,label,[Lbl],Ts,Spec#testspec.nodes), + add_tests(Ts1,Spec); +add_tests([{label,Node,Lbl}|Ts],Spec) -> + Labels = Spec#testspec.label, + Labels1 = [{ref2node(Node,Spec#testspec.nodes),Lbl} | + lists:keydelete(ref2node(Node,Spec#testspec.nodes),1,Labels)], + add_tests(Ts,Spec#testspec{label=Labels1}); +add_tests([{label,Lbl}|Ts],Spec) -> + add_tests([{label,all_nodes,Lbl}|Ts],Spec); + %% --- cover --- add_tests([{cover,all_nodes,File}|Ts],Spec) -> Tests = lists:map(fun(N) -> {cover,N,File} end, list_nodes(Spec)), @@ -415,6 +507,36 @@ add_tests([{cover,Node,File}|Ts],Spec) -> add_tests([{cover,File}|Ts],Spec) -> add_tests([{cover,all_nodes,File}|Ts],Spec); +%% --- multiply_timetraps --- +add_tests([{multiply_timetraps,all_nodes,MT}|Ts],Spec) -> + Tests = lists:map(fun(N) -> {multiply_timetraps,N,MT} end, list_nodes(Spec)), + add_tests(Tests++Ts,Spec); +add_tests([{multiply_timetraps,Nodes,MT}|Ts],Spec) when is_list(Nodes) -> + Ts1 = separate(Nodes,multiply_timetraps,[MT],Ts,Spec#testspec.nodes), + add_tests(Ts1,Spec); +add_tests([{multiply_timetraps,Node,MT}|Ts],Spec) -> + MTs = Spec#testspec.multiply_timetraps, + MTs1 = [{ref2node(Node,Spec#testspec.nodes),MT} | + lists:keydelete(ref2node(Node,Spec#testspec.nodes),1,MTs)], + add_tests(Ts,Spec#testspec{multiply_timetraps=MTs1}); +add_tests([{multiply_timetraps,MT}|Ts],Spec) -> + add_tests([{multiply_timetraps,all_nodes,MT}|Ts],Spec); + +%% --- scale_timetraps --- +add_tests([{scale_timetraps,all_nodes,ST}|Ts],Spec) -> + Tests = lists:map(fun(N) -> {scale_timetraps,N,ST} end, list_nodes(Spec)), + add_tests(Tests++Ts,Spec); +add_tests([{scale_timetraps,Nodes,ST}|Ts],Spec) when is_list(Nodes) -> + Ts1 = separate(Nodes,scale_timetraps,[ST],Ts,Spec#testspec.nodes), + add_tests(Ts1,Spec); +add_tests([{scale_timetraps,Node,ST}|Ts],Spec) -> + STs = Spec#testspec.scale_timetraps, + STs1 = [{ref2node(Node,Spec#testspec.nodes),ST} | + lists:keydelete(ref2node(Node,Spec#testspec.nodes),1,STs)], + add_tests(Ts,Spec#testspec{scale_timetraps=STs1}); +add_tests([{scale_timetraps,ST}|Ts],Spec) -> + add_tests([{scale_timetraps,all_nodes,ST}|Ts],Spec); + %% --- config --- add_tests([{config,all_nodes,Files}|Ts],Spec) -> Tests = lists:map(fun(N) -> {config,N,Files} end, list_nodes(Spec)), @@ -434,6 +556,27 @@ add_tests([{config,Node,F}|Ts],Spec) -> add_tests([{config,Files}|Ts],Spec) -> add_tests([{config,all_nodes,Files}|Ts],Spec); + +%% --- userconfig --- +add_tests([{userconfig,all_nodes,CBF}|Ts],Spec) -> + Tests = lists:map(fun(N) -> {userconfig,N,CBF} end, list_nodes(Spec)), + add_tests(Tests++Ts,Spec); +add_tests([{userconfig,Nodes,CBF}|Ts],Spec) when is_list(Nodes) -> + Ts1 = separate(Nodes,userconfig,[CBF],Ts,Spec#testspec.nodes), + add_tests(Ts1,Spec); +add_tests([{userconfig,Node,[{Callback, Config}|CBF]}|Ts],Spec) -> + Cfgs = Spec#testspec.userconfig, + Node1 = ref2node(Node,Spec#testspec.nodes), + add_tests([{userconfig,Node,CBF}|Ts], + Spec#testspec{userconfig=[{Node1,{Callback, + get_absfile(Callback, Config ,Spec)}}|Cfgs]}); +add_tests([{userconfig,_Node,[]}|Ts],Spec) -> + add_tests(Ts,Spec); +add_tests([{userconfig,Node,CBF}|Ts],Spec) -> + add_tests([{userconfig,Node,[CBF]}|Ts],Spec); +add_tests([{userconfig,CBF}|Ts],Spec) -> + add_tests([{userconfig,all_nodes,CBF}|Ts],Spec); + %% --- event_handler --- add_tests([{event_handler,all_nodes,Hs}|Ts],Spec) -> Tests = lists:map(fun(N) -> {event_handler,N,Hs,[]} end, list_nodes(Spec)), @@ -516,6 +659,38 @@ add_tests([{suites,Node,Dir,Ss}|Ts],Spec) -> Ss,Tests), add_tests(Ts,Spec#testspec{tests=Tests1}); +%% --- groups --- +%% Later make it possible to specify group execution properties +%% that will override thse in the suite. Also make it possible +%% create dynamic groups in specification, i.e. to group test cases +%% by means of groups defined only in the test specification. +add_tests([{groups,all_nodes,Dir,Suite,Gs}|Ts],Spec) -> + add_tests([{groups,list_nodes(Spec),Dir,Suite,Gs}|Ts],Spec); +add_tests([{groups,all_nodes,Dir,Suite,Gs,{cases,TCs}}|Ts],Spec) -> + add_tests([{groups,list_nodes(Spec),Dir,Suite,Gs,{cases,TCs}}|Ts],Spec); +add_tests([{groups,Dir,Suite,Gs}|Ts],Spec) -> + add_tests([{groups,all_nodes,Dir,Suite,Gs}|Ts],Spec); +add_tests([{groups,Dir,Suite,Gs,{cases,TCs}}|Ts],Spec) -> + add_tests([{groups,all_nodes,Dir,Suite,Gs,{cases,TCs}}|Ts],Spec); +add_tests([{groups,Nodes,Dir,Suite,Gs}|Ts],Spec) when is_list(Nodes) -> + Ts1 = separate(Nodes,groups,[Dir,Suite,Gs],Ts,Spec#testspec.nodes), + add_tests(Ts1,Spec); +add_tests([{groups,Nodes,Dir,Suite,Gs,{cases,TCs}}|Ts],Spec) when is_list(Nodes) -> + Ts1 = separate(Nodes,groups,[Dir,Suite,Gs,{cases,TCs}],Ts,Spec#testspec.nodes), + add_tests(Ts1,Spec); +add_tests([{groups,Node,Dir,Suite,Gs}|Ts],Spec) -> + Tests = Spec#testspec.tests, + Tests1 = insert_groups(ref2node(Node,Spec#testspec.nodes), + ref2dir(Dir,Spec#testspec.alias), + Suite,Gs,all,Tests), + add_tests(Ts,Spec#testspec{tests=Tests1}); +add_tests([{groups,Node,Dir,Suite,Gs,{cases,TCs}}|Ts],Spec) -> + Tests = Spec#testspec.tests, + Tests1 = insert_groups(ref2node(Node,Spec#testspec.nodes), + ref2dir(Dir,Spec#testspec.alias), + Suite,Gs,TCs,Tests), + add_tests(Ts,Spec#testspec{tests=Tests1}); + %% --- cases --- add_tests([{cases,all_nodes,Dir,Suite,Cs}|Ts],Spec) -> add_tests([{cases,list_nodes(Spec),Dir,Suite,Cs}|Ts],Spec); @@ -546,6 +721,34 @@ add_tests([{skip_suites,Node,Dir,Ss,Cmt}|Ts],Spec) -> Ss,Cmt,Tests), add_tests(Ts,Spec#testspec{tests=Tests1}); +%% --- skip_groups --- +add_tests([{skip_groups,all_nodes,Dir,Suite,Gs,Cmt}|Ts],Spec) -> + add_tests([{skip_groups,list_nodes(Spec),Dir,Suite,Gs,Cmt}|Ts],Spec); +add_tests([{skip_groups,all_nodes,Dir,Suite,Gs,{cases,TCs},Cmt}|Ts],Spec) -> + add_tests([{skip_groups,list_nodes(Spec),Dir,Suite,Gs,{cases,TCs},Cmt}|Ts],Spec); +add_tests([{skip_groups,Dir,Suite,Gs,Cmt}|Ts],Spec) -> + add_tests([{skip_groups,all_nodes,Dir,Suite,Gs,Cmt}|Ts],Spec); +add_tests([{skip_groups,Dir,Suite,Gs,{cases,TCs},Cmt}|Ts],Spec) -> + add_tests([{skip_groups,all_nodes,Dir,Suite,Gs,{cases,TCs},Cmt}|Ts],Spec); +add_tests([{skip_groups,Nodes,Dir,Suite,Gs,Cmt}|Ts],Spec) when is_list(Nodes) -> + Ts1 = separate(Nodes,skip_groups,[Dir,Suite,Gs,Cmt],Ts,Spec#testspec.nodes), + add_tests(Ts1,Spec); +add_tests([{skip_groups,Nodes,Dir,Suite,Gs,{cases,TCs},Cmt}|Ts],Spec) when is_list(Nodes) -> + Ts1 = separate(Nodes,skip_groups,[Dir,Suite,Gs,{cases,TCs},Cmt],Ts,Spec#testspec.nodes), + add_tests(Ts1,Spec); +add_tests([{skip_groups,Node,Dir,Suite,Gs,Cmt}|Ts],Spec) -> + Tests = Spec#testspec.tests, + Tests1 = skip_groups(ref2node(Node,Spec#testspec.nodes), + ref2dir(Dir,Spec#testspec.alias), + Suite,Gs,all,Cmt,Tests), + add_tests(Ts,Spec#testspec{tests=Tests1}); +add_tests([{skip_groups,Node,Dir,Suite,Gs,{cases,TCs},Cmt}|Ts],Spec) -> + Tests = Spec#testspec.tests, + Tests1 = skip_groups(ref2node(Node,Spec#testspec.nodes), + ref2dir(Dir,Spec#testspec.alias), + Suite,Gs,TCs,Cmt,Tests), + add_tests(Ts,Spec#testspec{tests=Tests1}); + %% --- skip_cases --- add_tests([{skip_cases,all_nodes,Dir,Suite,Cs,Cmt}|Ts],Spec) -> add_tests([{skip_cases,list_nodes(Spec),Dir,Suite,Cs,Cmt}|Ts],Spec); @@ -614,8 +817,11 @@ separate([],_,_,_) -> %% Representation: -%% {{Node,Dir},[{Suite1,[case11,case12,...]},{Suite2,[case21,case22,...]},...]} -%% {{Node,Dir},[{Suite1,{skip,Cmt}},{Suite2,[{case21,{skip,Cmt}},case22,...]},...]} +%% {{Node,Dir},[{Suite1,[GrOrCase11,GrOrCase12,...]}, +%% {Suite2,[GrOrCase21,GrOrCase22,...]},...]} +%% {{Node,Dir},[{Suite1,{skip,Cmt}}, +%% {Suite2,[{GrOrCase21,{skip,Cmt}},GrOrCase22,...]},...]} +%% GrOrCase = {GroupName,[Case1,Case2,...]} | Case insert_suites(Node,Dir,[S|Ss],Tests) -> Tests1 = insert_cases(Node,Dir,S,all,Tests), @@ -625,6 +831,54 @@ insert_suites(_Node,_Dir,[],Tests) -> insert_suites(Node,Dir,S,Tests) -> insert_suites(Node,Dir,[S],Tests). +insert_groups(Node,Dir,Suite,Group,Cases,Tests) when is_atom(Group) -> + insert_groups(Node,Dir,Suite,[Group],Cases,Tests); +insert_groups(Node,Dir,Suite,Groups,Cases,Tests) when + ((Cases == all) or is_list(Cases)) and is_list(Groups) -> + case lists:keysearch({Node,Dir},1,Tests) of + {value,{{Node,Dir},[{all,_}]}} -> + Tests; + {value,{{Node,Dir},Suites0}} -> + Suites1 = insert_groups1(Suite, + [{Gr,Cases} || Gr <- Groups], + Suites0), + insert_in_order({{Node,Dir},Suites1},Tests); + false -> + Groups1 = [{Gr,Cases} || Gr <- Groups], + insert_in_order({{Node,Dir},[{Suite,Groups1}]},Tests) + end; +insert_groups(Node,Dir,Suite,Groups,Case,Tests) when is_atom(Case) -> + Cases = if Case == all -> all; true -> [Case] end, + insert_groups(Node,Dir,Suite,Groups,Cases,Tests). + +insert_groups1(_Suite,_Groups,all) -> + all; +insert_groups1(Suite,Groups,Suites0) -> + case lists:keysearch(Suite,1,Suites0) of + {value,{Suite,all}} -> + Suites0; + {value,{Suite,GrAndCases0}} -> + GrAndCases = insert_groups2(Groups,GrAndCases0), + insert_in_order({Suite,GrAndCases},Suites0); + false -> + insert_in_order({Suite,Groups},Suites0) + end. + +insert_groups2(_Groups,all) -> + all; +insert_groups2([Group={GrName,Cases}|Groups],GrAndCases) -> + case lists:keysearch(GrName,1,GrAndCases) of + {value,{GrName,all}} -> + GrAndCases; + {value,{GrName,Cases0}} -> + Cases1 = insert_in_order(Cases,Cases0), + insert_groups2(Groups,insert_in_order({GrName,Cases1},GrAndCases)); + false -> + insert_groups2(Groups,insert_in_order(Group,GrAndCases)) + end; +insert_groups2([],GrAndCases) -> + GrAndCases. + insert_cases(Node,Dir,Suite,Cases,Tests) when is_list(Cases) -> case lists:keysearch({Node,Dir},1,Tests) of {value,{{Node,Dir},[{all,_}]}} -> @@ -659,6 +913,40 @@ skip_suites(_Node,_Dir,[],_Cmt,Tests) -> skip_suites(Node,Dir,S,Cmt,Tests) -> skip_suites(Node,Dir,[S],Cmt,Tests). +skip_groups(Node,Dir,Suite,Group,all,Cmt,Tests) when is_atom(Group) -> + skip_groups(Node,Dir,Suite,[Group],all,Cmt,Tests); +skip_groups(Node,Dir,Suite,Group,Cases,Cmt,Tests) when is_atom(Group) -> + skip_groups(Node,Dir,Suite,[Group],Cases,Cmt,Tests); +skip_groups(Node,Dir,Suite,Groups,Case,Cmt,Tests) when is_atom(Case), + Case =/= all -> + skip_groups(Node,Dir,Suite,Groups,[Case],Cmt,Tests); +skip_groups(Node,Dir,Suite,Groups,Cases,Cmt,Tests) when + ((Cases == all) or is_list(Cases)) and is_list(Groups) -> + Suites = + case lists:keysearch({Node,Dir},1,Tests) of + {value,{{Node,Dir},Suites0}} -> + Suites0; + false -> + [] + end, + Suites1 = skip_groups1(Suite,[{Gr,Cases} || Gr <- Groups],Cmt,Suites), + insert_in_order({{Node,Dir},Suites1},Tests); +skip_groups(Node,Dir,Suite,Groups,Case,Cmt,Tests) when is_atom(Case) -> + Cases = if Case == all -> all; true -> [Case] end, + skip_groups(Node,Dir,Suite,Groups,Cases,Cmt,Tests). + +skip_groups1(Suite,Groups,Cmt,Suites0) -> + SkipGroups = lists:map(fun(Group) -> + {Group,{skip,Cmt}} + end,Groups), + case lists:keysearch(Suite,1,Suites0) of + {value,{Suite,GrAndCases0}} -> + GrAndCases1 = GrAndCases0 ++ SkipGroups, + insert_in_order({Suite,GrAndCases1},Suites0); + false -> + insert_in_order({Suite,SkipGroups},Suites0) + end. + skip_cases(Node,Dir,Suite,Cases,Cmt,Tests) when is_list(Cases) -> Suites = case lists:keysearch({Node,Dir},1,Tests) of @@ -753,21 +1041,34 @@ valid_terms() -> {cover,3}, {config,2}, {config,3}, + {userconfig,2}, + {userconfig,3}, {alias,3}, {logdir,2}, {logdir,3}, + {label,2}, + {label,3}, {event_handler,2}, {event_handler,3}, {event_handler,4}, + {multiply_timetraps,2}, + {multiply_timetraps,3}, + {scale_timetraps,2}, + {scale_timetraps,3}, {include,2}, {include,3}, - {suites,3}, {suites,4}, + {groups,4}, + {groups,5}, + {groups,6}, {cases,4}, {cases,5}, {skip_suites,4}, {skip_suites,5}, + {skip_groups,5}, + {skip_groups,6}, + {skip_groups,7}, {skip_cases,5}, {skip_cases,6} ]. @@ -816,7 +1117,3 @@ common_letters([L|Ls],Term,Count) -> end; common_letters([],_,Count) -> Count. - - - - diff --git a/lib/common_test/src/ct_util.erl b/lib/common_test/src/ct_util.erl index ba3d789f8d..0a434666fa 100644 --- a/lib/common_test/src/ct_util.erl +++ b/lib/common_test/src/ct_util.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2003-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -30,10 +30,7 @@ -export([register_connection/4,unregister_connection/1, does_connection_exist/3,get_key_from_name/1]). --export([require/1, require/2, get_config/1, get_config/2, get_config/3, - set_default_config/2, set_default_config/3, delete_default_config/1, - get_all_config/0, update_config/2, - release_allocated/0, close_connections/0]). +-export([close_connections/0]). -export([save_suite_data/3, save_suite_data/2, read_suite_data/1, delete_suite_data/0, delete_suite_data/1, match_delete_suite_data/1, @@ -46,6 +43,8 @@ silence_all_connections/0, silence_connections/1, is_silenced/1, reset_silent_connections/0]). +-export([get_mode/0, create_table/3, read_opts/0]). + -export([set_cwd/1, reset_cwd/0]). -export([parse_table/1]). @@ -56,23 +55,15 @@ -export([is_test_dir/1, get_testdir/2]). --export([encrypt_config_file/2, encrypt_config_file/3, - decrypt_config_file/2, decrypt_config_file/3]). - --export([kill_attached/2, get_attached/1]). +-export([kill_attached/2, get_attached/1, ct_make_ref/0]). -export([warn_duplicates/1]). -include("ct_event.hrl"). -include("ct_util.hrl"). --record(ct_conf,{key,value,ref,name='_UNDEF',default=false}). -%% default = {true,suite} | {true,testcase} | false - -record(suite_data, {key,name,value}). --define(cryptfile, ".ct_config.crypt"). - %%%----------------------------------------------------------------- %%% @spec start(Mode) -> Pid | exit(Error) %%% Mode = normal | interactive @@ -119,7 +110,6 @@ start(Mode,LogDir) -> do_start(Parent,Mode,LogDir) -> process_flag(trap_exit,true), register(ct_util_server,self()), - create_table(?attr_table,bag,#ct_conf.key), create_table(?conn_table,#conn.handle), create_table(?board_table,2), create_table(?suite_table,#suite_data.key), @@ -135,7 +125,6 @@ do_start(Parent,Mode,LogDir) -> Parent ! {self(),Error}, exit(Error) end, - %% start an event manager (if not already started by master) case ct_event:start_link() of {error,{already_started,_}} -> @@ -148,38 +137,34 @@ do_start(Parent,Mode,LogDir) -> ct_event:add_handler([{vts,VtsPid}]) end end, - case read_config_files(Opts) of - ok -> - %% add user handlers - case lists:keysearch(event_handler,1,Opts) of - {value,{_,Handlers}} -> - Add = fun({H,Args}) -> - case catch gen_event:add_handler(?CT_EVMGR_REF,H,Args) of - ok -> ok; - {'EXIT',Why} -> exit(Why); - Other -> exit({event_handler,Other}) - end - end, - case catch lists:foreach(Add,Handlers) of - {'EXIT',Reason} -> - Parent ! {self(),Reason}; - _ -> - ok - end; - false -> + %% start ct_config server + ct_config:start(Mode), + %% add user event handlers + case lists:keysearch(event_handler,1,Opts) of + {value,{_,Handlers}} -> + Add = fun({H,Args}) -> + case catch gen_event:add_handler(?CT_EVMGR_REF,H,Args) of + ok -> ok; + {'EXIT',Why} -> exit(Why); + Other -> exit({event_handler,Other}) + end + end, + case catch lists:foreach(Add,Handlers) of + {'EXIT',Reason} -> + Parent ! {self(),Reason}; + _ -> ok - end, - {StartTime,TestLogDir} = ct_logs:init(Mode), - ct_event:notify(#event{name=test_start, - node=node(), - data={StartTime, - lists:flatten(TestLogDir)}}), - Parent ! {self(),started}, - loop(Mode,[],StartDir); - ReadError -> - Parent ! {self(),ReadError}, - exit(ReadError) - end. + end; + false -> + ok + end, + {StartTime,TestLogDir} = ct_logs:init(Mode), + ct_event:notify(#event{name=test_start, + node=node(), + data={StartTime, + lists:flatten(TestLogDir)}}), + Parent ! {self(),started}, + loop(Mode,[],StartDir). create_table(TableName,KeyPos) -> create_table(TableName,set,KeyPos). @@ -197,106 +182,6 @@ read_opts() -> {error,{bad_installation,Error}} end. -read_config_files(Opts) -> - ConfigFiles = - lists:foldl(fun({config,Files},Acc) -> - Acc ++ Files; - (_,Acc) -> - Acc - end,[],Opts), - read_config_files1(ConfigFiles). - -read_config_files1([ConfigFile|Files]) -> - case file:consult(ConfigFile) of - {ok,Config} -> - set_config(Config), - read_config_files1(Files); - {error,enoent} -> - {user_error,{config_file_error,ConfigFile,enoent}}; - {error,Reason} -> - Key = - case application:get_env(common_test, decrypt) of - {ok,KeyOrFile} -> - case KeyOrFile of - {key,K} -> - K; - {file,F} -> - get_crypt_key_from_file(F) - end; - _ -> - get_crypt_key_from_file() - end, - case Key of - {error,no_crypt_file} -> - {user_error,{config_file_error,ConfigFile,Reason}}; - {error,CryptError} -> - {user_error,{decrypt_file_error,ConfigFile,CryptError}}; - _ when is_list(Key) -> - case decrypt_config_file(ConfigFile, undefined, {key,Key}) of - {ok,CfgBin} -> - case read_config_terms(CfgBin) of - {error,ReadFail} -> - {user_error,{config_file_error,ConfigFile,ReadFail}}; - Config -> - set_config(Config), - read_config_files1(Files) - end; - {error,DecryptFail} -> - {user_error,{decrypt_config_error,ConfigFile,DecryptFail}} - end; - _ -> - {user_error,{bad_decrypt_key,ConfigFile,Key}} - end - end; -read_config_files1([]) -> - ok. - -read_config_terms(Bin) when is_binary(Bin) -> - case catch binary_to_list(Bin) of - {'EXIT',_} -> - {error,invalid_textfile}; - Lines -> - read_config_terms(Lines) - end; -read_config_terms(Lines) when is_list(Lines) -> - read_config_terms1(erl_scan:tokens([], Lines, 0), 1, [], []). - -read_config_terms1({done,{ok,Ts,EL},Rest}, L, Terms, _) -> - case erl_parse:parse_term(Ts) of - {ok,Term} when Rest == [] -> - lists:reverse([Term|Terms]); - {ok,Term} -> - read_config_terms1(erl_scan:tokens([], Rest, 0), - EL+1, [Term|Terms], Rest); - _ -> - {error,{bad_term,{L,EL}}} - end; -read_config_terms1({done,{eof,_},_}, _, Terms, Rest) when Rest == [] -> - lists:reverse(Terms); -read_config_terms1({done,{eof,EL},_}, L, _, _) -> - {error,{bad_term,{L,EL}}}; -read_config_terms1({done,{error,Info,EL},_}, L, _, _) -> - {error,{Info,{L,EL}}}; -read_config_terms1({more,_}, L, Terms, Rest) -> - case string:tokens(Rest, [$\n,$\r,$\t]) of - [] -> - lists:reverse(Terms); - _ -> - {error,{bad_term,L}} - end. - -set_default_config(NewConfig, Scope) -> - call({set_default_config, {NewConfig, Scope}}). - -set_default_config(Name, NewConfig, Scope) -> - call({set_default_config, {Name, NewConfig, Scope}}). - -delete_default_config(Scope) -> - call({delete_default_config, Scope}). - -update_config(Name, Config) -> - call({update_config, {Name, Config}}). - save_suite_data(Key, Value) -> call({save_suite_data, {Key, undefined, Value}}). @@ -342,26 +227,6 @@ loop(Mode,TestData,StartDir) -> ct_logs:make_last_run_index(), return(From,ok), loop(Mode,TestData,StartDir); - {{require,Name,Tag,SubTags},From} -> - Result = do_require(Name,Tag,SubTags), - return(From,Result), - loop(Mode,TestData,StartDir); - {{set_default_config,{Config,Scope}},From} -> - set_config(Config,{true,Scope}), - return(From,ok), - loop(Mode,TestData,StartDir); - {{set_default_config,{Name,Config,Scope}},From} -> - set_config(Name,Config,{true,Scope}), - return(From,ok), - loop(Mode,TestData,StartDir); - {{delete_default_config,Scope},From} -> - delete_config({true,Scope}), - return(From,ok), - loop(Mode,TestData,StartDir); - {{update_config,{Name,NewConfig}},From} -> - update_conf(Name,NewConfig), - return(From,ok), - loop(Mode,TestData,StartDir); {{save_suite_data,{Key,Name,Value}},From} -> ets:insert(?suite_table, #suite_data{key=Key, name=Name, @@ -434,14 +299,14 @@ loop(Mode,TestData,StartDir) -> ct_event:sync_notify(#event{name=test_done, node=node(), data=Time}), - ets:delete(?attr_table), close_connections(ets:tab2list(?conn_table)), ets:delete(?conn_table), ets:delete(?board_table), ets:delete(?suite_table), ct_logs:close(How), - file:set_cwd(StartDir), ct_event:stop(), + ct_config:stop(), + file:set_cwd(StartDir), return(From,ok); {get_mode,From} -> return(From,Mode), @@ -463,6 +328,8 @@ close_connections([#conn{handle=Handle,callback=CB}|Conns]) -> close_connections([]) -> ok. +get_key_from_name(Name)-> + ct_config:get_key_from_name(Name). %%%----------------------------------------------------------------- %%% @spec register_connection(TargetName,Address,Callback,Handle) -> @@ -480,7 +347,7 @@ close_connections([]) -> %%% test is finished by calling <code>Callback:close/1</code>.</p> register_connection(TargetName,Address,Callback,Handle) -> TargetRef = - case get_ref_from_name(TargetName) of + case ct_config:get_ref_from_name(TargetName) of {ok,Ref} -> Ref; _ -> @@ -518,7 +385,7 @@ unregister_connection(Handle) -> %%% %%% @doc Check if a connection already exists. does_connection_exist(TargetName,Address,Callback) -> - case get_ref_from_name(TargetName) of + case ct_config:get_ref_from_name(TargetName) of {ok,TargetRef} -> case ets:select(?conn_table,[{#conn{handle='$1', targetref=TargetRef, @@ -548,7 +415,7 @@ does_connection_exist(TargetName,Address,Callback) -> %%% @doc Return all connections for the <code>Callback</code> on the %%% given target (<code>TargetName</code>). get_connections(TargetName,Callback) -> - case get_ref_from_name(TargetName) of + case ct_config:get_ref_from_name(TargetName) of {ok,Ref} -> {ok,ets:select(?conn_table,[{#conn{handle='$1', address='$2', @@ -568,250 +435,11 @@ get_target_name(ConnPid) -> [], ['$1']}]) of [TargetRef] -> - get_name_from_ref(TargetRef); + ct_config:get_name_from_ref(TargetRef); [] -> {error,{unknown_connection,ConnPid}} end. - -%%%----------------------------------------------------------------- -%%% @hidden -%%% @equiv ct:require/1 -require(Key) when is_atom(Key) -> - require({Key,[]}); -require({Key,SubKeys}) when is_atom(Key) -> - allocate('_UNDEF',Key,to_list(SubKeys)); -require(Key) -> - {error,{invalid,Key}}. - - -%%%----------------------------------------------------------------- -%%% @hidden -%%% @equiv ct:require/2 -require(Name,Key) when is_atom(Key) -> - require(Name,{Key,[]}); -require(Name,{Key,SubKeys}) when is_atom(Name), is_atom(Key) -> - call({require,Name,Key,to_list(SubKeys)}); -require(Name,Keys) -> - {error,{invalid,{Name,Keys}}}. - -to_list(X) when is_list(X) -> X; -to_list(X) -> [X]. - -do_require(Name,Key,SubKeys) when is_list(SubKeys) -> - case get_key_from_name(Name) of - {error,_} -> - allocate(Name,Key,SubKeys); - {ok,Key} -> - %% already allocated - check that it has all required subkeys - Vals = [Val || {_Ref,Val} <- lookup_name(Name)], - case get_subconfig(SubKeys,Vals) of - {ok,_SubMapped} -> - ok; - Error -> - Error - end; - {ok,OtherKey} -> - {error,{name_in_use,Name,OtherKey}} - end. - -allocate(Name,Key,SubKeys) -> - case ets:match_object(?attr_table,#ct_conf{key=Key,name='_UNDEF',_='_'}) of - [] -> - {error,{not_available,Key}}; - Available -> - case allocate_subconfig(Name,SubKeys,Available,false) of - ok -> - ok; - Error -> - Error - end - end. - -allocate_subconfig(Name,SubKeys,[C=#ct_conf{value=Value}|Rest],Found) -> - case do_get_config(SubKeys,Value,[]) of - {ok,_SubMapped} -> - ets:insert(?attr_table,C#ct_conf{name=Name}), - allocate_subconfig(Name,SubKeys,Rest,true); - _Error -> - allocate_subconfig(Name,SubKeys,Rest,Found) - end; -allocate_subconfig(_Name,_SubKeys,[],true) -> - ok; -allocate_subconfig(_Name,SubKeys,[],false) -> - {error,{not_available,SubKeys}}. - - - - -%%%----------------------------------------------------------------- -%%% @hidden -%%% @equiv ct:get_config/1 -get_config(KeyOrName) -> - get_config(KeyOrName,undefined,[]). - -%%%----------------------------------------------------------------- -%%% @hidden -%%% @equiv ct:get_config/2 -get_config(KeyOrName,Default) -> - get_config(KeyOrName,Default,[]). - -%%%----------------------------------------------------------------- -%%% @hidden -%%% @equiv ct:get_config/3 -get_config(KeyOrName,Default,Opts) when is_atom(KeyOrName) -> - case lookup_config(KeyOrName) of - [] -> - Default; - [{_Ref,Val}|_] = Vals -> - case {lists:member(all,Opts),lists:member(element,Opts)} of - {true,true} -> - [{KeyOrName,V} || {_R,V} <- lists:sort(Vals)]; - {true,false} -> - [V || {_R,V} <- lists:sort(Vals)]; - {false,true} -> - {KeyOrName,Val}; - {false,false} -> - Val - end - end; - -get_config({KeyOrName,SubKey},Default,Opts) -> - case lookup_config(KeyOrName) of - [] -> - Default; - Vals -> - Vals1 = case [Val || {_Ref,Val} <- lists:sort(Vals)] of - Result=[L|_] when is_list(L) -> - case L of - [{_,_}|_] -> - Result; - _ -> - [] - end; - _ -> - [] - end, - case get_subconfig([SubKey],Vals1,[],Opts) of - {ok,[{_,SubVal}|_]=SubVals} -> - case {lists:member(all,Opts),lists:member(element,Opts)} of - {true,true} -> - [{{KeyOrName,SubKey},Val} || {_,Val} <- SubVals]; - {true,false} -> - [Val || {_SubKey,Val} <- SubVals]; - {false,true} -> - {{KeyOrName,SubKey},SubVal}; - {false,false} -> - SubVal - end; - _ -> - Default - end - end. - - -get_subconfig(SubKeys,Values) -> - get_subconfig(SubKeys,Values,[],[]). - -get_subconfig(SubKeys,[Value|Rest],Mapped,Opts) -> - case do_get_config(SubKeys,Value,[]) of - {ok,SubMapped} -> - case lists:member(all,Opts) of - true -> - get_subconfig(SubKeys,Rest,Mapped++SubMapped,Opts); - false -> - {ok,SubMapped} - end; - _Error -> - get_subconfig(SubKeys,Rest,Mapped,Opts) - end; -get_subconfig(SubKeys,[],[],_) -> - {error,{not_available,SubKeys}}; -get_subconfig(_SubKeys,[],Mapped,_) -> - {ok,Mapped}. - -do_get_config([Key|Required],Available,Mapped) -> - case lists:keysearch(Key,1,Available) of - {value,{Key,Value}} -> - NewAvailable = lists:keydelete(Key,1,Available), - NewMapped = [{Key,Value}|Mapped], - do_get_config(Required,NewAvailable,NewMapped); - false -> - {error,{not_available,Key}} - end; -do_get_config([],_Available,Mapped) -> - {ok,lists:reverse(Mapped)}. - -get_all_config() -> - ets:select(?attr_table,[{#ct_conf{name='$1',key='$2',value='$3', - default='$4',_='_'}, - [], - [{{'$1','$2','$3','$4'}}]}]). - -lookup_config(KeyOrName) -> - case lookup_name(KeyOrName) of - [] -> - lookup_key(KeyOrName); - Values -> - Values - end. - -lookup_name(Name) -> - ets:select(?attr_table,[{#ct_conf{ref='$1',value='$2',name=Name,_='_'}, - [], - [{{'$1','$2'}}]}]). -lookup_key(Key) -> - ets:select(?attr_table,[{#ct_conf{key=Key,ref='$1',value='$2',name='_UNDEF',_='_'}, - [], - [{{'$1','$2'}}]}]). - -set_config(Config) -> - set_config('_UNDEF',Config,false). - -set_config(Config,Default) -> - set_config('_UNDEF',Config,Default). - -set_config(Name,Config,Default) -> - [ets:insert(?attr_table, - #ct_conf{key=Key,value=Val,ref=ct_make_ref(), - name=Name,default=Default}) || - {Key,Val} <- Config]. - -delete_config(Default) -> - ets:match_delete(?attr_table,#ct_conf{default=Default,_='_'}), - ok. - - -%%%----------------------------------------------------------------- -%%% @spec release_allocated() -> ok -%%% -%%% @doc Release all allocated resources, but don't take down any -%%% connections. -release_allocated() -> - Allocated = ets:select(?attr_table,[{#ct_conf{name='$1',_='_'}, - [{'=/=','$1','_UNDEF'}], - ['$_']}]), - release_allocated(Allocated). -release_allocated([H|T]) -> - ets:delete_object(?attr_table,H), - ets:insert(?attr_table,H#ct_conf{name='_UNDEF'}), - release_allocated(T); -release_allocated([]) -> - ok. - -%%%----------------------------------------------------------------- -%%% @spec -%%% -%%% @doc -update_conf(Name, NewConfig) -> - Old = ets:select(?attr_table,[{#ct_conf{name=Name,_='_'},[],['$_']}]), - lists:foreach(fun(OldElem) -> - NewElem = OldElem#ct_conf{value=NewConfig}, - ets:delete_object(?attr_table, OldElem), - ets:insert(?attr_table, NewElem) - end, Old), - ok. - %%%----------------------------------------------------------------- %%% @spec close_connections() -> ok %%% @@ -995,167 +623,6 @@ get_testdir(Dir, _) -> %%% @spec %%% %%% @doc -encrypt_config_file(SrcFileName, EncryptFileName) -> - case get_crypt_key_from_file() of - {error,_} = E -> - E; - Key -> - encrypt_config_file(SrcFileName, EncryptFileName, {key,Key}) - end. - -%%%----------------------------------------------------------------- -%%% @spec -%%% -%%% @doc -encrypt_config_file(SrcFileName, EncryptFileName, {file,KeyFile}) -> - case get_crypt_key_from_file(KeyFile) of - {error,_} = E -> - E; - Key -> - encrypt_config_file(SrcFileName, EncryptFileName, {key,Key}) - end; - -encrypt_config_file(SrcFileName, EncryptFileName, {key,Key}) -> - crypto:start(), - {K1,K2,K3,IVec} = make_crypto_key(Key), - case file:read_file(SrcFileName) of - {ok,Bin0} -> - Bin1 = term_to_binary({SrcFileName,Bin0}), - Bin2 = case byte_size(Bin1) rem 8 of - 0 -> Bin1; - N -> list_to_binary([Bin1,random_bytes(8-N)]) - end, - EncBin = crypto:des3_cbc_encrypt(K1, K2, K3, IVec, Bin2), - case file:write_file(EncryptFileName, EncBin) of - ok -> - io:format("~s --(encrypt)--> ~s~n", - [SrcFileName,EncryptFileName]), - ok; - {error,Reason} -> - {error,{Reason,EncryptFileName}} - end; - {error,Reason} -> - {error,{Reason,SrcFileName}} - end. - -%%%----------------------------------------------------------------- -%%% @spec -%%% -%%% @doc -decrypt_config_file(EncryptFileName, TargetFileName) -> - case get_crypt_key_from_file() of - {error,_} = E -> - E; - Key -> - decrypt_config_file(EncryptFileName, TargetFileName, {key,Key}) - end. - - -%%%----------------------------------------------------------------- -%%% @spec -%%% -%%% @doc -decrypt_config_file(EncryptFileName, TargetFileName, {file,KeyFile}) -> - case get_crypt_key_from_file(KeyFile) of - {error,_} = E -> - E; - Key -> - decrypt_config_file(EncryptFileName, TargetFileName, {key,Key}) - end; - -decrypt_config_file(EncryptFileName, TargetFileName, {key,Key}) -> - crypto:start(), - {K1,K2,K3,IVec} = make_crypto_key(Key), - case file:read_file(EncryptFileName) of - {ok,Bin} -> - DecBin = crypto:des3_cbc_decrypt(K1, K2, K3, IVec, Bin), - case catch binary_to_term(DecBin) of - {'EXIT',_} -> - {error,bad_file}; - {_SrcFile,SrcBin} -> - case TargetFileName of - undefined -> - {ok,SrcBin}; - _ -> - case file:write_file(TargetFileName, SrcBin) of - ok -> - io:format("~s --(decrypt)--> ~s~n", - [EncryptFileName,TargetFileName]), - ok; - {error,Reason} -> - {error,{Reason,TargetFileName}} - end - end - end; - {error,Reason} -> - {error,{Reason,EncryptFileName}} - end. - - -get_crypt_key_from_file(File) -> - case file:read_file(File) of - {ok,Bin} -> - case catch string:tokens(binary_to_list(Bin), [$\n,$\r]) of - [Key] -> - Key; - _ -> - {error,{bad_crypt_file,File}} - end; - {error,Reason} -> - {error,{Reason,File}} - end. - -get_crypt_key_from_file() -> - CwdFile = filename:join(".",?cryptfile), - {Result,FullName} = - case file:read_file(CwdFile) of - {ok,Bin} -> - {Bin,CwdFile}; - _ -> - case init:get_argument(home) of - {ok,[[Home]]} -> - HomeFile = filename:join(Home,?cryptfile), - case file:read_file(HomeFile) of - {ok,Bin} -> - {Bin,HomeFile}; - _ -> - {{error,no_crypt_file},noent} - end; - _ -> - {{error,no_crypt_file},noent} - end - end, - case FullName of - noent -> - Result; - _ -> - case catch string:tokens(binary_to_list(Result), [$\n,$\r]) of - [Key] -> - io:format("~nCrypt key file: ~s~n", [FullName]), - Key; - _ -> - {error,{bad_crypt_file,FullName}} - end - end. - -make_crypto_key(String) -> - <<K1:8/binary,K2:8/binary>> = First = erlang:md5(String), - <<K3:8/binary,IVec:8/binary>> = erlang:md5([First|lists:reverse(String)]), - {K1,K2,K3,IVec}. - -random_bytes(N) -> - {A,B,C} = now(), - random:seed(A, B, C), - random_bytes_1(N, []). - -random_bytes_1(0, Acc) -> Acc; -random_bytes_1(N, Acc) -> random_bytes_1(N-1, [random:uniform(255)|Acc]). - - -%%%----------------------------------------------------------------- -%%% @spec -%%% -%%% @doc get_attached(TCPid) -> case dbg_iserver:safe_call({get_attpid,TCPid}) of {ok,AttPid} when is_pid(AttPid) -> @@ -1210,7 +677,7 @@ call(Msg) -> ct_util_server ! {Msg,{self(),Ref}}, receive {Ref, Result} -> - erlang:demonitor(MRef), + erlang:demonitor(MRef, [flush]), Result; {'DOWN',MRef,process,_,Reason} -> {error,{ct_util_server_down,Reason}} @@ -1244,37 +711,6 @@ ct_make_ref_loop(N) -> From ! {self(),N}, ct_make_ref_loop(N+1) end. - -get_ref_from_name(Name) -> - case ets:select(?attr_table,[{#ct_conf{name=Name,ref='$1',_='_'}, - [], - ['$1']}]) of - [Ref] -> - {ok,Ref}; - _ -> - {error,{no_such_name,Name}} - end. - -get_name_from_ref(Ref) -> - case ets:select(?attr_table,[{#ct_conf{name='$1',ref=Ref,_='_'}, - [], - ['$1']}]) of - [Name] -> - {ok,Name}; - _ -> - {error,{no_such_ref,Ref}} - end. - -get_key_from_name(Name) -> - case ets:select(?attr_table,[{#ct_conf{name=Name,key='$1',_='_'}, - [], - ['$1']}]) of - [Key|_] -> - {ok,Key}; - _ -> - {error,{no_such_name,Name}} - end. - abs_name(Dir0) -> Abs = filename:absname(Dir0), diff --git a/lib/common_test/src/ct_util.hrl b/lib/common_test/src/ct_util.hrl index c1dc14f943..ee973f6220 100644 --- a/lib/common_test/src/ct_util.hrl +++ b/lib/common_test/src/ct_util.hrl @@ -29,11 +29,16 @@ -record(testspec, {spec_dir, nodes=[], + init=[], + label=[], logdir=["."], cover=[], config=[], + userconfig=[], event_handler=[], include=[], + multiply_timetraps=[], + scale_timetraps=[], alias=[], tests=[]}). @@ -50,3 +55,4 @@ -define(CT_MEVMGR_REF, ct_master_event). -define(missing_suites_info, "missing_suites.info"). +-define(ct_config_txt, ct_config_plain). diff --git a/lib/common_test/src/vts.erl b/lib/common_test/src/vts.erl index ad4845a7c3..2ee982d726 100644 --- a/lib/common_test/src/vts.erl +++ b/lib/common_test/src/vts.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2003-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -100,7 +100,7 @@ start_link() -> MRef = erlang:monitor(process,Pid), receive {Pid,started} -> - erlang:demonitor(MRef), + erlang:demonitor(MRef, [flush]), {ok,Pid}; {'DOWN',MRef,process,_,Reason} -> {error,{vts,died,Reason}} @@ -160,11 +160,13 @@ init(Parent) -> loop(State) -> receive - {{init_data,ConfigFiles,EvHandlers,LogDir,Tests},From} -> - ct_install(State), + {{init_data,Config,EvHandlers,LogDir,Tests},From} -> + %% ct:pal("State#state.current_log_dir=~p", [State#state.current_log_dir]), + NewState = State#state{config=Config,event_handler=EvHandlers, + current_log_dir=LogDir,tests=Tests}, + ct_install(NewState), return(From,ok), - loop(#state{config=ConfigFiles,event_handler=EvHandlers, - current_log_dir=LogDir,tests=Tests}); + loop(NewState); {start_page,From} -> return(From,start_page1()), loop(State); @@ -257,7 +259,7 @@ call(Msg) -> Pid ! {Msg,{self(),Ref}}, receive {Ref, Result} -> - erlang:demonitor(MRef), + erlang:demonitor(MRef, [flush]), Result; {'DOWN',MRef,process,_,Reason} -> {error,{process_down,Pid,Reason}} diff --git a/lib/common_test/test/Makefile b/lib/common_test/test/Makefile index 35ba22aa59..3fb0d627a0 100644 --- a/lib/common_test/test/Makefile +++ b/lib/common_test/test/Makefile @@ -27,13 +27,18 @@ include $(ERL_TOP)/make/$(TARGET)/otp.mk MODULES= \ ct_test_support \ ct_test_support_eh \ + ct_userconfig_callback \ ct_smoke_test_SUITE \ ct_event_handler_SUITE \ ct_groups_test_1_SUITE \ ct_groups_test_2_SUITE \ + ct_testspec_1_SUITE \ ct_skip_SUITE \ ct_error_SUITE \ - ct_test_server_if_1_SUITE + ct_test_server_if_1_SUITE \ + ct_config_SUITE \ + ct_master_SUITE \ + ct_misc_1_SUITE ERL_FILES= $(MODULES:%=%.erl) @@ -64,15 +69,15 @@ EBIN = . #.PHONY: make_emakefile #make_emakefile: -# $(ERL_TOP)/make/make_emakefile $(ERL_COMPILE_FLAGS) -o$(EBIN) \ -# '*_SUITE_make' > $(EMAKEFILE) # $(ERL_TOP)/make/make_emakefile $(ERL_COMPILE_FLAGS) -o$(EBIN) $(MODULES)\ -# >> $(EMAKEFILE) +# > $(EMAKEFILE) tests debug opt: + erl $(ERL_MAKE_FLAGS) -make clean: rm -f $(TARGET_FILES) +# rm -f $(EMAKEFILE) rm -f core docs: diff --git a/lib/common_test/test/ct_config_SUITE.erl b/lib/common_test/test/ct_config_SUITE.erl new file mode 100644 index 0000000000..72ff781f82 --- /dev/null +++ b/lib/common_test/test/ct_config_SUITE.erl @@ -0,0 +1,255 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%------------------------------------------------------------------- +%%% File: ct_config_SUITE +%%% +%%% Description: +%%% Test configuration handling in Common Test suites. +%%% +%%% The suites used for the test are located in the data directory. +%%%------------------------------------------------------------------- +-module(ct_config_SUITE). + +-compile(export_all). + +-include_lib("test_server/include/test_server.hrl"). +-include_lib("common_test/include/ct_event.hrl"). + +-define(eh, ct_test_support_eh). + +%%-------------------------------------------------------------------- +%% TEST SERVER CALLBACK FUNCTIONS +%%-------------------------------------------------------------------- + +%%-------------------------------------------------------------------- +%% Description: Since Common Test starts another Test Server +%% instance, the tests need to be performed on a separate node (or +%% there will be clashes with logging processes etc). +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + DataDir = ?config(data_dir, Config), + PathDir = filename:join(DataDir, "config/test"), + Config1 = ct_test_support:init_per_suite([{path_dirs,[PathDir]} | Config]), + PrivDir = ?config(priv_dir, Config1), + ConfigDir = filename:join(PrivDir, "config"), + ok = file:make_dir(ConfigDir), + [{config_dir,ConfigDir} | Config1]. + +end_per_suite(Config) -> + ct_test_support:end_per_suite(Config). + +init_per_testcase(TestCase, Config) -> + ct_test_support:init_per_testcase(TestCase, Config). + +end_per_testcase(TestCase, Config) -> + ct_test_support:end_per_testcase(TestCase, Config). + +all(doc) -> + [""]; + +all(suite) -> + [ + require, + userconfig_static, + userconfig_dynamic, + testspec_legacy, + testspec_static, + testspec_dynamic + ]. + +%%-------------------------------------------------------------------- +%% TEST CASES +%%-------------------------------------------------------------------- +require(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + run_test(config_static_SUITE, + Config, + {config, filename:join(DataDir, "config/config.txt")}, + ["config_static_SUITE"]). + +userconfig_static(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + run_test(config_static_SUITE, + Config, + {userconfig, {ct_config_xml, filename:join(DataDir, "config/config.xml")}}, + ["config_static_SUITE"]). + +userconfig_dynamic(Config) when is_list(Config) -> + run_test(config_dynamic_SUITE, + Config, + {userconfig, {config_driver, "config_server"}}, + ["config_dynamic_SUITE"]). + +testspec_legacy(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + ConfigDir = ?config(config_dir, Config), + make_spec(DataDir, ConfigDir, + "spec_legacy.spec", + [config_static_SUITE], + [{config, filename:join(DataDir, "config/config.txt")}]), + run_test(config_static_SUITE, + Config, + {spec, filename:join(ConfigDir, "spec_legacy.spec")}, + []), + file:delete(filename:join(ConfigDir, "spec_legacy.spec")). + +testspec_static(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + ConfigDir = ?config(config_dir, Config), + make_spec(DataDir, ConfigDir, + "spec_static.spec", + [config_static_SUITE], + [{userconfig, {ct_config_xml, filename:join(DataDir, "config/config.xml")}}]), + run_test(config_static_SUITE, + Config, + {spec, filename:join(ConfigDir, "spec_static.spec")}, + []), + file:delete(filename:join(ConfigDir, "spec_static.spec")). + +testspec_dynamic(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + ConfigDir = ?config(config_dir, Config), + make_spec(DataDir, ConfigDir, "spec_dynamic.spec", + [config_dynamic_SUITE], + [{userconfig, {config_driver, "config_server"}}]), + run_test(config_dynamic_SUITE, + Config, + {spec, filename:join(ConfigDir, "spec_dynamic.spec")}, + []), + file:delete(filename:join(ConfigDir, "spec_dynamic.spec")). + +%%%----------------------------------------------------------------- +%%% HELP FUNCTIONS +%%%----------------------------------------------------------------- +make_spec(DataDir, ConfigDir, Filename, Suites, Config)-> + {ok, Fd} = file:open(filename:join(ConfigDir, Filename), [write]), + ok = file:write(Fd, + io_lib:format("{suites, \"~sconfig/test/\", ~p}.~n", [DataDir, Suites])), + lists:foreach(fun(C)-> ok=file:write(Fd, io_lib:format("~p.~n", [C])) end, Config), + ok = file:close(Fd). + +run_test(Name, Config, CTConfig, SuiteNames)-> + DataDir = ?config(data_dir, Config), + Joiner = fun(Suite) -> filename:join(DataDir, "config/test/"++Suite) end, + Suites = lists:map(Joiner, SuiteNames), + {Opts,ERPid} = setup_env({suite,Suites}, Config, CTConfig), + ok = ct_test_support:run(Opts, Config), + TestEvents = ct_test_support:get_events(ERPid, Config), + ct_test_support:log_events(Name, + reformat_events(TestEvents, ?eh), + ?config(config_dir, Config)), + ExpEvents = events_to_check(Name), + ok = ct_test_support:verify_events(ExpEvents, TestEvents, Config). + +setup_env(Test, Config, CTConfig) -> + Opts0 = ct_test_support:get_opts(Config), + Level = ?config(trace_level, Config), + EvHArgs = [{cbm,ct_test_support},{trace_level,Level}], + Opts = Opts0 ++ [Test,{event_handler,{?eh,EvHArgs}}, CTConfig], + ERPid = ct_test_support:start_event_receiver(Config), + {Opts,ERPid}. + +reformat_events(Events, EH) -> + ct_test_support:reformat(Events, EH). + +%%%----------------------------------------------------------------- +%%% TEST EVENTS +%%%----------------------------------------------------------------- +events_to_check(Test) -> + %% 2 tests (ct:run_test + script_start) is default + events_to_check(Test, 2). + +events_to_check(_, 0) -> + []; +events_to_check(Test, N) -> + expected_events(Test) ++ events_to_check(Test, N-1). + +expected_events(config_static_SUITE)-> +[ + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{1,1,8}}, + {?eh,tc_start,{config_static_SUITE,init_per_suite}}, + {?eh,tc_done,{config_static_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{config_static_SUITE,test_get_config_simple}}, + {?eh,tc_done,{config_static_SUITE,test_get_config_simple,ok}}, + {?eh,test_stats,{1,0,{0,0}}}, + {?eh,tc_start,{config_static_SUITE,test_get_config_nested}}, + {?eh,tc_done,{config_static_SUITE,test_get_config_nested,ok}}, + {?eh,test_stats,{2,0,{0,0}}}, + {?eh,tc_start,{config_static_SUITE,test_default_suitewide}}, + {?eh,tc_done,{config_static_SUITE,test_default_suitewide,ok}}, + {?eh,test_stats,{3,0,{0,0}}}, + {?eh,tc_start,{config_static_SUITE,test_config_name_already_in_use1}}, + {?eh,tc_done, + {config_static_SUITE,test_config_name_already_in_use1,{skipped,{config_name_already_in_use,[x1]}}}}, + {?eh,test_stats,{3,0,{1,0}}}, + {?eh,tc_start,{config_static_SUITE,test_default_tclocal}}, + {?eh,tc_done,{config_static_SUITE,test_default_tclocal,ok}}, + {?eh,test_stats,{4,0,{1,0}}}, + {?eh,tc_start,{config_static_SUITE,test_config_name_already_in_use2}}, + {?eh,tc_done, + {config_static_SUITE,test_config_name_already_in_use2, + {skipped,{config_name_already_in_use,[x1,alias]}}}}, + {?eh,test_stats,{4,0,{2,0}}}, + {?eh,tc_start,{config_static_SUITE,test_alias_tclocal}}, + {?eh,tc_done,{config_static_SUITE,test_alias_tclocal,ok}}, + {?eh,test_stats,{5,0,{2,0}}}, + {?eh,tc_start,{config_static_SUITE,test_get_config_undefined}}, + {?eh,tc_done,{config_static_SUITE,test_get_config_undefined,ok}}, + {?eh,test_stats,{6,0,{2,0}}}, + {?eh,tc_start,{config_static_SUITE,end_per_suite}}, + {?eh,tc_done,{config_static_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]} +]; + +expected_events(config_dynamic_SUITE)-> +[ + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{1,1,5}}, + {?eh,tc_start,{config_dynamic_SUITE,init_per_suite}}, + {?eh,tc_done,{config_dynamic_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{config_dynamic_SUITE,test_get_known_variable}}, + {?eh,tc_done, + {config_dynamic_SUITE,test_get_known_variable,ok}}, + {?eh,test_stats,{1,0,{0,0}}}, + {?eh,tc_start,{config_dynamic_SUITE,test_localtime_update}}, + {?eh,tc_done,{config_dynamic_SUITE,test_localtime_update,ok}}, + {?eh,test_stats,{2,0,{0,0}}}, + {?eh,tc_start,{config_dynamic_SUITE,test_server_pid}}, + {?eh,tc_done,{config_dynamic_SUITE,test_server_pid,ok}}, + {?eh,test_stats,{3,0,{0,0}}}, + {?eh,tc_start, + {config_dynamic_SUITE,test_disappearable_variable}}, + {?eh,tc_done, + {config_dynamic_SUITE,test_disappearable_variable,ok}}, + {?eh,test_stats,{4,0,{0,0}}}, + {?eh,tc_start, + {config_dynamic_SUITE,test_disappearable_variable_alias}}, + {?eh,tc_done, + {config_dynamic_SUITE,test_disappearable_variable_alias,ok}}, + {?eh,test_stats,{5,0,{0,0}}}, + {?eh,tc_start,{config_dynamic_SUITE,end_per_suite}}, + {?eh,tc_done,{config_dynamic_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]} +]. diff --git a/lib/common_test/test/ct_config_SUITE_data/config/config.txt b/lib/common_test/test/ct_config_SUITE_data/config/config.txt new file mode 100644 index 0000000000..fcbffcd7f3 --- /dev/null +++ b/lib/common_test/test/ct_config_SUITE_data/config/config.txt @@ -0,0 +1,31 @@ +{x, suite}. +{gen_cfg, + [ + {a,a_value}, + {b,b_value} + ]}. +{gen_cfg2, + [ + {c, "Hello, world!"}, + {d, atom_value}, + {e, {tuple,1,"third value",[]}}, + {f, []}, + {g, [1,atom,"string",13.6,{1,2,3}]} + ]}. +{gen_cfg3, + [ + {h, + [ + {i, third1}, + {j, "Third2"}, + {k, 'THIRD3'} + ]}, + {l, + [ + {m, + [ + {n, "N"}, + {o, 'O'} + ]} + ]} + ]}. diff --git a/lib/common_test/test/ct_config_SUITE_data/config/config.xml b/lib/common_test/test/ct_config_SUITE_data/config/config.xml new file mode 100644 index 0000000000..0a3e5f2e31 --- /dev/null +++ b/lib/common_test/test/ct_config_SUITE_data/config/config.xml @@ -0,0 +1,27 @@ +<config> + <x>suite</x> + <gen_cfg> + <a>a_value</a> + <b>b_value</b> + </gen_cfg> + <gen_cfg2> + <c>"Hello, world!"</c> + <d>atom_value</d> + <e>{tuple,1,"third value",[]}</e> + <f>[]</f> + <g>[1,atom,"string",13.6,{1,2,3}]</g> + </gen_cfg2> + <gen_cfg3> + <h> + <i>third1</i> + <j>"Third2"</j> + <k>'THIRD3'</k> + </h> + <l> + <m> + <n>"N"</n> + <o>'O'</o> + </m> + </l> + </gen_cfg3> +</config> diff --git a/lib/common_test/test/ct_config_SUITE_data/config/test/config_driver.erl b/lib/common_test/test/ct_config_SUITE_data/config/test/config_driver.erl new file mode 100644 index 0000000000..d93faf6ec6 --- /dev/null +++ b/lib/common_test/test/ct_config_SUITE_data/config/test/config_driver.erl @@ -0,0 +1,46 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%------------------------------------------------------------------- +%%% File: ct_config_SUITE +%%% +%%% Description: +%%% Config driver used in the CT's tests (config_2_SUITE) +%%%------------------------------------------------------------------- +-module(config_driver). +-export([read_config/1, check_parameter/1]). + +read_config(ServerName)-> + ServerModule = list_to_atom(ServerName), + ServerModule:start(), + ServerModule:get_config(). + +check_parameter(ServerName)-> + ServerModule = list_to_atom(ServerName), + case code:is_loaded(ServerModule) of + {file, _}-> + {ok, {config, ServerName}}; + false-> + case code:load_file(ServerModule) of + {module, ServerModule}-> + {ok, {config, ServerName}}; + {error, nofile}-> + {error, {wrong_config, "File not found: " ++ ServerName ++ ".beam"}} + end + end. diff --git a/lib/common_test/test/ct_config_SUITE_data/config/test/config_dynamic_SUITE.erl b/lib/common_test/test/ct_config_SUITE_data/config/test/config_dynamic_SUITE.erl new file mode 100644 index 0000000000..8ee12a2e4d --- /dev/null +++ b/lib/common_test/test/ct_config_SUITE_data/config/test/config_dynamic_SUITE.erl @@ -0,0 +1,145 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%------------------------------------------------------------------- +%%% File: config_dynamic_SUITE +%%% +%%% Description: +%%% Test suite for common_test which tests the userconfig functionality +%%%------------------------------------------------------------------- +-module(config_dynamic_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%% This suite will be run with dynamic userconfig +%% test_driver.erl is compliant to ct_config_* callback +%% and test_server is simple server for getting runtime-changing data +%% which will return the list with the following variables: +%% localtime = the erlang:localtime() result in list [{date, Date}, {time, Time}] +%% node = erlang:node() - can be compared in the testcase +%% now = erlang:now() - easier to compare than localtime() +%% config_server_pid - pid of the config server, should NOT change! +%% config_server_vsn - .19 +%% config_server_iteration - a number of iteration config_server's loop done +%% disappearable_variable - hereAmI - will be absent on even iterations + +suite() -> + [ + {timetrap, {seconds,10}} + ]. + +init_per_suite(Config) -> + Config. + +end_per_suite(_) -> + ok. + +all() -> [test_get_known_variable, test_localtime_update, + test_server_pid, test_disappearable_variable, + test_disappearable_variable_alias]. + +init_per_testcase(_, Config) -> + Config. + +end_per_testcase(_, _) -> + ok. + +% test that usual config works +test_get_known_variable(_)-> + Node = erlang:node(), + 0.19 = ct:get_config(config_server_vsn), + Node = ct:get_config(node), + ok. + +% localtime will be updated in 5 seconds, check that +test_localtime_update(_)-> + Seconds = 5, + LT1 = ct:get_config(localtime), + timer:sleep(Seconds*1000), + LT2 = ct:reload_config(localtime), + case is_diff_ok(LT1, LT2, Seconds) of + {false, Actual, Exp}-> + ct:fail(io_lib:format("Time difference ~p is not ok, expected ~p", [Actual, Exp])); + true-> + ok + end. + +% server pid should not change +test_server_pid()-> + [{require, cfvsn, config_server_vsn}]. +test_server_pid(_)-> + Pid = ct:get_config(config_server_pid), + Pid = ct:reload_config(config_server_pid), + Vsn = ct:get_config(config_server_vsn), + % aliases remain after config reloading + Vsn = ct:get_config(cfvsn), + ok. + +% test that variables may disappear from the config_2_SUITE +test_disappearable_variable(_)-> + % ask CT for config_server_iteration variable + Iter = ct:reload_config(config_server_iteration), + % here we should reload this variable in case it's odd + if Iter rem 2 == 1-> + Iter2 = ct:reload_config(config_server_iteration), + Iter2 = Iter+1; + true->ok + end, + % now disappearable_variable should be in place + hereAmI = ct:get_config(disappearable_variable), + % and now it should disappear + undefined = ct:reload_config(disappearable_variable). + +% alias of disappearable_variable should disappear too +test_disappearable_variable_alias(_)-> + % the same rules apply for this testcase as for previous one + Iter = ct:reload_config(config_server_iteration), + Iter2 = if + Iter rem 2 == 1 -> + NewIter = ct:reload_config(config_server_iteration), + NewIter = Iter+1; + true-> + Iter + end, + ct:require(diav, disappearable_variable), + hereAmI = ct:get_config(disappearable_variable), + hereAmI = ct:get_config(diav), + ct:reload_config(disappearable_variable), + undefined = ct:get_config(disappearable_variable), + % after reloading, it's even again + Iter3=ct:get_config(config_server_iteration), + Iter3 = Iter2+1, + % and alias does not exist + undefined = ct:get_config(diav). + +my_dt_to_datetime([{date, D},{time, T}])-> + {D, T}. + +is_diff_ok(DT1, DT2, Seconds)-> + GS1 = calendar:datetime_to_gregorian_seconds(my_dt_to_datetime(DT1)), + GS2 = calendar:datetime_to_gregorian_seconds(my_dt_to_datetime(DT2)), + if + GS2-GS1 > Seconds+Seconds/2; + GS2-GS1 < Seconds-Seconds/2-> + {false, GS2-GS1, Seconds}; + true-> + true + end. diff --git a/lib/common_test/test/ct_config_SUITE_data/config/test/config_server.erl b/lib/common_test/test/ct_config_SUITE_data/config/test/config_server.erl new file mode 100644 index 0000000000..8463fea645 --- /dev/null +++ b/lib/common_test/test/ct_config_SUITE_data/config/test/config_server.erl @@ -0,0 +1,93 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%------------------------------------------------------------------- +%%% File: ct_config_SUITE +%%% +%%% Description: +%%% Config server used in the CT's tests (config_2_SUITE) +%%%------------------------------------------------------------------- +-module(config_server). +-export([start/0, stop/0, loop/1, init/1, get_config/0]). + +-define(REGISTERED_NAME, ct_test_config_server). +-define(vsn, 0.19). + +start()-> + case whereis(?REGISTERED_NAME) of + undefined-> + spawn(?MODULE, init, [?REGISTERED_NAME]), + wait(); + _Pid-> + ok + end, + ?REGISTERED_NAME. + +init(Name)-> + register(Name, self()), + loop(0). + +get_config()-> + call(self(), get_config). + +stop()-> + call(self(), stop). + +call(Client, Request)-> + case whereis(?REGISTERED_NAME) of + undefined-> + {error, not_started, Request}; + Pid-> + Pid ! {Client, Request}, + receive + Reply-> + {ok, Reply} + after 4000-> + {error, timeout, Request} + end + end. + +loop(Iteration)-> + receive + {Pid, stop}-> + Pid ! ok; + {Pid, get_config}-> + {D,T} = erlang:localtime(), + Config = + [{localtime, [{date, D}, {time, T}]}, + {node, erlang:node()}, + {config_server_iteration, Iteration}, + {now, erlang:now()}, + {config_server_pid, self()}, + {config_server_vsn, ?vsn}], + Config2 = if Iteration rem 2 == 0-> + Config ++ [{disappearable_variable, hereAmI}]; + true-> Config + end, + Pid ! Config2, + ?MODULE:loop(Iteration+1) + end. + +wait()-> + case whereis(?REGISTERED_NAME) of + undefined-> + wait(); + _Pid-> + ok + end. diff --git a/lib/common_test/test/ct_config_SUITE_data/config/test/config_static_SUITE.erl b/lib/common_test/test/ct_config_SUITE_data/config/test/config_static_SUITE.erl new file mode 100644 index 0000000000..8751a2e8f3 --- /dev/null +++ b/lib/common_test/test/ct_config_SUITE_data/config/test/config_static_SUITE.erl @@ -0,0 +1,123 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%------------------------------------------------------------------- +%%% File: config_static_SUITE +%%% +%%% Description: +%%% Test suite for common_test which tests the get_config and require +%%% functionality +%%%------------------------------------------------------------------- +-module(config_static_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +% The config contains variables: +% x - atom +% gen_cfg - list with two key-values tagged with a and b +% gen_cfg2 - list of five key-values tagged with c, d, e, f and g +% gen_cfg3 - list of two complex key-values taggen with: +% h: three elements inside - i, j and k +% l: m inside, contains n and o + +suite() -> + [ + {timetrap, {seconds,10}}, + %% x1 doesn't exist in cfg-file! + {require, x1, x}, + {require, gen_cfg3}, + {require, alias, gen_cfg}, + %% x1 default value + {x1, {x,suite}} + ]. + +init_per_suite(Config) -> + Config. + +end_per_suite(_) -> + ok. + +all() -> [test_get_config_simple, test_get_config_nested, test_default_suitewide, + test_config_name_already_in_use1, test_default_tclocal, + test_config_name_already_in_use2, test_alias_tclocal, + test_get_config_undefined]. + +init_per_testcase(_, Config) -> + Config. + +end_per_testcase(_, _) -> + ok. + +%% test getting a simple value +test_get_config_simple(_)-> + suite = ct:get_config(x), + ok. + +%% test getting a nested value +test_get_config_nested(_)-> + a_value = ct:get_config({gen_cfg, a}), + ok. + +%% test suite-wide default value +test_default_suitewide(_)-> + suite = ct:get_config(x1), + ok. + +%% should get skipped +test_config_name_already_in_use1() -> + [{timetrap, {seconds,2}}, + {require, x1, x}, + {x1, {x,test2}}]. +test_config_name_already_in_use1(_) -> + ct:fail("Test should've been skipped, you shouldn't see this!"), + ok. + +%% test defaults in a testcase +test_default_tclocal() -> + [{timetrap, {seconds,3}}, + {require, y1, y}, + {y1, {y,test3}}]. +test_default_tclocal(_) -> + test3 = ct:get_config(y1), + ok. + +%% should get skipped +test_config_name_already_in_use2() -> + [{require,alias,something}, + {alias,{something,else}}, + {require, x1, x}, + {x1, {x,test4}}]. +test_config_name_already_in_use2(_) -> + ct:fail("Test should've been skipped, you shouldn't see this!"), + ok. + +%% test aliases +test_alias_tclocal() -> + [{require,newalias,gen_cfg}]. +test_alias_tclocal(_) -> + A = [{a,a_value},{b,b_value}] = ct:get_config(newalias), + A = ct:get_config(gen_cfg), + ok. + +%% test for getting undefined variables +test_get_config_undefined(_) -> + undefined = ct:get_config(y1), + ok. diff --git a/lib/common_test/test/ct_error_SUITE.erl b/lib/common_test/test/ct_error_SUITE.erl index be75d768fc..2fa031b884 100644 --- a/lib/common_test/test/ct_error_SUITE.erl +++ b/lib/common_test/test/ct_error_SUITE.erl @@ -63,7 +63,10 @@ all(suite) -> [ cfg_error, lib_error, - no_compile + no_compile, + timetrap_end_conf, + timetrap_normal, + timetrap_extended ]. @@ -84,17 +87,22 @@ cfg_error(Config) when is_list(Config) -> Join(DataDir, "cfg_error_6_SUITE"), Join(DataDir, "cfg_error_7_SUITE"), Join(DataDir, "cfg_error_8_SUITE"), - Join(DataDir, "cfg_error_9_SUITE") + Join(DataDir, "cfg_error_9_SUITE"), + Join(DataDir, "cfg_error_10_SUITE"), + Join(DataDir, "cfg_error_11_SUITE"), + Join(DataDir, "cfg_error_12_SUITE"), + Join(DataDir, "cfg_error_13_SUITE"), + Join(DataDir, "cfg_error_14_SUITE") ], - {Opts,ERPid} = setup({suite,Suites}, Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + {Opts,ERPid} = setup([{suite,Suites}], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), ct_test_support:log_events(cfg_error, reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(cfg_error), + TestEvents = events_to_check(cfg_error), ok = ct_test_support:verify_events(TestEvents, Events, Config). @@ -104,15 +112,15 @@ lib_error(Config) when is_list(Config) -> DataDir = ?config(data_dir, Config), Join = fun(D, S) -> filename:join(D, "error/test/"++S) end, Suites = [Join(DataDir, "lib_error_1_SUITE")], - {Opts,ERPid} = setup({suite,Suites}, Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + {Opts,ERPid} = setup([{suite,Suites}], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), ct_test_support:log_events(lib_error, reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(lib_error), + TestEvents = events_to_check(lib_error), ok = ct_test_support:verify_events(TestEvents, Events, Config). @@ -122,17 +130,75 @@ no_compile(Config) when is_list(Config) -> DataDir = ?config(data_dir, Config), Join = fun(D, S) -> filename:join(D, "error/test/"++S) end, Suites = [Join(DataDir, "no_compile_SUITE")], - {Opts,ERPid} = setup({suite,Suites}, Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + {Opts,ERPid} = setup([{suite,Suites}], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), ct_test_support:log_events(no_compile, reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(no_compile), + TestEvents = events_to_check(no_compile), ok = ct_test_support:verify_events(TestEvents, Events, Config). +%%%----------------------------------------------------------------- +%%% +timetrap_end_conf(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + Join = fun(D, S) -> filename:join(D, "error/test/"++S) end, + Suites = [Join(DataDir, "timetrap_1_SUITE")], + {Opts,ERPid} = setup([{suite,Suites}], Config), + ok = ct_test_support:run(Opts, Config), + Events = ct_test_support:get_events(ERPid, Config), + + ct_test_support:log_events(timetrap_end_conf, + reformat(Events, ?eh), + ?config(priv_dir, Config)), + + TestEvents = events_to_check(timetrap_end_conf), + ok = ct_test_support:verify_events(TestEvents, Events, Config). + +%%%----------------------------------------------------------------- +%%% +timetrap_normal(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + Join = fun(D, S) -> filename:join(D, "error/test/"++S) end, + Suite = Join(DataDir, "timetrap_2_SUITE"), + {Opts,ERPid} = setup([{suite,Suite}, + {userconfig,{ct_userconfig_callback, + "multiply 1 scale false"}}], + Config), + ok = ct_test_support:run(Opts, Config), + Events = ct_test_support:get_events(ERPid, Config), + + ct_test_support:log_events(timetrap_normal, + reformat(Events, ?eh), + ?config(priv_dir, Config)), + + TestEvents = events_to_check(timetrap_normal), + ok = ct_test_support:verify_events(TestEvents, Events, Config). + +%%%----------------------------------------------------------------- +%%% +timetrap_extended(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + Join = fun(D, S) -> filename:join(D, "error/test/"++S) end, + Suite = Join(DataDir, "timetrap_2_SUITE"), + {Opts,ERPid} = setup([{suite,Suite}, + {multiply_timetraps,2}, + {scale_timetraps,false}, + {userconfig,{ct_userconfig_callback, + "multiply 2 scale false"}}], + Config), + ok = ct_test_support:run(Opts, Config), + Events = ct_test_support:get_events(ERPid, Config), + + ct_test_support:log_events(timetrap_extended, + reformat(Events, ?eh), + ?config(priv_dir, Config)), + + TestEvents = events_to_check(timetrap_extended), + ok = ct_test_support:verify_events(TestEvents, Events, Config). %%%----------------------------------------------------------------- %%% HELP FUNCTIONS @@ -142,7 +208,7 @@ setup(Test, Config) -> Opts0 = ct_test_support:get_opts(Config), Level = ?config(trace_level, Config), EvHArgs = [{cbm,ct_test_support},{trace_level,Level}], - Opts = Opts0 ++ [Test,{event_handler,{?eh,EvHArgs}}], + Opts = Opts0 ++ [{event_handler,{?eh,EvHArgs}}|Test], ERPid = ct_test_support:start_event_receiver(Config), {Opts,ERPid}. @@ -154,11 +220,20 @@ reformat(Events, EH) -> %%%----------------------------------------------------------------- %%% TEST EVENTS %%%----------------------------------------------------------------- +events_to_check(Test) -> + %% 2 tests (ct:run_test + script_start) is default + events_to_check(Test, 2). + +events_to_check(_, 0) -> + []; +events_to_check(Test, N) -> + test_events(Test) ++ events_to_check(Test, N-1). + test_events(cfg_error) -> [ {?eh,start_logging,{'DEF','RUNDIR'}}, {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, - {?eh,start_info,{9,9,33}}, + {?eh,start_info,{14,14,42}}, {?eh,tc_start,{cfg_error_1_SUITE,init_per_suite}}, {?eh,tc_done, @@ -470,6 +545,59 @@ test_events(cfg_error) -> {?eh,tc_start,{cfg_error_9_SUITE,end_per_suite}}, {?eh,tc_done,{cfg_error_9_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{cfg_error_10_SUITE,init_per_suite}}, + {?eh,tc_done,{cfg_error_10_SUITE,init_per_suite, + {failed,{error,fail_init_per_suite}}}}, + {?eh,tc_auto_skip,{cfg_error_10_SUITE,tc1, + {failed,{cfg_error_10_SUITE,init_per_suite, + {failed,fail_init_per_suite}}}}}, + {?eh,test_stats,{12,3,{0,19}}}, + {?eh,tc_auto_skip,{cfg_error_10_SUITE,end_per_suite, + {failed,{cfg_error_10_SUITE,init_per_suite, + {failed,fail_init_per_suite}}}}}, + {?eh,tc_start,{cfg_error_11_SUITE,init_per_suite}}, + {?eh,tc_done,{cfg_error_11_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{cfg_error_11_SUITE,tc1}}, + {?eh,tc_done,{cfg_error_11_SUITE,tc1, + {skipped,{config_name_already_in_use,[dummy0]}}}}, + {?eh,test_stats,{12,3,{1,19}}}, + {?eh,tc_start,{cfg_error_11_SUITE,tc2}}, + {?eh,tc_done,{cfg_error_11_SUITE,tc2,ok}}, + {?eh,test_stats,{13,3,{1,19}}}, + {?eh,tc_start,{cfg_error_11_SUITE,end_per_suite}}, + {?eh,tc_done,{cfg_error_11_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{cfg_error_12_SUITE,tc1}}, + {?eh,tc_done,{cfg_error_12_SUITE,tc1,{failed,{timetrap_timeout,500}}}}, + {?eh,test_stats,{13,4,{1,19}}}, + {?eh,tc_start,{cfg_error_12_SUITE,tc2}}, + {?eh,tc_done,{cfg_error_12_SUITE,tc2,{failed, + {cfg_error_12_SUITE,end_per_testcase, + {timetrap_timeout,500}}}}}, + {?eh,test_stats,{14,4,{1,19}}}, + {?eh,tc_start,{cfg_error_12_SUITE,tc3}}, + {?eh,tc_done,{cfg_error_12_SUITE,tc3,ok}}, + {?eh,test_stats,{15,4,{1,19}}}, + {?eh,tc_start,{cfg_error_12_SUITE,tc4}}, + {?eh,tc_done,{cfg_error_12_SUITE,tc4,{failed, + {cfg_error_12_SUITE,end_per_testcase, + {timetrap_timeout,500}}}}}, + {?eh,test_stats,{16,4,{1,19}}}, + {?eh,tc_start,{cfg_error_13_SUITE,init_per_suite}}, + {?eh,tc_done,{cfg_error_13_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{cfg_error_13_SUITE,tc1}}, + {?eh,tc_done,{cfg_error_13_SUITE,tc1,ok}}, + {?eh,test_stats,{17,4,{1,19}}}, + {?eh,tc_start,{cfg_error_13_SUITE,end_per_suite}}, + {?eh,tc_done,{cfg_error_13_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{cfg_error_14_SUITE,init_per_suite}}, + {?eh,tc_done,{cfg_error_14_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{cfg_error_14_SUITE,tc1}}, + {?eh,tc_done,{cfg_error_14_SUITE,tc1,ok}}, + {?eh,test_stats,{18,4,{1,19}}}, + {?eh,tc_start,{cfg_error_14_SUITE,end_per_suite}}, + {?eh,tc_done,{cfg_error_14_SUITE,end_per_suite, + {comment, + "should succeed since ct_fw cancels timetrap in end_tc"}}}, {?eh,test_done,{'DEF','STOP_TIME'}}, {?eh,stop_logging,[]} ]; @@ -555,4 +683,91 @@ test_events(lib_error) -> ]; test_events(no_compile) -> - []. + []; + +test_events(timetrap_end_conf) -> + [ + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{1,1,6}}, + {?eh,tc_start,{timetrap_1_SUITE,init_per_suite}}, + {?eh,tc_done,{timetrap_1_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{timetrap_1_SUITE,tc1}}, + {?eh,tc_done, + {timetrap_1_SUITE,tc1,{failed,{timetrap_timeout,1000}}}}, + {?eh,test_stats,{0,1,{0,0}}}, + {?eh,tc_start,{timetrap_1_SUITE,tc2}}, + {?eh,tc_done, + {timetrap_1_SUITE,tc2,{failed,{timetrap_timeout,1000}}}}, + {?eh,test_stats,{0,2,{0,0}}}, + {?eh,tc_start,{timetrap_1_SUITE,tc3}}, + {?eh,tc_done, + {timetrap_1_SUITE,tc3,{failed,{testcase_aborted,testing_end_conf}}}}, + {?eh,test_stats,{0,3,{0,0}}}, + {?eh,tc_start,{timetrap_1_SUITE,tc4}}, + {?eh,tc_done, + {timetrap_1_SUITE,tc4,{failed,{testcase_aborted,testing_end_conf}}}}, + {?eh,test_stats,{0,4,{0,0}}}, + {?eh,tc_start,{timetrap_1_SUITE,tc5}}, + {?eh,tc_done, + {timetrap_1_SUITE,tc5,{failed,{timetrap_timeout,1000}}}}, + {?eh,test_stats,{0,5,{0,0}}}, + {?eh,tc_start,{timetrap_1_SUITE,tc6}}, + {?eh,tc_done, + {timetrap_1_SUITE,tc6,{failed,{testcase_aborted,testing_end_conf}}}}, + {?eh,test_stats,{0,6,{0,0}}}, + {?eh,tc_start,{timetrap_1_SUITE,end_per_suite}}, + {?eh,tc_done,{timetrap_1_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]} + ]; + +test_events(timetrap_normal) -> + [ + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{1,1,3}}, + {?eh,tc_start,{timetrap_2_SUITE,init_per_suite}}, + {?eh,tc_done,{timetrap_2_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{timetrap_2_SUITE,tc0}}, + {?eh,tc_done, + {timetrap_2_SUITE,tc0,{failed,{timetrap_timeout,3000}}}}, + {?eh,test_stats,{0,1,{0,0}}}, + {?eh,tc_start,{timetrap_2_SUITE,tc1}}, + {?eh,tc_done, + {timetrap_2_SUITE,tc1,{failed,{timetrap_timeout,1000}}}}, + {?eh,test_stats,{0,2,{0,0}}}, + {?eh,tc_start,{timetrap_2_SUITE,tc2}}, + {?eh,tc_done, + {timetrap_2_SUITE,tc2,{failed,{timetrap_timeout,500}}}}, + {?eh,test_stats,{0,3,{0,0}}}, + {?eh,tc_start,{timetrap_2_SUITE,end_per_suite}}, + {?eh,tc_done,{timetrap_2_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]} + ]; + +test_events(timetrap_extended) -> + [ + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{1,1,3}}, + {?eh,tc_start,{timetrap_2_SUITE,init_per_suite}}, + {?eh,tc_done,{timetrap_2_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{timetrap_2_SUITE,tc0}}, + {?eh,tc_done, + {timetrap_2_SUITE,tc0,{failed,{timetrap_timeout,6000}}}}, + {?eh,test_stats,{0,1,{0,0}}}, + {?eh,tc_start,{timetrap_2_SUITE,tc1}}, + {?eh,tc_done, + {timetrap_2_SUITE,tc1,{failed,{timetrap_timeout,2000}}}}, + {?eh,test_stats,{0,2,{0,0}}}, + {?eh,tc_start,{timetrap_2_SUITE,tc2}}, + {?eh,tc_done, + {timetrap_2_SUITE,tc2,{failed,{timetrap_timeout,1000}}}}, + {?eh,test_stats,{0,3,{0,0}}}, + {?eh,tc_start,{timetrap_2_SUITE,end_per_suite}}, + {?eh,tc_done,{timetrap_2_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]} + ]. diff --git a/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_10_SUITE.erl b/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_10_SUITE.erl new file mode 100644 index 0000000000..9f9a90372b --- /dev/null +++ b/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_10_SUITE.erl @@ -0,0 +1,123 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(cfg_error_10_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% Function: suite() -> Info +%% Info = [tuple()] +%%-------------------------------------------------------------------- +suite() -> + [{timetrap,{seconds,2}}, + {require, dummy0}, {default_config, dummy0, "suite/0"}, + {require, dummy1}, {default_config, dummy1, "suite/0"}, + {require, dummy2}, {default_config, dummy2, "suite/0"}]. + +%%-------------------------------------------------------------------- +%% Function: init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_suite() -> + put('$test_server_framework_test', + fun(init_tc, _Default) -> {fail,fail_init_per_suite}; + (_, Default) -> Default + end), + [{require, dummy3}, {default_config, dummy3, "init_per_suite/0"}, + {timetrap,3000}]. + +init_per_suite(Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_suite(Config0) -> void() | {save_config,Config1} +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_suite(Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Function: init_per_group(GroupName, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_group(_GroupName, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_group(GroupName, Config0) -> +%% void() | {save_config,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_group(_GroupName, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Function: init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_testcase(_, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_testcase(_TestCase, _Config) -> + done. + +%%-------------------------------------------------------------------- +%% Function: groups() -> [Group] +%% Group = {GroupName,Properties,GroupsAndTestCases} +%% GroupName = atom() +%% Properties = [parallel | sequence | Shuffle | {RepeatType,N}] +%% GroupsAndTestCases = [Group | {group,GroupName} | TestCase] +%% TestCase = atom() +%% Shuffle = shuffle | {shuffle,{integer(),integer(),integer()}} +%% RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | +%% repeat_until_any_ok | repeat_until_any_fail +%% N = integer() | forever +%%-------------------------------------------------------------------- +groups() -> + []. + +%%-------------------------------------------------------------------- +%% Function: all() -> GroupsAndTestCases | {skip,Reason} +%% GroupsAndTestCases = [{group,GroupName} | TestCase] +%% GroupName = atom() +%% TestCase = atom() +%% Reason = term() +%%-------------------------------------------------------------------- +all() -> + [tc1]. + +tc1(_) -> + exit(should_never_run). diff --git a/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_11_SUITE.erl b/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_11_SUITE.erl new file mode 100644 index 0000000000..ce94533110 --- /dev/null +++ b/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_11_SUITE.erl @@ -0,0 +1,134 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(cfg_error_11_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% Function: suite() -> Info +%% Info = [tuple()] +%%-------------------------------------------------------------------- +suite() -> + [{timetrap,{seconds,2}}, + {require, dummy0}, {default_config, dummy0, "suite/0"}, + {require, dummy1}, {default_config, dummy1, "suite/0"}, + {require, dummy2}, {default_config, dummy2, "suite/0"}]. + + + +%%-------------------------------------------------------------------- +%% Function: init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_suite(Config0) -> void() | {save_config,Config1} +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_suite() -> + put('$test_server_framework_test', + fun(end_tc, _Default) -> {fail,fail_end_per_suite}; + (_, Default) -> Default + end), + [{require, dummy3}, {default_config, dummy3, "end_per_suite/0"}, + {timetrap,3000}]. + +end_per_suite(_Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Function: init_per_group(GroupName, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_group(_GroupName, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_group(GroupName, Config0) -> +%% void() | {save_config,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_group(_GroupName, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Function: init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_testcase(_, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_testcase(_TestCase, _Config) -> + done. + +%%-------------------------------------------------------------------- +%% Function: groups() -> [Group] +%% Group = {GroupName,Properties,GroupsAndTestCases} +%% GroupName = atom() +%% Properties = [parallel | sequence | Shuffle | {RepeatType,N}] +%% GroupsAndTestCases = [Group | {group,GroupName} | TestCase] +%% TestCase = atom() +%% Shuffle = shuffle | {shuffle,{integer(),integer(),integer()}} +%% RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | +%% repeat_until_any_ok | repeat_until_any_fail +%% N = integer() | forever +%%-------------------------------------------------------------------- +groups() -> + []. + +%%-------------------------------------------------------------------- +%% Function: all() -> GroupsAndTestCases | {skip,Reason} +%% GroupsAndTestCases = [{group,GroupName} | TestCase] +%% GroupName = atom() +%% TestCase = atom() +%% Reason = term() +%%-------------------------------------------------------------------- +all() -> + [tc1, tc2]. + +tc1() -> + [{require, dummy0}, {default_config, dummy0, "tc1"}]. + +tc1(_) -> + dummy. + +tc2() -> + [{timetrap,1}]. + +tc2(_) -> + dummy. diff --git a/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_12_SUITE.erl b/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_12_SUITE.erl new file mode 100644 index 0000000000..806d3caf72 --- /dev/null +++ b/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_12_SUITE.erl @@ -0,0 +1,88 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(cfg_error_12_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +init_per_testcase(_, Config) -> + Config. + +end_per_testcase(tc2, _Config) -> + timer:sleep(2000), + exit(this_should_not_be_printed); +end_per_testcase(tc4, _Config) -> + timer:sleep(2000), + exit(this_should_not_be_printed); +end_per_testcase(_, _) -> + ok. + +all() -> + [tc1, tc2, tc3, tc4]. + +%%%----------------------------------------------------------------- +tc1() -> + put('$test_server_framework_test', + fun(init_tc, _Default) -> + ct:pal("init_tc(~p): Night time...",[self()]), + timer:sleep(2000), + ct:pal("init_tc(~p): Day time!",[self()]), + exit(this_should_not_be_printed); + (_, Default) -> Default + end), + [{timetrap,500}]. + +tc1(_) -> + exit(this_should_not_be_printed). + +%%%----------------------------------------------------------------- +tc2() -> + [{timetrap,500}]. + +tc2(_) -> + ok. + +%%%----------------------------------------------------------------- +tc3() -> + [{timetrap,500}]. + +tc3(_) -> + put('$test_server_framework_test', + fun(end_tc, _Default) -> + ct:pal("end_tc(~p): Night time...",[self()]), + timer:sleep(1000), + ct:pal("end_tc(~p): Day time!",[self()]); + (_, Default) -> Default + end), + {comment,"should succeed since ct_fw cancels timetrap in end_tc"}. + +%%%----------------------------------------------------------------- +tc4() -> + put('$test_server_framework_test', + fun(end_tc, _Default) -> + ct:pal("end_tc(~p): Night time...",[self()]), + timer:sleep(1000), + ct:pal("end_tc(~p): Day time!",[self()]); + (_, Default) -> Default + end), + [{timetrap,500}]. + +tc4(_) -> + ok. diff --git a/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_13_SUITE.erl b/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_13_SUITE.erl new file mode 100644 index 0000000000..c8a3c1d15e --- /dev/null +++ b/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_13_SUITE.erl @@ -0,0 +1,47 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(cfg_error_13_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +init_per_suite() -> + put('$test_server_framework_test', + fun(end_tc, _Default) -> + ct:pal("end_tc(~p): Night time...",[self()]), + timer:sleep(1000), + ct:pal("end_tc(~p): Day time!",[self()]); + (_, Default) -> Default + end), + [{timetrap,500}]. + +init_per_suite(Config) -> + ct:comment("should succeed since ct_fw cancels timetrap in end_tc"), + Config. + +end_per_suite(_) -> + ok. + +all() -> + [tc1]. + +%%%----------------------------------------------------------------- +tc1(_) -> + dummy. diff --git a/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_14_SUITE.erl b/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_14_SUITE.erl new file mode 100644 index 0000000000..960d0f61b0 --- /dev/null +++ b/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_14_SUITE.erl @@ -0,0 +1,46 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(cfg_error_14_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +init_per_suite(Config) -> + Config. + +end_per_suite() -> + put('$test_server_framework_test', + fun(end_tc, _Default) -> + ct:pal("end_tc(~p): Night time...",[self()]), + timer:sleep(1000), + ct:pal("end_tc(~p): Day time!",[self()]); + (_, Default) -> Default + end), + [{timetrap,500}]. + +end_per_suite(_Config) -> + {comment,"should succeed since ct_fw cancels timetrap in end_tc"}. + +all() -> + [tc1]. + +%%%----------------------------------------------------------------- +tc1(_) -> + dummy. diff --git a/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_3_SUITE.erl b/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_3_SUITE.erl index bf01bb52d9..08c57887ef 100644 --- a/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_3_SUITE.erl +++ b/lib/common_test/test/ct_error_SUITE_data/error/test/cfg_error_3_SUITE.erl @@ -37,7 +37,8 @@ suite() -> %%-------------------------------------------------------------------- init_per_suite(Config) -> timer:sleep(5000), - Config. + exit(shouldnt_happen). +% Config. %%-------------------------------------------------------------------- %% Function: end_per_suite(Config0) -> void() | {save_config,Config1} diff --git a/lib/common_test/test/ct_error_SUITE_data/error/test/timetrap_1_SUITE.erl b/lib/common_test/test/ct_error_SUITE_data/error/test/timetrap_1_SUITE.erl new file mode 100644 index 0000000000..cb3109349b --- /dev/null +++ b/lib/common_test/test/ct_error_SUITE_data/error/test/timetrap_1_SUITE.erl @@ -0,0 +1,194 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(timetrap_1_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% Function: suite() -> Info +%% Info = [tuple()] +%%-------------------------------------------------------------------- +suite() -> + [{timetrap,{seconds,1}}]. + +%%-------------------------------------------------------------------- +%% Function: init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + TabPid = spawn(fun() -> + ets:new(?MODULE, [named_table, set, public]), + ets:insert(?MODULE, {last_case,ok}), + receive _ -> ok end + end), + [{tab,TabPid} | Config]. + +%%-------------------------------------------------------------------- +%% Function: end_per_suite(Config0) -> void() | {save_config,Config1} +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_suite(Config) -> + exit(?config(tab, Config), kill), + ok. + +%%-------------------------------------------------------------------- +%% Function: init_per_group(GroupName, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_group(_GroupName, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_group(GroupName, Config0) -> +%% void() | {save_config,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_group(_GroupName, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Function: init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_testcase(TC, Config) -> + {_,_} = process_info(?config(tab, Config), priority), + [{_,ok}] = ets:lookup(?MODULE, last_case), + ets:insert(?MODULE, {last_case,fail}), + init_per_testcase1(TC, Config). + +init_per_testcase1(tc1, Config) -> + [{tc,tc1}|Config]; + +init_per_testcase1(tc2, Config) -> + [{tc,tc2}|Config]; + +init_per_testcase1(tc3, Config) -> + [{tc,tc3}|Config]; + +init_per_testcase1(tc4, Config) -> + [{tc,tc4},{default_timeout,5000}|Config]; + +init_per_testcase1(tc5, Config) -> + [{tc,tc5}|Config]; + +init_per_testcase1(tc6, Config) -> + [{tc,tc6}|Config]. + +%%-------------------------------------------------------------------- +%% Function: end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_testcase(TC, Config) -> + {_,_} = process_info(?config(tab, Config), priority), + [{_,fail}] = ets:lookup(?MODULE, last_case), + ets:insert(?MODULE, {last_case,ok}), + end_per_testcase1(TC, Config). + +end_per_testcase1(tc1, Config) -> + ct:pal("end_per_testcase(tc1): ~p", [Config]), + tc1 = ?config(tc, Config), + {failed,timetrap_timeout} = ?config(tc_status, Config), + ok; + +end_per_testcase1(tc2, Config) -> + ct:pal("end_per_testcase(tc2): ~p", [Config]), + tc2 = ?config(tc, Config), + {failed,timetrap_timeout} = ?config(tc_status, Config), + timer:sleep(2000); + +end_per_testcase1(tc3, Config) -> + ct:pal("end_per_testcase(tc3): ~p", [Config]), + tc3 = ?config(tc, Config), + {failed,{testcase_aborted,testing_end_conf}} = ?config(tc_status, Config), + ok; + +end_per_testcase1(tc4, Config) -> + ct:pal("end_per_testcase(tc4): ~p", [Config]), + tc4 = ?config(tc, Config), + {failed,{testcase_aborted,testing_end_conf}} = ?config(tc_status, Config), + timer:sleep(2000); + +end_per_testcase1(tc5, Config) -> + ct:pal("end_per_testcase(tc5): ~p", [Config]), + tc5 = ?config(tc, Config), + exit(end_per_tc_fail_after_timeout); + +end_per_testcase1(tc6, Config) -> + ct:pal("end_per_testcase(tc6): ~p", [Config]), + tc6 = ?config(tc, Config), + exit(end_per_tc_fail_after_abort). + +%%-------------------------------------------------------------------- +%% Function: groups() -> [Group] +%% Group = {GroupName,Properties,GroupsAndTestCases} +%% GroupName = atom() +%% Properties = [parallel | sequence | Shuffle | {RepeatType,N}] +%% GroupsAndTestCases = [Group | {group,GroupName} | TestCase] +%% TestCase = atom() +%% Shuffle = shuffle | {shuffle,{integer(),integer(),integer()}} +%% RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | +%% repeat_until_any_ok | repeat_until_any_fail +%% N = integer() | forever +%%-------------------------------------------------------------------- +groups() -> + []. + +%%-------------------------------------------------------------------- +%% Function: all() -> GroupsAndTestCases | {skip,Reason} +%% GroupsAndTestCases = [{group,GroupName} | TestCase] +%% GroupName = atom() +%% TestCase = atom() +%% Reason = term() +%%-------------------------------------------------------------------- +all() -> + [tc1, tc2, tc3, tc4, tc5, tc6]. + +tc1(_) -> + timer:sleep(2000). + +tc2(_) -> + timer:sleep(2000). + +tc3(_) -> + spawn(ct, abort_current_testcase, [testing_end_conf]), + timer:sleep(2000). + +tc4(_) -> + spawn(ct, abort_current_testcase, [testing_end_conf]), + timer:sleep(2000). + +tc5(_) -> + timer:sleep(2000). + +tc6(_) -> + spawn(ct, abort_current_testcase, [testing_end_conf]), + timer:sleep(2000). diff --git a/lib/common_test/test/ct_error_SUITE_data/error/test/timetrap_2_SUITE.erl b/lib/common_test/test/ct_error_SUITE_data/error/test/timetrap_2_SUITE.erl new file mode 100644 index 0000000000..99bb400137 --- /dev/null +++ b/lib/common_test/test/ct_error_SUITE_data/error/test/timetrap_2_SUITE.erl @@ -0,0 +1,138 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(timetrap_2_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% Function: suite() -> Info +%% Info = [tuple()] +%%-------------------------------------------------------------------- +suite() -> + [{timetrap,{seconds,3}}, + {require,multiply}, + {require,scale}]. + +%%-------------------------------------------------------------------- +%% Function: init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_suite(Config0) -> void() | {save_config,Config1} +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_suite(_Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Function: init_per_group(GroupName, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_group(_GroupName, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_group(GroupName, Config0) -> +%% void() | {save_config,Config1} +%% GroupName = atom() +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_group(_GroupName, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Function: init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%% Reason = term() +%%-------------------------------------------------------------------- +init_per_testcase(tc1, Config) -> + ct:timetrap({seconds,1}), + Config; + +init_per_testcase(tc3, Config) -> + ct:timetrap({seconds,1}), + Config; + +init_per_testcase(_TestCase, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} +%% TestCase = atom() +%% Config0 = Config1 = [tuple()] +%%-------------------------------------------------------------------- +end_per_testcase(_, Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Function: groups() -> [Group] +%% Group = {GroupName,Properties,GroupsAndTestCases} +%% GroupName = atom() +%% Properties = [parallel | sequence | Shuffle | {RepeatType,N}] +%% GroupsAndTestCases = [Group | {group,GroupName} | TestCase] +%% TestCase = atom() +%% Shuffle = shuffle | {shuffle,{integer(),integer(),integer()}} +%% RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | +%% repeat_until_any_ok | repeat_until_any_fail +%% N = integer() | forever +%%-------------------------------------------------------------------- +groups() -> + []. + +%%-------------------------------------------------------------------- +%% Function: all() -> GroupsAndTestCases | {skip,Reason} +%% GroupsAndTestCases = [{group,GroupName} | TestCase] +%% GroupName = atom() +%% TestCase = atom() +%% Reason = term() +%%-------------------------------------------------------------------- +all() -> + [tc0,tc1,tc2]. + +tc0(_) -> + N = list_to_integer(ct:get_config(multiply)), + ct:comment(io_lib:format("TO after ~w sec", [3*N])), + ct:sleep({seconds,5}), + ok. + +tc1(_) -> + N =list_to_integer( ct:get_config(multiply)), + ct:comment(io_lib:format("TO after ~w sec", [1*N])), + ct:sleep({seconds,5}), + ok. + +tc2(_) -> + N = list_to_integer(ct:get_config(multiply)), + ct:comment(io_lib:format("TO after ~w sec", [0.5*N])), + ct:timetrap(500), + ct:sleep(2000), + ok. diff --git a/lib/common_test/test/ct_event_handler_SUITE.erl b/lib/common_test/test/ct_event_handler_SUITE.erl index bafd32f937..00a4c4ded3 100644 --- a/lib/common_test/test/ct_event_handler_SUITE.erl +++ b/lib/common_test/test/ct_event_handler_SUITE.erl @@ -88,7 +88,7 @@ start_stop(Config) when is_list(Config) -> ERPid = ct_test_support:start_event_receiver(Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), @@ -110,8 +110,7 @@ start_stop(Config) when is_list(Config) -> {eh_A,test_done,{'DEF','STOP_TIME'}}, {eh_A,stop_logging,[]}], - ok = ct_test_support:verify_events(TestEvents, Events, Config), - {comment,"NOTE! Known problem with test_start event!"}. + ok = ct_test_support:verify_events(TestEvents++TestEvents, Events, Config). results(doc) -> @@ -135,7 +134,7 @@ results(Config) when is_list(Config) -> ERPid = ct_test_support:start_event_receiver(Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), @@ -163,7 +162,7 @@ results(Config) when is_list(Config) -> {eh_A,test_done,{'DEF','STOP_TIME'}}, {eh_A,stop_logging,[]}], - ok = ct_test_support:verify_events(TestEvents, Events, Config). + ok = ct_test_support:verify_events(TestEvents++TestEvents, Events, Config). %%%----------------------------------------------------------------- diff --git a/lib/common_test/test/ct_event_handler_SUITE_data/eh_A.erl b/lib/common_test/test/ct_event_handler_SUITE_data/eh_A.erl index 6e526f15a2..54cf3a22e7 100644 --- a/lib/common_test/test/ct_event_handler_SUITE_data/eh_A.erl +++ b/lib/common_test/test/ct_event_handler_SUITE_data/eh_A.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2008-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -44,6 +44,19 @@ %% Description: Whenever a new event handler is added to an event manager, %% this function is called to initialize the event handler. %%-------------------------------------------------------------------- +init(String = [X|_]) when is_integer(X) -> + case erl_scan:string(String++".") of + {ok,Ts,_} -> + case erl_parse:parse_term(Ts) of + {ok,Args} -> + init(Args); + _ -> + init(String) + end; + _ -> + init(String) + end; + init(Args) -> S1 = case lists:keysearch(cbm, 1, Args) of diff --git a/lib/common_test/test/ct_groups_test_1_SUITE.erl b/lib/common_test/test/ct_groups_test_1_SUITE.erl index 1761b773f5..64d61fc104 100644 --- a/lib/common_test/test/ct_groups_test_1_SUITE.erl +++ b/lib/common_test/test/ct_groups_test_1_SUITE.erl @@ -76,14 +76,14 @@ groups_suite_1(Config) when is_list(Config) -> Suite = filename:join(DataDir, "groups_1/test/groups_11_SUITE"), {Opts,ERPid} = setup({suite,Suite}, Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), ct_test_support:log_events(groups_suite_1, reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(groups_suite_1), + TestEvents = events_to_check(groups_suite_1), ok = ct_test_support:verify_events(TestEvents, Events, Config). @@ -96,14 +96,14 @@ groups_suite_2(Config) when is_list(Config) -> Suite = filename:join(DataDir, "groups_1/test/groups_12_SUITE"), {Opts,ERPid} = setup({suite,Suite}, Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), ct_test_support:log_events(groups_suite_2, reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(groups_suite_2), + TestEvents = events_to_check(groups_suite_2), ok = ct_test_support:verify_events(TestEvents, Events, Config). @@ -117,14 +117,14 @@ groups_suites_1(Config) when is_list(Config) -> filename:join(DataDir, "groups_1/test/groups_12_SUITE")], {Opts,ERPid} = setup({suite,Suites}, Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), ct_test_support:log_events(groups_suites_1, reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(groups_suites_1), + TestEvents = events_to_check(groups_suites_1), ok = ct_test_support:verify_events(TestEvents, Events, Config). @@ -137,14 +137,14 @@ groups_dir_1(Config) when is_list(Config) -> Dir = filename:join(DataDir, "groups_1"), {Opts,ERPid} = setup({dir,Dir}, Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), ct_test_support:log_events(groups_dir_1, reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(groups_dir_1), + TestEvents = events_to_check(groups_dir_1), ok = ct_test_support:verify_events(TestEvents, Events, Config). %%%----------------------------------------------------------------- @@ -157,14 +157,14 @@ groups_dirs_1(Config) when is_list(Config) -> filename:join(DataDir, "groups_2")], {Opts,ERPid} = setup({dir,Dirs}, Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), ct_test_support:log_events(groups_dirs_1, reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(groups_dirs_1), + TestEvents = events_to_check(groups_dirs_1), ok = ct_test_support:verify_events(TestEvents, Events, Config). @@ -188,6 +188,14 @@ reformat(Events, EH) -> %%%----------------------------------------------------------------- %%% TEST EVENTS %%%----------------------------------------------------------------- +events_to_check(Test) -> + %% 2 tests (ct:run_test + script_start) is default + events_to_check(Test, 2). + +events_to_check(_, 0) -> + []; +events_to_check(Test, N) -> + test_events(Test) ++ events_to_check(Test, N-1). test_events(groups_suite_1) -> [{?eh,start_logging,{'DEF','RUNDIR'}}, @@ -327,14 +335,14 @@ test_events(groups_suite_2) -> {?eh,tc_start,{groups_12_SUITE,testcase_2a}}, {?eh,tc_done,{groups_12_SUITE,testcase_2a,ok}}, - [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,1}]}}}, - {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,1}]},ok}}, + [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,2}]}}}, + {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,2}]},ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_3a}}, {?eh,tc_done,{groups_12_SUITE,testcase_3a,ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_3b}}, {?eh,tc_done,{groups_12_SUITE,testcase_3b,ok}}, - {?eh,tc_start,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,1}]}}}, - {?eh,tc_done,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,1}]},ok}}], + {?eh,tc_start,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,2}]}}}, + {?eh,tc_done,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,2}]},ok}}], [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_3,[]}}}, {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_3,[]},ok}}, @@ -361,12 +369,8 @@ test_events(groups_suite_2) -> {parallel,[{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_5,[parallel]}}}, {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_5,[parallel]},ok}}, - - %% the done event could come in during the parallel subgroup - %% and we can't test that, yet... - %% {?eh,tc_start,{groups_12_SUITE,testcase_5a}}, - %% {?eh,tc_done,{groups_12_SUITE,testcase_5a,ok}}, - + {?eh,tc_start,{groups_12_SUITE,testcase_5a}}, + {?eh,tc_done,{groups_12_SUITE,testcase_5a,ok}}, {parallel,[{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_6,[parallel]}}}, {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_6,[parallel]},ok}}, @@ -525,14 +529,14 @@ test_events(groups_suites_1) -> {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_2,[parallel]},ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_2a}}, {?eh,tc_done,{groups_12_SUITE,testcase_2a,ok}}, - [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,1}]}}}, - {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,1}]},ok}}, + [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,2}]}}}, + {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,2}]},ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_3a}}, {?eh,tc_done,{groups_12_SUITE,testcase_3a,ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_3b}}, {?eh,tc_done,{groups_12_SUITE,testcase_3b,ok}}, - {?eh,tc_start,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,1}]}}}, - {?eh,tc_done,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,1}]},ok}}], + {?eh,tc_start,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,2}]}}}, + {?eh,tc_done,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,2}]},ok}}], [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_3,[]}}}, {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_3,[]},ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_3a}}, @@ -555,12 +559,8 @@ test_events(groups_suites_1) -> {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_4,[]},ok}}, {parallel,[{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_5,[parallel]}}}, {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_5,[parallel]},ok}}, - - %% the done event could come in during the parallel subgroup - %% and we can't test that, yet... - %% {?eh,tc_start,{groups_12_SUITE,testcase_5a}}, - %% {?eh,tc_done,{groups_12_SUITE,testcase_5a,ok}}, - + {?eh,tc_start,{groups_12_SUITE,testcase_5a}}, + {?eh,tc_done,{groups_12_SUITE,testcase_5a,ok}}, {parallel,[{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_6,[parallel]}}}, {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_6,[parallel]},ok}}, [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_7,[sequence]}}}, @@ -715,14 +715,14 @@ test_events(groups_dir_1) -> {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_2,[parallel]},ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_2a}}, {?eh,tc_done,{groups_12_SUITE,testcase_2a,ok}}, - [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,1}]}}}, - {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,1}]},ok}}, + [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,2}]}}}, + {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,2}]},ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_3a}}, {?eh,tc_done,{groups_12_SUITE,testcase_3a,ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_3b}}, {?eh,tc_done,{groups_12_SUITE,testcase_3b,ok}}, - {?eh,tc_start,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,1}]}}}, - {?eh,tc_done,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,1}]},ok}}], + {?eh,tc_start,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,2}]}}}, + {?eh,tc_done,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,2}]},ok}}], [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_3,[]}}}, {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_3,[]},ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_3a}}, @@ -745,12 +745,8 @@ test_events(groups_dir_1) -> {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_4,[]},ok}}, {parallel,[{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_5,[parallel]}}}, {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_5,[parallel]},ok}}, - - %% the done event could come in during the parallel subgroup - %% and we can't test that, yet... - %% {?eh,tc_start,{groups_12_SUITE,testcase_5a}}, - %% {?eh,tc_done,{groups_12_SUITE,testcase_5a,ok}}, - + {?eh,tc_start,{groups_12_SUITE,testcase_5a}}, + {?eh,tc_done,{groups_12_SUITE,testcase_5a,ok}}, {parallel,[{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_6,[parallel]}}}, {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_6,[parallel]},ok}}, [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_7,[sequence]}}}, @@ -906,14 +902,14 @@ test_events(groups_dirs_1) -> {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_2,[parallel]},ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_2a}}, {?eh,tc_done,{groups_12_SUITE,testcase_2a,ok}}, - [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,1}]}}}, - {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,1}]},ok}}, + [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,2}]}}}, + {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_3,[{repeat,2}]},ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_3a}}, {?eh,tc_done,{groups_12_SUITE,testcase_3a,ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_3b}}, {?eh,tc_done,{groups_12_SUITE,testcase_3b,ok}}, - {?eh,tc_start,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,1}]}}}, - {?eh,tc_done,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,1}]},ok}}], + {?eh,tc_start,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,2}]}}}, + {?eh,tc_done,{groups_12_SUITE,{end_per_group,test_group_3,[{repeat,2}]},ok}}], [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_3,[]}}}, {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_3,[]},ok}}, {?eh,tc_start,{groups_12_SUITE,testcase_3a}}, @@ -936,12 +932,8 @@ test_events(groups_dirs_1) -> {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_4,[]},ok}}, {parallel,[{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_5,[parallel]}}}, {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_5,[parallel]},ok}}, - - %% the done event could come in during the parallel subgroup - %% and we can't test that, yet... - %% {?eh,tc_start,{groups_12_SUITE,testcase_5a}}, - %% {?eh,tc_done,{groups_12_SUITE,testcase_5a,ok}}, - + {?eh,tc_start,{groups_12_SUITE,testcase_5a}}, + {?eh,tc_done,{groups_12_SUITE,testcase_5a,ok}}, {parallel,[{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_6,[parallel]}}}, {?eh,tc_done,{groups_12_SUITE,{init_per_group,test_group_6,[parallel]},ok}}, [{?eh,tc_start,{groups_12_SUITE,{init_per_group,test_group_7,[sequence]}}}, @@ -1138,17 +1130,17 @@ test_events(groups_dirs_1) -> {?eh,tc_start,{groups_22_SUITE,testcase_2a}}, {?eh,tc_done,{groups_22_SUITE,testcase_2a,ok}}, [{?eh,tc_start, - {groups_22_SUITE,{init_per_group,test_group_3,[{repeat,1}]}}}, + {groups_22_SUITE,{init_per_group,test_group_3,[{repeat,2}]}}}, {?eh,tc_done, - {groups_22_SUITE,{init_per_group,test_group_3,[{repeat,1}]},ok}}, + {groups_22_SUITE,{init_per_group,test_group_3,[{repeat,2}]},ok}}, {?eh,tc_start,{groups_22_SUITE,testcase_3a}}, {?eh,tc_done,{groups_22_SUITE,testcase_3a,ok}}, {?eh,tc_start,{groups_22_SUITE,testcase_3b}}, {?eh,tc_done,{groups_22_SUITE,testcase_3b,ok}}, {?eh,tc_start, - {groups_22_SUITE,{end_per_group,test_group_3,[{repeat,1}]}}}, + {groups_22_SUITE,{end_per_group,test_group_3,[{repeat,2}]}}}, {?eh,tc_done, - {groups_22_SUITE,{end_per_group,test_group_3,[{repeat,1}]},ok}}], + {groups_22_SUITE,{end_per_group,test_group_3,[{repeat,2}]},ok}}], [{?eh,tc_start, {groups_22_SUITE,{init_per_group,test_group_3,[]}}}, {?eh,tc_done, @@ -1181,12 +1173,8 @@ test_events(groups_dirs_1) -> {groups_22_SUITE,{init_per_group,test_group_5,[parallel]}}}, {?eh,tc_done, {groups_22_SUITE,{init_per_group,test_group_5,[parallel]},ok}}, - - %% the done event could come in during the parallel subgroup - %% and we can't test that, yet... - %% {?eh,tc_start,{groups_22_SUITE,testcase_5a}}, - %% {?eh,tc_done,{groups_22_SUITE,testcase_5a,ok}}, - + {?eh,tc_start,{groups_22_SUITE,testcase_5a}}, + {?eh,tc_done,{groups_22_SUITE,testcase_5a,ok}}, {parallel, [{?eh,tc_start, {groups_22_SUITE,{init_per_group,test_group_6,[parallel]}}}, diff --git a/lib/common_test/test/ct_groups_test_1_SUITE_data/groups_1/test/groups_11_SUITE.erl b/lib/common_test/test/ct_groups_test_1_SUITE_data/groups_1/test/groups_11_SUITE.erl index c6d50443d0..c69400e938 100644 --- a/lib/common_test/test/ct_groups_test_1_SUITE_data/groups_1/test/groups_11_SUITE.erl +++ b/lib/common_test/test/ct_groups_test_1_SUITE_data/groups_1/test/groups_11_SUITE.erl @@ -99,6 +99,7 @@ init_per_group(Group, Config) -> {Grs,_} = grs_and_tcs(), case lists:member(Group, Grs) of true -> + ct:comment(Group), init = ?config(suite,Config), [{Group,Group} | Config]; false -> @@ -109,6 +110,7 @@ end_per_group(Group, Config) -> {Grs,_} = grs_and_tcs(), case lists:member(Group, Grs) of true -> + ct:comment(Group), init = ?config(suite,Config), Group = ?config(Group,Config), ok; diff --git a/lib/common_test/test/ct_groups_test_1_SUITE_data/groups_1/test/groups_12_SUITE.erl b/lib/common_test/test/ct_groups_test_1_SUITE_data/groups_1/test/groups_12_SUITE.erl index b261ef581f..ec90ef95d1 100644 --- a/lib/common_test/test/ct_groups_test_1_SUITE_data/groups_1/test/groups_12_SUITE.erl +++ b/lib/common_test/test/ct_groups_test_1_SUITE_data/groups_1/test/groups_12_SUITE.erl @@ -37,7 +37,7 @@ groups() -> {test_group_2, [parallel], [testcase_2a, - {test_group_3, [{repeat,1}], + {test_group_3, [{repeat,2}], [testcase_3a, testcase_3b]}, testcase_2b]}, @@ -102,8 +102,8 @@ init_per_group(Group, Config) -> io_lib:format("shuffled, ~w", [S]); {test_group_1b,[{name,test_group_1b},parallel]} -> "parallel"; {test_group_2,[{name,test_group_2},parallel]} -> "parallel"; - {test_group_3,[{name,test_group_3},{repeat,1}]} -> "repeat 1"; - {test_group_3,[{name,test_group_3}]} -> "repeat 0"; + {test_group_3,[{name,test_group_3},{repeat,2}]} -> "repeat 2"; + {test_group_3,[{name,test_group_3}]} -> "repeat 1"; {test_group_4,[{name,test_group_4}]} -> ok; {test_group_5,[{name,test_group_5},parallel]} -> "parallel"; {test_group_6,[{name,test_group_6},parallel]} -> "parallel"; @@ -275,6 +275,10 @@ testcase_5a(Config) -> test_group_5 = ?config(test_group_5,Config), undefined = ?config(testcase_3,Config), testcase_5a = ?config(testcase_5a,Config), + %% increase chance the done event will come + %% during execution of subgroup (could be + %% tricky to handle) + timer:sleep(3), ok. testcase_5b() -> []. diff --git a/lib/common_test/test/ct_groups_test_1_SUITE_data/groups_2/test/groups_22_SUITE.erl b/lib/common_test/test/ct_groups_test_1_SUITE_data/groups_2/test/groups_22_SUITE.erl index 2e19cf6310..ec0adc5df0 100644 --- a/lib/common_test/test/ct_groups_test_1_SUITE_data/groups_2/test/groups_22_SUITE.erl +++ b/lib/common_test/test/ct_groups_test_1_SUITE_data/groups_2/test/groups_22_SUITE.erl @@ -37,7 +37,7 @@ groups() -> {test_group_2, [parallel], [testcase_2a, - {test_group_3, [{repeat,1}], + {test_group_3, [{repeat,2}], [testcase_3a, testcase_3b]}, testcase_2b]}, @@ -102,8 +102,8 @@ init_per_group(Group, Config) -> io_lib:format("shuffled, ~w", [S]); {test_group_1b,[{name,test_group_1b},parallel]} -> "parallel"; {test_group_2,[{name,test_group_2},parallel]} -> "parallel"; - {test_group_3,[{name,test_group_3},{repeat,1}]} -> "repeat 1"; - {test_group_3,[{name,test_group_3}]} -> "repeat 0"; + {test_group_3,[{name,test_group_3},{repeat,2}]} -> "repeat 2"; + {test_group_3,[{name,test_group_3}]} -> "repeat 1"; {test_group_4,[{name,test_group_4}]} -> ok; {test_group_5,[{name,test_group_5},parallel]} -> "parallel"; {test_group_6,[{name,test_group_6},parallel]} -> "parallel"; diff --git a/lib/common_test/test/ct_groups_test_2_SUITE.erl b/lib/common_test/test/ct_groups_test_2_SUITE.erl index 5a60d855b7..c4371501b3 100644 --- a/lib/common_test/test/ct_groups_test_2_SUITE.erl +++ b/lib/common_test/test/ct_groups_test_2_SUITE.erl @@ -60,7 +60,7 @@ all(doc) -> ["Run smoke tests of Common Test."]; all(suite) -> - [missing_conf]. + [missing_conf, repeat_1]. %%-------------------------------------------------------------------- %% TEST CASES @@ -75,16 +75,35 @@ missing_conf(Config) when is_list(Config) -> Suite = filename:join(DataDir, "groups_1/missing_conf_SUITE"), {Opts,ERPid} = setup({suite,Suite}, Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), ct_test_support:log_events(missing_conf_SUITE, reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(missing_conf), + TestEvents = events_to_check(missing_conf), ok = ct_test_support:verify_events(TestEvents, Events, Config). - + +%%%----------------------------------------------------------------- +%%% + +repeat_1(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + Suite = filename:join(DataDir, "groups_1/repeat_1_SUITE"), + + {Opts,ERPid} = setup({suite,Suite}, Config), + ok = ct_test_support:run(Opts, Config), + Events = ct_test_support:get_events(ERPid, Config), + + ct_test_support:log_events(repeat_1, + reformat(Events, ?eh), + ?config(priv_dir, Config)), + + TestEvents = events_to_check(repeat_1), + ok = ct_test_support:verify_events(TestEvents, Events, Config). + %%%----------------------------------------------------------------- %%% HELP FUNCTIONS %%%----------------------------------------------------------------- @@ -105,6 +124,127 @@ reformat(Events, EH) -> %%%----------------------------------------------------------------- %%% TEST EVENTS %%%----------------------------------------------------------------- +events_to_check(Test) -> + %% 2 tests (ct:run_test + script_start) is default + events_to_check(Test, 2). + +events_to_check(_, 0) -> + []; +events_to_check(Test, N) -> + test_events(Test) ++ events_to_check(Test, N-1). test_events(missing_conf) -> - exit(must_handle_this). + [ + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{1,1,2}}, + {?eh,tc_start,{ct_framework,ct_init_per_group}}, + {?eh,tc_done,{ct_framework,ct_init_per_group,ok}}, + {?eh,test_stats,{1,0,{0,0}}}, + {?eh,tc_start,{missing_conf_SUITE,tc1}}, + {?eh,tc_done,{missing_conf_SUITE,tc1,ok}}, + {?eh,test_stats,{2,0,{0,0}}}, + {?eh,tc_start,{missing_conf_SUITE,tc2}}, + {?eh,tc_done,{missing_conf_SUITE,tc2,ok}}, + {?eh,test_stats,{3,0,{0,0}}}, + {?eh,tc_start,{ct_framework,ct_end_per_group}}, + {?eh,tc_done,{ct_framework,ct_end_per_group,ok}}, + {?eh,test_stats,{4,0,{0,0}}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]} + ]; + +test_events(repeat_1) -> + [ + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{1,1,unknown}}, + {?eh,tc_start,{repeat_1_SUITE,init_per_suite}}, + {?eh,tc_done,{repeat_1_SUITE,init_per_suite,ok}}, + [{?eh,tc_start, + {repeat_1_SUITE,{init_per_group,test_group_1,[{repeat,2}]}}}, + {?eh,tc_done, + {repeat_1_SUITE,{init_per_group,test_group_1,[{repeat,2}]},ok}}, + {?eh,tc_start,{repeat_1_SUITE,testcase_1a}}, + {?eh,tc_done,{repeat_1_SUITE,testcase_1a,ok}}, + {?eh,test_stats,{1,0,{0,0}}}, + {?eh,tc_start,{repeat_1_SUITE,testcase_1b}}, + {?eh,tc_done,{repeat_1_SUITE,testcase_1b,ok}}, + {?eh,test_stats,{2,0,{0,0}}}, + {?eh,tc_start, + {repeat_1_SUITE,{end_per_group,test_group_1,[{repeat,2}]}}}, + {?eh,tc_done, + {repeat_1_SUITE,{end_per_group,test_group_1,[{repeat,2}]},ok}}], + [{?eh,tc_start, + {repeat_1_SUITE,{init_per_group,test_group_1,[]}}}, + {?eh,tc_done, + {repeat_1_SUITE,{init_per_group,test_group_1,[]},ok}}, + {?eh,tc_start,{repeat_1_SUITE,testcase_1a}}, + {?eh,tc_done,{repeat_1_SUITE,testcase_1a,ok}}, + {?eh,test_stats,{3,0,{0,0}}}, + {?eh,tc_start,{repeat_1_SUITE,testcase_1b}}, + {?eh,tc_done,{repeat_1_SUITE,testcase_1b,ok}}, + {?eh,test_stats,{4,0,{0,0}}}, + {?eh,tc_start, + {repeat_1_SUITE,{end_per_group,test_group_1,[]}}}, + {?eh,tc_done, + {repeat_1_SUITE,{end_per_group,test_group_1,[]},ok}}], + [{?eh,tc_start, + {repeat_1_SUITE,{init_per_group,test_group_2,[]}}}, + {?eh,tc_done, + {repeat_1_SUITE,{init_per_group,test_group_2,[]},ok}}, + {?eh,tc_start,{repeat_1_SUITE,testcase_2a}}, + {?eh,tc_done,{repeat_1_SUITE,testcase_2a,ok}}, + {?eh,test_stats,{5,0,{0,0}}}, + {?eh,tc_start,{repeat_1_SUITE,testcase_2b}}, + {?eh,tc_done,{repeat_1_SUITE,testcase_2b,ok}}, + {?eh,test_stats,{6,0,{0,0}}}, + {?eh,tc_start, + {repeat_1_SUITE,{end_per_group,test_group_2,[]}}}, + {?eh,tc_done, + {repeat_1_SUITE,{end_per_group,test_group_2,[]},ok}}], + [{?eh,tc_start, + {repeat_1_SUITE, + {init_per_group,test_group_3,[]}}}, + {?eh,tc_done, + {repeat_1_SUITE, + {init_per_group,test_group_3,[]}, + ok}}, + {?eh,tc_start,{repeat_1_SUITE,testcase_3a}}, + {?eh,tc_done,{repeat_1_SUITE,testcase_3a,ok}}, + {?eh,test_stats,{7,0,{0,0}}}, + [{?eh,tc_start, + {repeat_1_SUITE, + {init_per_group,test_group_4,[]}}}, + {?eh,tc_done, + {repeat_1_SUITE, + {init_per_group,test_group_4,[]}, + ok}}, + {?eh,tc_start,{repeat_1_SUITE,testcase_4a}}, + {?eh,tc_done,{repeat_1_SUITE,testcase_4a,ok}}, + {?eh,test_stats,{8,0,{0,0}}}, + {?eh,tc_start,{repeat_1_SUITE,testcase_4b}}, + {?eh,tc_done,{repeat_1_SUITE,testcase_4b,ok}}, + {?eh,test_stats,{9,0,{0,0}}}, + {?eh,tc_start, + {repeat_1_SUITE, + {end_per_group,test_group_4,[]}}}, + {?eh,tc_done, + {repeat_1_SUITE, + {end_per_group,test_group_4,[]}, + ok}}], + {?eh,tc_start,{repeat_1_SUITE,testcase_3b}}, + {?eh,tc_done,{repeat_1_SUITE,testcase_3b,ok}}, + {?eh,test_stats,{10,0,{0,0}}}, + {?eh,tc_start, + {repeat_1_SUITE, + {end_per_group,test_group_3,[]}}}, + {?eh,tc_done, + {repeat_1_SUITE, + {end_per_group,test_group_3,[]}, + ok}}], + {?eh,tc_start,{repeat_1_SUITE,end_per_suite}}, + {?eh,tc_done,{repeat_1_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]} + ]. diff --git a/lib/common_test/test/ct_groups_test_2_SUITE_data/cfgs/groups_2.1.cfg b/lib/common_test/test/ct_groups_test_2_SUITE_data/cfgs/groups_2.1.cfg new file mode 100644 index 0000000000..4928505157 --- /dev/null +++ b/lib/common_test/test/ct_groups_test_2_SUITE_data/cfgs/groups_2.1.cfg @@ -0,0 +1 @@ +{dummy_key, "dummy_data"}. diff --git a/lib/common_test/test/ct_groups_test_2_SUITE_data/groups_1/repeat_1_SUITE.erl b/lib/common_test/test/ct_groups_test_2_SUITE_data/groups_1/repeat_1_SUITE.erl new file mode 100644 index 0000000000..b4b9b03ca5 --- /dev/null +++ b/lib/common_test/test/ct_groups_test_2_SUITE_data/groups_1/repeat_1_SUITE.erl @@ -0,0 +1,105 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(repeat_1_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%==================================================================== +%% COMMON TEST CALLBACK FUNCTIONS +%%==================================================================== + +suite() -> + [{timetrap,{minutes,1}}]. + +groups() -> + [ + {test_group_1, [{repeat,2}], [testcase_1a,testcase_1b]}, + {test_group_2, [{repeat,1}], [testcase_2a,testcase_2b]}, + + {test_group_3, [{repeat_until_all_fail,1}], + [testcase_3a, + {test_group_4, [{repeat_until_any_fail,1}], + [testcase_4a, testcase_4b]}, + testcase_3b]} + ]. + +all() -> + [ + {group, test_group_1}, + {group, test_group_2}, + {group, test_group_3} + ]. + +%%-------------------------------------------------------------------- +%% Suite Configuration +%%-------------------------------------------------------------------- + +init_per_suite(Config) -> + Config. + +end_per_suite(Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Group Configuration +%%-------------------------------------------------------------------- + +init_per_group(Group, Config) -> + Group = proplists:get_value(name,?config(tc_group_properties,Config)), + ct:comment(Group), + Config. + +end_per_group(_Group, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Testcase Configuration +%%-------------------------------------------------------------------- + +init_per_testcase(_TestCase, Config) -> + Config. + +end_per_testcase(_TestCase, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Testcases +%%-------------------------------------------------------------------- + +testcase_1a(_) -> + ok. +testcase_1b(_) -> + ok. + +testcase_2a(_) -> + ok. +testcase_2b(_) -> + ok. + +testcase_3a(_) -> + ok. +testcase_3b(_) -> + ok. + +testcase_4a(_) -> + ok. +testcase_4b(_) -> + ok. diff --git a/lib/common_test/test/ct_groups_test_2_SUITE_data/groups_2/groups_21_SUITE.erl b/lib/common_test/test/ct_groups_test_2_SUITE_data/groups_2/groups_21_SUITE.erl new file mode 100644 index 0000000000..2533ac8e84 --- /dev/null +++ b/lib/common_test/test/ct_groups_test_2_SUITE_data/groups_2/groups_21_SUITE.erl @@ -0,0 +1,281 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(groups_21_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%==================================================================== +%% COMMON TEST CALLBACK FUNCTIONS +%%==================================================================== + +suite() -> + [{timetrap,{minutes,1}}]. + +groups() -> + [ + {test_group_1a, [testcase_1a,testcase_1b]}, + + {test_group_1b, [], [testcase_1a,testcase_1b]}, + + {test_group_2, [], [testcase_2a, + + {test_group_3, [], [testcase_3a, + testcase_3b]}, + testcase_2b]}, + + {test_group_4, [{test_group_5, [], [testcase_5a, + + {group, test_group_6}, + + testcase_5b]}]}, + + {test_group_6, [{group, test_group_7}]}, + + {test_group_7, [testcase_7a,testcase_7b]} + ]. + +all() -> + [testcase_1, + {group, test_group_1a}, + {group, test_group_1b}, + testcase_2, + {group, test_group_2}, + testcase_3, + {group, test_group_4}]. + +%% this func only for internal test purposes +grs_and_tcs() -> + {[ + test_group_1a, test_group_1b, + test_group_2, test_group_3, + test_group_4, test_group_5, + test_group_6, test_group_7 + ], + [ + testcase_1, + testcase_1a, testcase_1b, + testcase_2, + testcase_2a, testcase_2b, + testcase_3a, testcase_3b, + testcase_3, + testcase_5a, testcase_5b, + testcase_7a, testcase_7b + ]}. + +%%-------------------------------------------------------------------- +%% Suite Configuration +%%-------------------------------------------------------------------- + +init_per_suite(Config) -> + [{suite,init}|Config]. + +end_per_suite(Config) -> + init = ?config(suite,Config). + +%%-------------------------------------------------------------------- +%% Group Configuration +%%-------------------------------------------------------------------- + +init_per_group(Group, Config) -> + [{name,Group}] = ?config(tc_group_properties,Config), + {Grs,_} = grs_and_tcs(), + case lists:member(Group, Grs) of + true -> + ct:comment(io_lib:format("~w", [Group])), + init = ?config(suite,Config), + [{Group,Group} | Config]; + false -> + ct:fail({bad_group,Group}) + end. + +end_per_group(Group, Config) -> + {Grs,_} = grs_and_tcs(), + case lists:member(Group, Grs) of + true -> + init = ?config(suite,Config), + Group = ?config(Group,Config), + ok; + false -> + ct:fail({bad_group,Group}) + end. + +%%-------------------------------------------------------------------- +%% Testcase Configuration +%%-------------------------------------------------------------------- + +init_per_testcase(TestCase, Config) -> + {_,TCs} = grs_and_tcs(), + case lists:member(TestCase, TCs) of + true -> + init = ?config(suite,Config), + [{TestCase,TestCase} | Config]; + false -> + ct:fail({unknown_testcase,TestCase}) + end. + +end_per_testcase(TestCase, Config) -> + {_,TCs} = grs_and_tcs(), + case lists:member(TestCase, TCs) of + true -> + init = ?config(suite,Config), + TestCase = ?config(TestCase,Config), + ok; + false -> + ct:fail({unknown_testcase,TestCase}) + end. + + +%%-------------------------------------------------------------------- +%% Testcases +%%-------------------------------------------------------------------- + +testcase_1() -> + []. +testcase_1(Config) -> + init = ?config(suite,Config), + testcase_1 = ?config(testcase_1,Config), + ok. + +testcase_1a() -> + []. +testcase_1a(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + testcase_1a = ?config(testcase_1a,Config), + ok. +testcase_1b() -> + []. +testcase_1b(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + undefined = ?config(testcase_1a,Config), + testcase_1b = ?config(testcase_1b,Config), + ok. + +testcase_2() -> + []. +testcase_2(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_1a,Config), + undefined = ?config(test_group_1b,Config), + testcase_2 = ?config(testcase_2,Config), + ok. + +testcase_2a() -> + []. +testcase_2a(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + testcase_2a = ?config(testcase_2a,Config), + ok. +testcase_2b() -> + []. +testcase_2b(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + undefined = ?config(testcase_2a,Config), + testcase_2b = ?config(testcase_2b,Config), + ok. + +testcase_3a() -> + []. +testcase_3a(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + test_group_3 = ?config(test_group_3,Config), + undefined = ?config(testcase_2b,Config), + testcase_3a = ?config(testcase_3a,Config), + ok. +testcase_3b() -> + []. +testcase_3b(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + test_group_3 = ?config(test_group_3,Config), + undefined = ?config(testcase_3a,Config), + testcase_3b = ?config(testcase_3b,Config), + ok. + +testcase_3() -> + []. +testcase_3(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_2,Config), + undefined = ?config(test_group_3,Config), + testcase_3 = ?config(testcase_3,Config), + ok. + +testcase_5a() -> + []. +testcase_5a(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_3,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + undefined = ?config(testcase_3,Config), + testcase_5a = ?config(testcase_5a,Config), + ok. +testcase_5b() -> + []. +testcase_5b(Config) -> + init = ?config(suite,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + undefined = ?config(testcase_5a,Config), + testcase_5b = ?config(testcase_5b,Config), + ok. + +testcase_7a() -> + []. +testcase_7a(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_3,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_7 = ?config(test_group_7,Config), + testcase_7a = ?config(testcase_7a,Config), + ok. +testcase_7b() -> + []. +testcase_7b(Config) -> + init = ?config(suite,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_7 = ?config(test_group_7,Config), + undefined = ?config(testcase_7a,Config), + testcase_7b = ?config(testcase_7b,Config), + ok. diff --git a/lib/common_test/test/ct_groups_test_2_SUITE_data/groups_2/groups_22_SUITE.erl b/lib/common_test/test/ct_groups_test_2_SUITE_data/groups_2/groups_22_SUITE.erl new file mode 100644 index 0000000000..cd517876df --- /dev/null +++ b/lib/common_test/test/ct_groups_test_2_SUITE_data/groups_2/groups_22_SUITE.erl @@ -0,0 +1,314 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(groups_22_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%==================================================================== +%% COMMON TEST CALLBACK FUNCTIONS +%%==================================================================== + +suite() -> + [{timetrap,{minutes,1}}]. + +groups() -> + [ + {test_group_1a, [shuffle], [testcase_1a,testcase_1b,testcase_1c]}, + + {test_group_1b, [parallel], [testcase_1a,testcase_1b]}, + + {test_group_2, [parallel], [testcase_2a, + + {test_group_3, [{repeat,1}], + [testcase_3a, testcase_3b]}, + + testcase_2b]}, + + {test_group_4, [{test_group_5, [parallel], [testcase_5a, + + {group, test_group_6}, + + testcase_5b]}]}, + + {test_group_6, [parallel], [{group, test_group_7}]}, + + {test_group_7, [sequence], [testcase_7a,testcase_7b]} + ]. + +all() -> + [{group, test_group_1a}, + {group, test_group_1b}, + testcase_1, + testcase_2, + {group, test_group_2}, + testcase_3, + {group, test_group_4}]. + +%% this func only for internal test purposes +grs_and_tcs() -> + {[ + test_group_1a, test_group_1b, + test_group_2, test_group_3, + test_group_4, test_group_5, + test_group_6, test_group_7 + ], + [ + testcase_1a, testcase_1b, testcase_1c, + testcase_1, + testcase_2, + testcase_2a, testcase_2b, + testcase_3a, testcase_3b, + testcase_3, + testcase_5a, testcase_5b, + testcase_7a, testcase_7b + ]}. + +%%-------------------------------------------------------------------- +%% Suite Configuration +%%-------------------------------------------------------------------- + +init_per_suite(Config) -> + [{suite,init}|Config]. + +end_per_suite(Config) -> + init = ?config(suite,Config). + +%%-------------------------------------------------------------------- +%% Group Configuration +%%-------------------------------------------------------------------- + +init_per_group(Group, Config) -> + Cmt = + case {Group,?config(tc_group_properties,Config)} of + {test_group_1a,[{shuffle,S},{name,test_group_1a}]} -> + io_lib:format("shuffled, ~w", [S]); + {test_group_1b,[{name,test_group_1b},parallel]} -> "parallel"; + {test_group_2,[{name,test_group_2},parallel]} -> "parallel"; + {test_group_3,[{name,test_group_3},{repeat,1}]} -> "repeat 1"; + {test_group_3,[{name,test_group_3}]} -> "repeat 0"; + {test_group_4,[{name,test_group_4}]} -> ok; + {test_group_5,[{name,test_group_5},parallel]} -> "parallel"; + {test_group_6,[{name,test_group_6},parallel]} -> "parallel"; + {test_group_7,[{name,test_group_7},sequence]} -> "sequence" + end, + {Grs,_} = grs_and_tcs(), + case lists:member(Group, Grs) of + true -> + init = ?config(suite,Config), + ct:comment(io_lib:format("~w, ~s", [Group,Cmt])), + [{Group,Group} | Config]; + false -> + ct:fail({bad_group,Group}) + end. + +end_per_group(Group, Config) -> + {Grs,_} = grs_and_tcs(), + case lists:member(Group, Grs) of + true -> + init = ?config(suite,Config), + Group = ?config(Group,Config), + ok; + false -> + ct:fail({bad_group,Group}) + end. + +%%-------------------------------------------------------------------- +%% Testcase Configuration +%%-------------------------------------------------------------------- + +init_per_testcase(TestCase, Config) -> + {_,TCs} = grs_and_tcs(), + case lists:member(TestCase, TCs) of + true -> + init = ?config(suite,Config), + [{TestCase,TestCase} | Config]; + false -> + ct:fail({unknown_testcase,TestCase}) + end. + +end_per_testcase(TestCase, Config) -> + {_,TCs} = grs_and_tcs(), + case lists:member(TestCase, TCs) of + true -> + init = ?config(suite,Config), + TestCase = ?config(TestCase,Config), + ok; + false -> + ct:fail({unknown_testcase,TestCase}) + end. + + +%%-------------------------------------------------------------------- +%% Testcases +%%-------------------------------------------------------------------- + +testcase_1a() -> + []. +testcase_1a(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + testcase_1a = ?config(testcase_1a,Config), + ok. +testcase_1b() -> + []. +testcase_1b(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + undefined = ?config(testcase_1a,Config), + testcase_1b = ?config(testcase_1b,Config), + ok. + +testcase_1c() -> + []. +testcase_1c(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + undefined = ?config(testcase_1b,Config), + testcase_1c = ?config(testcase_1c,Config), + ok. + +testcase_1() -> + []. +testcase_1(Config) -> + init = ?config(suite,Config), + testcase_1 = ?config(testcase_1,Config), + ok. + +testcase_2() -> + []. +testcase_2(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_1a,Config), + undefined = ?config(test_group_1b,Config), + testcase_2 = ?config(testcase_2,Config), + ok. + +testcase_2a() -> + []. +testcase_2a(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + testcase_2a = ?config(testcase_2a,Config), + ok. +testcase_2b() -> + []. +testcase_2b(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + undefined = ?config(testcase_2a,Config), + testcase_2b = ?config(testcase_2b,Config), + ok. + +testcase_3a() -> + []. +testcase_3a(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + test_group_3 = ?config(test_group_3,Config), + undefined = ?config(testcase_2b,Config), + testcase_3a = ?config(testcase_3a,Config), + ok. +testcase_3b() -> + []. +testcase_3b(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + test_group_3 = ?config(test_group_3,Config), + undefined = ?config(testcase_3a,Config), + testcase_3b = ?config(testcase_3b,Config), + ok. + +testcase_3() -> + []. +testcase_3(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_2,Config), + undefined = ?config(test_group_3,Config), + testcase_3 = ?config(testcase_3,Config), + ok. + +testcase_5a() -> + []. +testcase_5a(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_3,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + undefined = ?config(testcase_3,Config), + testcase_5a = ?config(testcase_5a,Config), + %% increase chance the done event will come + %% during execution of subgroup (could be + %% tricky to handle) + timer:sleep(3), + ok. +testcase_5b() -> + []. +testcase_5b(Config) -> + init = ?config(suite,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + undefined = ?config(testcase_5a,Config), + testcase_5b = ?config(testcase_5b,Config), + ok. + +testcase_7a() -> + []. +testcase_7a(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_3,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_7 = ?config(test_group_7,Config), + testcase_7a = ?config(testcase_7a,Config), + ok. +testcase_7b() -> + []. +testcase_7b(Config) -> + init = ?config(suite,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_7 = ?config(test_group_7,Config), + undefined = ?config(testcase_7a,Config), + testcase_7b = ?config(testcase_7b,Config), + ok. diff --git a/lib/common_test/test/ct_master_SUITE.erl b/lib/common_test/test/ct_master_SUITE.erl new file mode 100644 index 0000000000..87e2c3049a --- /dev/null +++ b/lib/common_test/test/ct_master_SUITE.erl @@ -0,0 +1,136 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%------------------------------------------------------------------- +%%% File: ct_master_SUITE +%%% +%%% Description: +%%% Test ct_master. +%%% +%%% The suites used for the test are located in the data directory. +%%%------------------------------------------------------------------- +-module(ct_master_SUITE). +-compile(export_all). + +-include_lib("test_server/include/test_server.hrl"). +-include_lib("common_test/include/ct_event.hrl"). + +-define(eh, ct_test_support_eh). + +%%-------------------------------------------------------------------- +%% TEST SERVER CALLBACK FUNCTIONS +%%-------------------------------------------------------------------- + +%%-------------------------------------------------------------------- +%% Description: Since Common Test starts another Test Server +%% instance, the tests need to be performed on a separate node (or +%% there will be clashes with logging processes etc). +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + Config1 = ct_test_support:init_per_suite(Config), + Config1. + +end_per_suite(Config) -> + ct_test_support:end_per_suite(Config). + +init_per_testcase(TestCase, Config) -> + ct_test_support:init_per_testcase(TestCase, [{master, true}|Config]). + +end_per_testcase(TestCase, Config) -> + ct_test_support:end_per_testcase(TestCase, Config). + +all(doc) -> + [""]; + +all(suite) -> + [ + ct_master_test + ]. + +%%-------------------------------------------------------------------- +%% TEST CASES +%%-------------------------------------------------------------------- +ct_master_test(Config) when is_list(Config)-> + NodeCount = 5, + DataDir = ?config(data_dir, Config), + PrivDir = ?config(priv_dir, Config), + NodeNames = [list_to_atom("testnode_"++integer_to_list(N)) || + N <- lists:seq(1, NodeCount)], + FileName = filename:join(PrivDir, "ct_master_spec.spec"), + Suites = [master_SUITE], + TSFile = make_spec(DataDir, FileName, NodeNames, Suites, Config), + [{TSFile, ok}] = run_test(ct_master_test, FileName, Config), + ok. + +%%%----------------------------------------------------------------- +%%% HELP FUNCTIONS +%%%----------------------------------------------------------------- +make_spec(DataDir, FileName, NodeNames, Suites, Config)-> + {ok, HostName} = inet:gethostname(), + + N = lists:map(fun(NodeName)-> + {node, NodeName, list_to_atom(atom_to_list(NodeName)++"@"++HostName)} + end, + NodeNames), + + C = lists:map(fun(NodeName)-> + Rnd = random:uniform(2), + if Rnd == 1-> + {config, NodeName, filename:join(DataDir, "master/config.txt")}; + true-> + {userconfig, NodeName, {ct_config_xml, filename:join(DataDir, "master/config.xml")}} + end + end, + NodeNames), + + NS = lists:map(fun(NodeName)-> + {init, NodeName, [ + {node_start, [{startup_functions, []}, {monitor_master, false}]}, + {eval, {erlang, nodes, []}} + ] + } + end, + NodeNames), + + S = [{suites, NodeNames, filename:join(DataDir, "master"), Suites}], + + PrivDir = ?config(priv_dir, Config), + LD = lists:map(fun(NodeName)-> + {logdir, NodeName, get_log_dir(PrivDir, NodeName)} + end, + NodeNames) ++ [{logdir, master, PrivDir}], + + ct_test_support:write_testspec(N++C++S++LD++NS, FileName). + +get_log_dir(PrivDir, NodeName)-> + LogDir = filename:join(PrivDir, io_lib:format("slave.~p", [NodeName])), + file:make_dir(LogDir), + LogDir. + +run_test(_Name, FileName, Config)-> + [{FileName, ok}] = ct_test_support:run(ct_master, run, [FileName], Config). + +reformat_events(Events, EH) -> + ct_test_support:reformat(Events, EH). + +%%%----------------------------------------------------------------- +%%% TEST EVENTS +%%%----------------------------------------------------------------- +expected_events(_)-> +[]. diff --git a/lib/common_test/test/ct_master_SUITE_data/master/config.txt b/lib/common_test/test/ct_master_SUITE_data/master/config.txt new file mode 100644 index 0000000000..3baf9e392c --- /dev/null +++ b/lib/common_test/test/ct_master_SUITE_data/master/config.txt @@ -0,0 +1,2 @@ +{a, b}. +{c, d}. diff --git a/lib/common_test/test/ct_master_SUITE_data/master/config.xml b/lib/common_test/test/ct_master_SUITE_data/master/config.xml new file mode 100644 index 0000000000..c031f45f35 --- /dev/null +++ b/lib/common_test/test/ct_master_SUITE_data/master/config.xml @@ -0,0 +1,4 @@ +<config> + <a>b</a> + <c>d</c> +</config> diff --git a/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl b/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl new file mode 100644 index 0000000000..e37ec3659c --- /dev/null +++ b/lib/common_test/test/ct_master_SUITE_data/master/master_SUITE.erl @@ -0,0 +1,57 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%------------------------------------------------------------------- +%%% File: master_SUITE +%%% +%%% Description: +%%% Test suite for common_test which tests the ct_master functionality +%%%------------------------------------------------------------------- +-module(master_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +suite() -> + []. + +init_per_suite(Config) -> + Config. + +end_per_suite(_) -> + ok. + +all() -> [first_testcase, second_testcase, third_testcase]. + +init_per_testcase(_, Config) -> + Config. + +end_per_testcase(_, _) -> + ok. + +first_testcase(_)-> + b = ct:get_config(a). + +second_testcase(_)-> + d = ct:get_config(c). + +third_testcase(_)-> + A = 4, + A = 2*2. diff --git a/lib/common_test/test/ct_misc_1_SUITE.erl b/lib/common_test/test/ct_misc_1_SUITE.erl new file mode 100644 index 0000000000..eb6c6aa101 --- /dev/null +++ b/lib/common_test/test/ct_misc_1_SUITE.erl @@ -0,0 +1,165 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%------------------------------------------------------------------- +%%% File: ct_misc_1_SUITE +%%% +%%% Description: +%%% Test misc things in Common Test suites. +%%% +%%% The suites used for the test are located in the data directory. +%%%------------------------------------------------------------------- +-module(ct_misc_1_SUITE). + +-compile(export_all). + +-include_lib("test_server/include/test_server.hrl"). +-include_lib("test_server/include/test_server_line.hrl"). +-include_lib("common_test/include/ct_event.hrl"). + +-define(eh, ct_test_support_eh). + +%%-------------------------------------------------------------------- +%% TEST SERVER CALLBACK FUNCTIONS +%%-------------------------------------------------------------------- + +%%-------------------------------------------------------------------- +%% Description: Since Common Test starts another Test Server +%% instance, the tests need to be performed on a separate node (or +%% there will be clashes with logging processes etc). +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + Config1 = ct_test_support:init_per_suite(Config), + Config1. + +end_per_suite(Config) -> + ct_test_support:end_per_suite(Config). + +init_per_testcase(TestCase, Config) -> + ct_test_support:init_per_testcase(TestCase, Config). + +end_per_testcase(TestCase, Config) -> + ct_test_support:end_per_testcase(TestCase, Config). + +all(doc) -> + [""]; + +all(suite) -> + [ + beam_me_up + ]. + +%%-------------------------------------------------------------------- +%% TEST CASES +%%-------------------------------------------------------------------- + +%%%----------------------------------------------------------------- +%%% +beam_me_up(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + CTNode = ?config(ct_node, Config), + + %% Path = rpc:call(CTNode, code, get_path, []), + %% [_ | Parts] = lists:reverse(filename:split(DataDir)), + %% TSDir = filename:join(lists:reverse(Parts)), + %% true = rpc:call(CTNode, code, del_path, [TSDir]), + + Mods = [beam_1_SUITE, beam_2_SUITE], + Suites = [atom_to_list(M) || M <- Mods], + [{error,_} = rpc:call(CTNode, code, load_file, [M]) || M <- Mods], + + code:add_path(DataDir), + CRes = + [compile:file(filename:join(DataDir,F), + [verbose,report_errors, + report_warnings,binary]) || F <- Suites], + + [{module,_} = rpc:call(CTNode, code, load_binary, + [Mod, atom_to_list(Mod), Bin]) || + {ok,Mod,Bin} <- CRes], + + {Opts,ERPid} = setup([{suite,Suites},{auto_compile,false}], Config), + + ok = ct_test_support:run(ct, run_test, [Opts], Config), + Events = ct_test_support:get_events(ERPid, Config), + + ct_test_support:log_events(beam_me_up, + reformat(Events, ?eh), + ?config(priv_dir, Config)), + + TestEvents = events_to_check(beam_me_up, 1), + ok = ct_test_support:verify_events(TestEvents, Events, Config). + +%%%----------------------------------------------------------------- +%%% HELP FUNCTIONS +%%%----------------------------------------------------------------- + +setup(Test, Config) -> + Opts0 = ct_test_support:get_opts(Config), + Level = ?config(trace_level, Config), + EvHArgs = [{cbm,ct_test_support},{trace_level,Level}], + Opts = Opts0 ++ [{event_handler,{?eh,EvHArgs}}|Test], + ERPid = ct_test_support:start_event_receiver(Config), + {Opts,ERPid}. + +reformat(Events, EH) -> + ct_test_support:reformat(Events, EH). +%reformat(Events, _EH) -> +% Events. + +%%%----------------------------------------------------------------- +%%% TEST EVENTS +%%%----------------------------------------------------------------- +events_to_check(Test) -> + %% 2 tests (ct:run_test + script_start) is default + events_to_check(Test, 2). + +events_to_check(_, 0) -> + []; +events_to_check(Test, N) -> + test_events(Test) ++ events_to_check(Test, N-1). + +test_events(beam_me_up) -> + [ + {?eh,start_logging,{'DEF','RUNDIR'}}, + {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, + {?eh,start_info,{2,2,4}}, + {?eh,tc_start,{beam_1_SUITE,init_per_suite}}, + {?eh,tc_done,{beam_1_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{beam_1_SUITE,tc1}}, + {?eh,tc_done,{beam_1_SUITE,tc1,ok}}, + {?eh,test_stats,{1,0,{0,0}}}, + {?eh,tc_start,{beam_1_SUITE,tc2}}, + {?eh,tc_done,{beam_1_SUITE,tc2,{failed,{error,'tc2 failed'}}}}, + {?eh,test_stats,{1,1,{0,0}}}, + {?eh,tc_start,{beam_1_SUITE,end_per_suite}}, + {?eh,tc_done,{beam_1_SUITE,end_per_suite,ok}}, + {?eh,tc_start,{beam_2_SUITE,init_per_suite}}, + {?eh,tc_done,{beam_2_SUITE,init_per_suite,ok}}, + {?eh,tc_start,{beam_2_SUITE,tc1}}, + {?eh,tc_done,{beam_2_SUITE,tc1,ok}}, + {?eh,test_stats,{2,1,{0,0}}}, + {?eh,tc_start,{beam_2_SUITE,tc2}}, + {?eh,tc_done,{beam_2_SUITE,tc2,{failed,{error,'tc2 failed'}}}}, + {?eh,test_stats,{2,2,{0,0}}}, + {?eh,tc_start,{beam_2_SUITE,end_per_suite}}, + {?eh,tc_done,{beam_2_SUITE,end_per_suite,ok}}, + {?eh,test_done,{'DEF','STOP_TIME'}}, + {?eh,stop_logging,[]} + ]. diff --git a/lib/common_test/test/ct_misc_1_SUITE_data/beam_1_SUITE.erl b/lib/common_test/test/ct_misc_1_SUITE_data/beam_1_SUITE.erl new file mode 100644 index 0000000000..382bdefded --- /dev/null +++ b/lib/common_test/test/ct_misc_1_SUITE_data/beam_1_SUITE.erl @@ -0,0 +1,134 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(beam_1_SUITE). + +%% Note: This directive should only be used in test suites. +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% COMMON TEST CALLBACK FUNCTIONS +%%-------------------------------------------------------------------- + +%%-------------------------------------------------------------------- +%% Function: suite() -> Info +%% +%% Info = [tuple()] +%% List of key/value pairs. +%% +%% Description: Returns list of tuples to set default properties +%% for the suite. +%% +%% Note: The suite/0 function is only meant to be used to return +%% default data values, not perform any other operations. +%%-------------------------------------------------------------------- +suite() -> + [ + {timetrap,{seconds,10}} + ]. + +%%-------------------------------------------------------------------- +%% Function: init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% Reason = term() +%% The reason for skipping the suite. +%% +%% Description: Initialization before the suite. +%% +%% Note: This function is free to add any key/value pairs to the Config +%% variable, but should NOT alter/remove any existing entries. +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_suite(Config0) -> void() | {save_config,Config1} +%% +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% +%% Description: Cleanup after the suite. +%%-------------------------------------------------------------------- +end_per_suite(_Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Function: init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% +%% TestCase = atom() +%% Name of the test case that is about to run. +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% Reason = term() +%% The reason for skipping the test case. +%% +%% Description: Initialization before each test case. +%% +%% Note: This function is free to add any key/value pairs to the Config +%% variable, but should NOT alter/remove any existing entries. +%%-------------------------------------------------------------------- +init_per_testcase(_TestCase, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} +%% +%% TestCase = atom() +%% Name of the test case that is finished. +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% +%% Description: Cleanup after each test case. +%%-------------------------------------------------------------------- +end_per_testcase(_TestCase, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Function: all() -> TestCases | {skip,Reason} +%% +%% TestCases = [TestCase | {sequence,SeqName}] +%% TestCase = atom() +%% Name of a test case. +%% SeqName = atom() +%% Name of a test case sequence. +%% Reason = term() +%% The reason for skipping all test cases. +%% +%% Description: Returns the list of test cases that are to be executed. +%%-------------------------------------------------------------------- +all() -> + [tc1, tc2]. + +%%-------------------------------------------------------------------- +%% TEST CASES +%%-------------------------------------------------------------------- + +tc1(_Config) -> + ct:comment("tc1 executed"), + ok. + +tc2(_Config) -> + exit('tc2 failed'). diff --git a/lib/common_test/test/ct_misc_1_SUITE_data/beam_2_SUITE.erl b/lib/common_test/test/ct_misc_1_SUITE_data/beam_2_SUITE.erl new file mode 100644 index 0000000000..70c1f2b471 --- /dev/null +++ b/lib/common_test/test/ct_misc_1_SUITE_data/beam_2_SUITE.erl @@ -0,0 +1,134 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(beam_2_SUITE). + +%% Note: This directive should only be used in test suites. +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% COMMON TEST CALLBACK FUNCTIONS +%%-------------------------------------------------------------------- + +%%-------------------------------------------------------------------- +%% Function: suite() -> Info +%% +%% Info = [tuple()] +%% List of key/value pairs. +%% +%% Description: Returns list of tuples to set default properties +%% for the suite. +%% +%% Note: The suite/0 function is only meant to be used to return +%% default data values, not perform any other operations. +%%-------------------------------------------------------------------- +suite() -> + [ + {timetrap,{seconds,10}} + ]. + +%%-------------------------------------------------------------------- +%% Function: init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% Reason = term() +%% The reason for skipping the suite. +%% +%% Description: Initialization before the suite. +%% +%% Note: This function is free to add any key/value pairs to the Config +%% variable, but should NOT alter/remove any existing entries. +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_suite(Config0) -> void() | {save_config,Config1} +%% +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% +%% Description: Cleanup after the suite. +%%-------------------------------------------------------------------- +end_per_suite(_Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Function: init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% +%% TestCase = atom() +%% Name of the test case that is about to run. +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% Reason = term() +%% The reason for skipping the test case. +%% +%% Description: Initialization before each test case. +%% +%% Note: This function is free to add any key/value pairs to the Config +%% variable, but should NOT alter/remove any existing entries. +%%-------------------------------------------------------------------- +init_per_testcase(_TestCase, Config) -> + Config. + +%%-------------------------------------------------------------------- +%% Function: end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} +%% +%% TestCase = atom() +%% Name of the test case that is finished. +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% +%% Description: Cleanup after each test case. +%%-------------------------------------------------------------------- +end_per_testcase(_TestCase, _Config) -> + ok. + +%%-------------------------------------------------------------------- +%% Function: all() -> TestCases | {skip,Reason} +%% +%% TestCases = [TestCase | {sequence,SeqName}] +%% TestCase = atom() +%% Name of a test case. +%% SeqName = atom() +%% Name of a test case sequence. +%% Reason = term() +%% The reason for skipping all test cases. +%% +%% Description: Returns the list of test cases that are to be executed. +%%-------------------------------------------------------------------- +all() -> + [tc1, tc2]. + +%%-------------------------------------------------------------------- +%% TEST CASES +%%-------------------------------------------------------------------- + +tc1(_Config) -> + ct:comment("tc1 executed"), + ok. + +tc2(_Config) -> + exit('tc2 failed'). diff --git a/lib/common_test/test/ct_skip_SUITE.erl b/lib/common_test/test/ct_skip_SUITE.erl index 9f428723f5..2e02061dec 100644 --- a/lib/common_test/test/ct_skip_SUITE.erl +++ b/lib/common_test/test/ct_skip_SUITE.erl @@ -89,14 +89,14 @@ auto_skip(Config) when is_list(Config) -> ], {Opts,ERPid} = setup({suite,Suites}, Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), ct_test_support:log_events(auto_skip, reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(auto_skip), + TestEvents = events_to_check(auto_skip), ok = ct_test_support:verify_events(TestEvents, Events, Config). @@ -112,14 +112,14 @@ user_skip(Config) when is_list(Config) -> Join(DataDir, "user_skip_5_SUITE")], {Opts,ERPid} = setup({suite,Suites}, Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), ct_test_support:log_events(user_skip, reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(user_skip), + TestEvents = events_to_check(user_skip), ok = ct_test_support:verify_events(TestEvents, Events, Config). %%%----------------------------------------------------------------- @@ -142,6 +142,15 @@ reformat(Events, EH) -> %%%----------------------------------------------------------------- %%% TEST EVENTS %%%----------------------------------------------------------------- +events_to_check(Test) -> + %% 2 tests (ct:run_test + script_start) is default + events_to_check(Test, 2). + +events_to_check(_, 0) -> + []; +events_to_check(Test, N) -> + test_events(Test) ++ events_to_check(Test, N-1). + test_events(auto_skip) -> [ {?eh,start_logging,{'DEF','RUNDIR'}}, diff --git a/lib/common_test/test/ct_smoke_test_SUITE.erl b/lib/common_test/test/ct_smoke_test_SUITE.erl index f1c695f614..05a2c20695 100644 --- a/lib/common_test/test/ct_smoke_test_SUITE.erl +++ b/lib/common_test/test/ct_smoke_test_SUITE.erl @@ -162,7 +162,7 @@ dir1(Config) when is_list(Config) -> ERPid = ct_test_support:start_event_receiver(Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), @@ -170,7 +170,7 @@ dir1(Config) when is_list(Config) -> ct_test_support:reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(dir1), + TestEvents = events_to_check(dir1), ok = ct_test_support:verify_events(TestEvents, Events, Config). %%%----------------------------------------------------------------- @@ -191,7 +191,7 @@ dir2(Config) when is_list(Config) -> ERPid = ct_test_support:start_event_receiver(Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), @@ -199,7 +199,7 @@ dir2(Config) when is_list(Config) -> ct_test_support:reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(dir2), + TestEvents = events_to_check(dir2), ok = ct_test_support:verify_events(TestEvents, Events, Config). %%%----------------------------------------------------------------- @@ -221,7 +221,7 @@ dir1_2(Config) when is_list(Config) -> ERPid = ct_test_support:start_event_receiver(Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), @@ -229,7 +229,7 @@ dir1_2(Config) when is_list(Config) -> ct_test_support:reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(dir1_2), + TestEvents = events_to_check(dir1_2), ok = ct_test_support:verify_events(TestEvents, Events, Config). %%%----------------------------------------------------------------- @@ -251,7 +251,7 @@ suite11(Config) when is_list(Config) -> ERPid = ct_test_support:start_event_receiver(Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), @@ -259,7 +259,7 @@ suite11(Config) when is_list(Config) -> ct_test_support:reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(suite11), + TestEvents = events_to_check(suite11), ok = ct_test_support:verify_events(TestEvents, Events, Config). %%%----------------------------------------------------------------- @@ -280,7 +280,7 @@ suite21(Config) when is_list(Config) -> ERPid = ct_test_support:start_event_receiver(Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), @@ -288,7 +288,7 @@ suite21(Config) when is_list(Config) -> ct_test_support:reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(suite21), + TestEvents = events_to_check(suite21), ok = ct_test_support:verify_events(TestEvents, Events, Config). %%%----------------------------------------------------------------- @@ -311,7 +311,7 @@ suite11_21(Config) when is_list(Config) -> ERPid = ct_test_support:start_event_receiver(Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), @@ -319,7 +319,7 @@ suite11_21(Config) when is_list(Config) -> ct_test_support:reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(suite11_21), + TestEvents = events_to_check(suite11_21), ok = ct_test_support:verify_events(TestEvents, Events, Config). %%%----------------------------------------------------------------- @@ -342,7 +342,7 @@ tc111(Config) when is_list(Config) -> ERPid = ct_test_support:start_event_receiver(Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), @@ -350,7 +350,7 @@ tc111(Config) when is_list(Config) -> ct_test_support:reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(tc111), + TestEvents = events_to_check(tc111), ok = ct_test_support:verify_events(TestEvents, Events, Config). %%%----------------------------------------------------------------- @@ -372,7 +372,7 @@ tc211(Config) when is_list(Config) -> ERPid = ct_test_support:start_event_receiver(Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), @@ -380,7 +380,7 @@ tc211(Config) when is_list(Config) -> ct_test_support:reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(tc211), + TestEvents = events_to_check(tc211), ok = ct_test_support:verify_events(TestEvents, Events, Config). %%%----------------------------------------------------------------- @@ -403,7 +403,7 @@ tc111_112(Config) when is_list(Config) -> ERPid = ct_test_support:start_event_receiver(Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), @@ -411,7 +411,7 @@ tc111_112(Config) when is_list(Config) -> ct_test_support:reformat(Events, ?eh), ?config(priv_dir, Config)), - TestEvents = test_events(tc111_112), + TestEvents = events_to_check(tc111_112), ok = ct_test_support:verify_events(TestEvents, Events, Config). @@ -423,8 +423,16 @@ eh_opts(Config) -> Level = ?config(trace_level, Config), [{event_handler,{?eh,[{cbm,ct_test_support},{trace_level,Level}]}}]. +events_to_check(Test) -> + %% 2 tests (ct:run_test + script_start) is default + events_to_check(Test, 2). -test_events(Test) when Test == dir1 ; Test == dir2 ; +events_to_check(_, 0) -> + []; +events_to_check(Test, N) -> + events(Test) ++ events_to_check(Test, N-1). + +events(Test) when Test == dir1 ; Test == dir2 ; Test == suite11 ; Test == suite21 -> Suite = if Test == dir1 ; Test == suite11 -> happy_11_SUITE; true -> happy_21_SUITE @@ -465,7 +473,7 @@ test_events(Test) when Test == dir1 ; Test == dir2 ; {?eh,test_done,{'DEF','STOP_TIME'}}, {?eh,stop_logging,[]} ]; -test_events(Test) when Test == dir1_2 ; Test == suite11_21 -> +events(Test) when Test == dir1_2 ; Test == suite11_21 -> [ {?eh,start_logging,{'DEF','RUNDIR'}}, {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, @@ -532,7 +540,7 @@ test_events(Test) when Test == dir1_2 ; Test == suite11_21 -> {?eh,stop_logging,[]} ]; -test_events(Test) when Test == tc111 ; Test == tc211 -> +events(Test) when Test == tc111 ; Test == tc211 -> Suite = if Test == tc111 -> happy_11_SUITE; true -> happy_21_SUITE end, [ {?eh,start_logging,{'DEF','RUNDIR'}}, @@ -549,7 +557,7 @@ test_events(Test) when Test == tc111 ; Test == tc211 -> {?eh,stop_logging,[]} ]; -test_events(tc111_112) -> +events(tc111_112) -> [ {?eh,start_logging,{'DEF','RUNDIR'}}, {?eh,test_start,{'DEF',{'START_TIME','LOGDIR'}}}, diff --git a/lib/common_test/test/ct_test_server_if_1_SUITE.erl b/lib/common_test/test/ct_test_server_if_1_SUITE.erl index 069f8c75fc..eb85409073 100644 --- a/lib/common_test/test/ct_test_server_if_1_SUITE.erl +++ b/lib/common_test/test/ct_test_server_if_1_SUITE.erl @@ -87,14 +87,14 @@ ts_if_1(Config) when is_list(Config) -> TestSpecName = ct_test_support:write_testspec(TestSpec, PrivDir, "ts_if_1_spec"), {Opts,ERPid} = setup({spec,TestSpecName}, Config), - ok = ct_test_support:run(ct, run_test, [Opts], Config), + ok = ct_test_support:run(Opts, Config), Events = ct_test_support:get_events(ERPid, Config), ct_test_support:log_events(ts_if_1, reformat(Events, ?eh), PrivDir), - TestEvents = test_events(ts_if_1), + TestEvents = events_to_check(ts_if_1), ok = ct_test_support:verify_events(TestEvents, Events, Config). @@ -119,6 +119,15 @@ reformat(Events, EH) -> %%%----------------------------------------------------------------- %%% TEST EVENTS %%%----------------------------------------------------------------- +events_to_check(Test) -> + %% 2 tests (ct:run_test + script_start) is default + events_to_check(Test, 2). + +events_to_check(_, 0) -> + []; +events_to_check(Test, N) -> + test_events(Test) ++ events_to_check(Test, N-1). + test_events(ts_if_1) -> [ {?eh,start_logging,{'DEF','RUNDIR'}}, @@ -193,14 +202,14 @@ test_events(ts_if_1) -> {?eh,tc_done,{ts_if_1_SUITE,{end_per_group,g2,[parallel]},ok}}]}, {?eh,tc_start,{ts_if_1_SUITE,tc12}}, - {?eh,tc_done,{undefined,undefined,{testcase_aborted,{abort_current_testcase,tc12},'_'}}}, + {?eh,tc_done,{ts_if_1_SUITE,tc12,{failed,{testcase_aborted,'stopping tc12'}}}}, {?eh,test_stats,{2,5,{3,6}}}, {?eh,tc_start,{ts_if_1_SUITE,tc13}}, {?eh,tc_done,{ts_if_1_SUITE,tc13,ok}}, {?eh,test_stats,{3,5,{3,6}}}, {?eh,tc_start,{ts_if_1_SUITE,end_per_suite}}, {?eh,tc_done,{ts_if_1_SUITE,end_per_suite,ok}}, -%%! + {?eh,tc_start,{ts_if_2_SUITE,init_per_suite}}, {?eh,tc_done,{ts_if_2_SUITE,init_per_suite, {failed,{error,{suite0_failed,{exited,suite0_goes_boom}}}}}}, diff --git a/lib/common_test/test/ct_test_server_if_1_SUITE_data/test_server_if/test/ts_if_1_SUITE.erl b/lib/common_test/test/ct_test_server_if_1_SUITE_data/test_server_if/test/ts_if_1_SUITE.erl index 8e90df21ce..47cea190dd 100644 --- a/lib/common_test/test/ct_test_server_if_1_SUITE_data/test_server_if/test/ts_if_1_SUITE.erl +++ b/lib/common_test/test/ct_test_server_if_1_SUITE_data/test_server_if/test/ts_if_1_SUITE.erl @@ -93,6 +93,10 @@ init_per_testcase(_TestCase, Config) -> %%-------------------------------------------------------------------- end_per_testcase(tc2, Config) -> timer:sleep(5000); +end_per_testcase(tc12, Config) -> + ct:comment("end_per_testcase(tc12) called!"), + ct:pal("end_per_testcase(tc12) called!", []), + ok; end_per_testcase(_TestCase, _Config) -> ok. @@ -180,9 +184,9 @@ gtc2(_) -> exit(should_have_been_skipped). tc12(_) -> - F = fun() -> ct:abort_current_testcase({abort_current_testcase,tc12}) end, + F = fun() -> ct:abort_current_testcase('stopping tc12') end, spawn(F), - timer:sleep(500), + timer:sleep(1000), exit(should_have_been_aborted). tc13(_) -> diff --git a/lib/common_test/test/ct_test_support.erl b/lib/common_test/test/ct_test_support.erl index 6148e3280e..7bfb9ffb49 100644 --- a/lib/common_test/test/ct_test_support.erl +++ b/lib/common_test/test/ct_test_support.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2008-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -23,12 +23,13 @@ %%% -module(ct_test_support). --include("test_server.hrl"). +-include_lib("test_server/include/test_server.hrl"). -include_lib("common_test/include/ct_event.hrl"). -export([init_per_suite/1, init_per_suite/2, end_per_suite/1, - init_per_testcase/2, end_per_testcase/2, write_testspec/3, - run/4, get_opts/1, wait_for_ct_stop/1]). + init_per_testcase/2, end_per_testcase/2, + write_testspec/2, write_testspec/3, + run/2, run/4, get_opts/1, wait_for_ct_stop/1]). -export([handle_event/2, start_event_receiver/1, get_events/2, verify_events/3, reformat/2, log_events/3]). @@ -55,17 +56,28 @@ init_per_suite(Config, Level) -> test_server:fail(Reason); {ok,CTNode} -> test_server:format(0, "Node ~p started~n", [CTNode]), + IsCover = test_server:is_cover(), + if IsCover -> + cover:start(CTNode); + true-> + ok + end, DataDir = ?config(data_dir, Config), PrivDir = ?config(priv_dir, Config), %% PrivDir as well as directory of Test Server suites %% have to be in code path on Common Test node. - true = rpc:call(CTNode, code, add_patha, [PrivDir]), [_ | Parts] = lists:reverse(filename:split(DataDir)), TSDir = filename:join(lists:reverse(Parts)), - true = rpc:call(CTNode, code, add_patha, [TSDir]), - test_server:format(Level, "Dirs added to code path (on ~w):~n" - "~s~n~s~n", [CTNode,TSDir,PrivDir]), + AddPathDirs = case ?config(path_dirs, Config) of + undefined -> []; + Ds -> Ds + end, + PathDirs = [PrivDir,TSDir | AddPathDirs], + [true = rpc:call(CTNode, code, add_patha, [D]) || D <- PathDirs], + test_server:format(Level, "Dirs added to code path (on ~w):~n", + [CTNode]), + [io:format("~s~n", [D]) || D <- PathDirs], TraceFile = filename:join(DataDir, "ct.trace"), case file:read_file_info(TraceFile) of @@ -87,6 +99,7 @@ end_per_suite(Config) -> CTNode = ?config(ct_node, Config), PrivDir = ?config(priv_dir, Config), true = rpc:call(CTNode, code, del_path, [filename:join(PrivDir,"")]), + cover:stop(CTNode), slave:stop(CTNode), ok. @@ -95,9 +108,17 @@ end_per_suite(Config) -> init_per_testcase(_TestCase, Config) -> {_,{_,LogDir}} = lists:keysearch(logdir, 1, get_opts(Config)), - test_server:format("See Common Test logs here:\n" - "<a href=\"file://~s/all_runs.html\">~s/all_runs.html</a>", - [LogDir,LogDir]), + case lists:keysearch(master, 1, Config) of + false-> + test_server:format("See Common Test logs here:\n\n" + "<a href=\"file://~s/all_runs.html\">~s/all_runs.html</a>\n" + "<a href=\"file://~s/index.html\">~s/index.html</a>", + [LogDir,LogDir,LogDir,LogDir]); + {value, _}-> + test_server:format("See CT Master Test logs here:\n\n" + "<a href=\"file://~s/master_runs.html\">~s/master_runs.html</a>", + [LogDir,LogDir]) + end, Config. %%%----------------------------------------------------------------- @@ -111,9 +132,10 @@ end_per_testcase(_TestCase, Config) -> %%%----------------------------------------------------------------- %%% - write_testspec(TestSpec, Dir, Name) -> - TSFile = filename:join(Dir, Name), + write_testspec(TestSpec, filename:join(Dir, Name)). + +write_testspec(TestSpec, TSFile) -> {ok,Dev} = file:open(TSFile, [write]), [io:format(Dev, "~p.~n", [Entry]) || Entry <- TestSpec], file:close(Dev), @@ -158,10 +180,32 @@ get_opts(Config) -> %%%----------------------------------------------------------------- %%% +run(Opts, Config) -> + CTNode = ?config(ct_node, Config), + Level = ?config(trace_level, Config), + %% use ct interface + test_server:format(Level, "~n[RUN #1] Calling ct:run_test(~p) on ~p~n", + [Opts, CTNode]), + Result1 = rpc:call(CTNode, ct, run_test, [Opts]), + + %% use run_test interface (simulated) + test_server:format(Level, "Saving start opts on ~p: ~p~n", [CTNode,Opts]), + rpc:call(CTNode, application, set_env, [common_test, run_test_start_opts, Opts]), + test_server:format(Level, "[RUN #2] Calling ct_run:script_start() on ~p~n", [CTNode]), + Result2 = rpc:call(CTNode, ct_run, script_start, []), + case {Result1,Result2} of + {ok,ok} -> + ok; + {E,_} when E =/= ok -> + E; + {_,E} when E =/= ok -> + E + end. + run(M, F, A, Config) -> CTNode = ?config(ct_node, Config), Level = ?config(trace_level, Config), - test_server:format(Level, "Calling ~w:~w(~p) on ~p~n", + test_server:format(Level, "~nCalling ~w:~w(~p) on ~p~n", [M, F, A, CTNode]), rpc:call(CTNode, M, F, A). @@ -231,8 +275,12 @@ verify_events(TEvs, Evs, Config) -> ok end. +verify_events1([TestEv|_], [{TEH,#event{name=stop_logging,node=Node,data=_}}|_], Node, _) + when element(1,TestEv) == TEH, element(2,TestEv) =/= stop_logging -> + test_server:format("Failed to find ~p in the list of events!~n", [TestEv]), + exit({event_not_found,TestEv}); + verify_events1(TEvs = [TestEv | TestEvs], Evs = [_|Events], Node, Config) -> -%% test_server:format("Next expected event: ~p~n", [TestEv]), case catch locate(TestEv, Node, Evs, Config) of nomatch -> verify_events1(TEvs, Events, Node, Config); @@ -332,6 +380,10 @@ locate({parallel,TEvs}, Node, Evs, Config) -> EH == TEH, EvNode == Node, Mod == M, Func == F -> false; + ({EH,#event{name=stop_logging, + node=EvNode,data=_}}) when + EH == TEH, EvNode == Node -> + exit({tc_start_not_found,TEv}); (_) -> true end, Evs1), @@ -345,6 +397,10 @@ locate({parallel,TEvs}, Node, Evs, Config) -> EH == TEH, EvNode == Node, Mod == M, Func == F -> false; + ({EH,#event{name=stop_logging, + node=EvNode,data=_}}) when + EH == TEH, EvNode == Node -> + exit({tc_done_not_found,TEv}); (_) -> true end, Evs2), @@ -388,6 +444,10 @@ locate({parallel,TEvs}, Node, Evs, Config) -> EH == TEH, EvNode == Node, Mod == M, EvGName == GroupName, EvProps == Props -> false; + ({EH,#event{name=stop_logging, + node=EvNode,data=_}}) when + EH == TEH, EvNode == Node -> + exit({tc_start_not_found,TEv}); (_) -> true end, RemEvs), @@ -410,6 +470,10 @@ locate({parallel,TEvs}, Node, Evs, Config) -> EH == TEH, EvNode == Node, Mod == M, EvGName == GroupName, EvProps == Props, Res == R -> false; + ({EH,#event{name=stop_logging, + node=EvNode,data=_}}) when + EH == TEH, EvNode == Node -> + exit({tc_done_not_found,TEv}); (_) -> true end, RemEvs), @@ -429,6 +493,10 @@ locate({parallel,TEvs}, Node, Evs, Config) -> data={Mod,end_per_group,Reason}}}) when EH == TEH, EvNode == Node, Mod == M, Reason == R -> false; + ({EH,#event{name=stop_logging, + node=EvNode,data=_}}) when + EH == TEH, EvNode == Node -> + exit({tc_auto_skip_not_found,TEv}); (_) -> true end, RemEvs), @@ -533,6 +601,10 @@ locate({shuffle,TEvs}, Node, Evs, Config) -> EH == TEH, EvNode == Node, Mod == M, Func == F -> false; + ({EH,#event{name=stop_logging, + node=EvNode,data=_}}) when + EH == TEH, EvNode == Node -> + exit({tc_start_not_found,TEv}); (_) -> true end, Evs1), @@ -576,6 +648,10 @@ locate({shuffle,TEvs}, Node, Evs, Config) -> EH == TEH, EvNode == Node, Mod == M, EvGName == GroupName -> false; + ({EH,#event{name=stop_logging, + node=EvNode,data=_}}) when + EH == TEH, EvNode == Node -> + exit({tc_start_not_found,TEv}); (_) -> true end, RemEvs), @@ -611,6 +687,10 @@ locate({shuffle,TEvs}, Node, Evs, Config) -> EH == TEH, EvNode == Node, Mod == M, EvGName == GroupName, Res == R -> false; + ({EH,#event{name=stop_logging, + node=EvNode,data=_}}) when + EH == TEH, EvNode == Node -> + exit({tc_done_not_found,TEv}); (_) -> true end, RemEvs), @@ -643,6 +723,10 @@ locate({shuffle,TEvs}, Node, Evs, Config) -> data={Mod,end_per_group,Reason}}}) when EH == TEH, EvNode == Node, Mod == M, Reason == R -> false; + ({EH,#event{name=stop_logging, + node=EvNode,data=_}}) when + EH == TEH, EvNode == Node -> + exit({tc_auto_skip_not_found,TEv}); (_) -> true end, RemEvs), @@ -785,7 +869,8 @@ log_events(TC, Events, PrivDir) -> io:format(Dev, "[~n", []), log_events1(Events, Dev, " "), file:close(Dev), - io:format("Events written to logfile: ~p~n", [LogFile]), + io:format("Events written to logfile: <a href=\"file://~s\">~s</a>~n", + [LogFile,LogFile]), io:format(user, "Events written to logfile: ~p~n", [LogFile]). log_events1(Evs, Dev, "") -> diff --git a/lib/common_test/test/ct_test_support_eh.erl b/lib/common_test/test/ct_test_support_eh.erl index fd3ae18746..70f73b9b81 100644 --- a/lib/common_test/test/ct_test_support_eh.erl +++ b/lib/common_test/test/ct_test_support_eh.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2009-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -44,8 +44,20 @@ %% Description: Whenever a new event handler is added to an event manager, %% this function is called to initialize the event handler. %%-------------------------------------------------------------------- +init(String = [X|_]) when is_integer(X) -> + case erl_scan:string(String++".") of + {ok,Ts,_} -> + case erl_parse:parse_term(Ts) of + {ok,Args} -> + init(Args); + _ -> + init(String) + end; + _ -> + init(String) + end; + init(Args) -> - S1 = case lists:keysearch(cbm, 1, Args) of {_,{cbm,CBM}} -> #state{cbm=CBM}; @@ -58,7 +70,8 @@ init(Args) -> _ -> S1 end, - print(S2#state.trace_level, "Event Handler ~w started!~n", [?MODULE]), + print(S2#state.trace_level, "Event Handler ~w started with ~p~n", + [?MODULE,Args]), {ok,S2}. %%-------------------------------------------------------------------- diff --git a/lib/common_test/test/ct_testspec_1_SUITE.erl b/lib/common_test/test/ct_testspec_1_SUITE.erl new file mode 100644 index 0000000000..dc399bfb4c --- /dev/null +++ b/lib/common_test/test/ct_testspec_1_SUITE.erl @@ -0,0 +1,433 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2009-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%%%------------------------------------------------------------------- +%%% File: ct_testspec_1_SUITE +%%% +%%% Description: +%%% Test test specifications +%%% +%%% The suites used for the test are located in the data directory. +%%%------------------------------------------------------------------- +-module(ct_testspec_1_SUITE). + +-compile(export_all). + +-include_lib("test_server/include/test_server.hrl"). +-include_lib("common_test/include/ct_event.hrl"). + +-define(eh, ct_test_support_eh). + +%%-------------------------------------------------------------------- +%% TEST SERVER CALLBACK FUNCTIONS +%%-------------------------------------------------------------------- + +%%-------------------------------------------------------------------- +%% Description: Since Common Test starts another Test Server +%% instance, the tests need to be performed on a separate node (or +%% there will be clashes with logging processes etc). +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + Config1 = ct_test_support:init_per_suite(Config), + Config1. + +end_per_suite(Config) -> + ct_test_support:end_per_suite(Config). + +init_per_testcase(TestCase, Config) -> + ct_test_support:init_per_testcase(TestCase, Config). + +end_per_testcase(TestCase, Config) -> + ct_test_support:end_per_testcase(TestCase, Config). + +all(doc) -> + ["Run smoke tests of Common Test."]; + +all(suite) -> + [all_suites, skip_all_suites, + suite, skip_suite, + all_testcases, skip_all_testcases, + testcase, skip_testcase, + all_groups, skip_all_groups, + group, skip_group, + group_all_testcases, skip_group_all_testcases, + group_testcase, skip_group_testcase, + topgroup, + subgroup, skip_subgroup, + subgroup_all_testcases, skip_subgroup_all_testcases, + subgroup_testcase, skip_subgroup_testcase, + sub_skipped_by_top, + testcase_in_multiple_groups]. + +%%-------------------------------------------------------------------- +%% TEST CASES +%%-------------------------------------------------------------------- + +%%%----------------------------------------------------------------- +%%% + +all_suites(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "suites_1"), + TestSpec = [{label,"all_suites"}, + {suites,TestDir,all}], + + setup_and_execute(all_suites, TestSpec, Config). + +skip_all_suites(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "suites_1"), + TestSpec = [{label,skip_all_suites}, + {suites,TestDir,all}, + {skip_suites,TestDir,all,"SKIPPED!"}], + + setup_and_execute(skip_all_suites, TestSpec, Config). + +%%%----------------------------------------------------------------- +%%% + +suite(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "suites_1"), + TestSpec = [{label,undefined}, + {suites,TestDir,simple_1_SUITE}], + + setup_and_execute(suite, TestSpec, Config). + +skip_suite(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "suites_1"), + TestSpec = [{suites,TestDir,[simple_1_SUITE,simple_2_SUITE]}, + {skip_suites,TestDir,simple_1_SUITE,"SKIPPED!"}], + + setup_and_execute(skip_suite, TestSpec, Config). + +%%%----------------------------------------------------------------- +%%% + +all_testcases(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "suites_1"), + TestSpec = [{cases,TestDir,simple_1_SUITE,all}], + + setup_and_execute(all_testcases, TestSpec, Config). + +skip_all_testcases(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "suites_1"), + TestSpec = [{suites,TestDir,[simple_1_SUITE]}, + {skip_cases,TestDir,simple_1_SUITE,all,"SKIPPED!"}], + + setup_and_execute(skip_all_testcases, TestSpec, Config). + +%%%----------------------------------------------------------------- +%%% + +testcase(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "suites_1"), + TestSpec = [{cases,TestDir,simple_1_SUITE,tc1}], + + setup_and_execute(testcase, TestSpec, Config). + +skip_testcase(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "suites_1"), + TestSpec = [{cases,TestDir,simple_1_SUITE,[tc1,tc2]}, + {cases,TestDir,simple_2_SUITE,[tc2,tc1]}, + {skip_cases,TestDir,simple_1_SUITE,[tc1],"SKIPPED!"}, + {skip_cases,TestDir,simple_2_SUITE,tc2,"SKIPPED!"}], + + setup_and_execute(skip_testcase, TestSpec, Config). + +%%%----------------------------------------------------------------- +%%% + +all_groups(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_11_SUITE,all}], + + setup_and_execute(all_groups, TestSpec, Config). + +skip_all_groups(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_11_SUITE,all}, + {skip_groups,TestDir,groups_11_SUITE,all,"SKIPPED!"}], + + setup_and_execute(skip_all_groups, TestSpec, Config). + +%%%----------------------------------------------------------------- +%%% + +group(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_11_SUITE,test_group_1a}], + + setup_and_execute(group, TestSpec, Config). + +skip_group(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_11_SUITE,[test_group_1a, + test_group_1b]}, + {skip_groups,TestDir,groups_11_SUITE, + [test_group_1b,test_group_2,test_group_7],"SKIPPED!"}], + + setup_and_execute(skip_group, TestSpec, Config). + + +%%%----------------------------------------------------------------- +%%% + +group_all_testcases(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_11_SUITE,test_group_1a,{cases,all}}], + + setup_and_execute(group_all_testcases, TestSpec, Config). + +skip_group_all_testcases(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_11_SUITE,[test_group_1a, + test_group_1b]}, + {skip_groups,TestDir,groups_11_SUITE, + test_group_1b,{cases,all},"SKIPPED!"}, + {skip_groups,TestDir,groups_11_SUITE, + test_group_1a,{cases,all},"SKIPPED!"}], + + setup_and_execute(skip_group_all_testcases, TestSpec, Config). + +%%%----------------------------------------------------------------- +%%% + +group_testcase(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_11_SUITE,test_group_1a,{cases,testcase_1a}}], + + setup_and_execute(group_testcase, TestSpec, Config). + +skip_group_testcase(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_11_SUITE,test_group_1a, + {cases,[testcase_1a,testcase_1b]}}, + {groups,TestDir,groups_11_SUITE,test_group_1b, + {cases,[testcase_1b,testcase_1a]}}, + {skip_groups,TestDir,groups_11_SUITE, + test_group_1a,{cases,testcase_1b},"SKIPPED!"}, + {skip_groups,TestDir,groups_11_SUITE, + test_group_1b,{cases,[testcase_1a]},"SKIPPED!"}], + + setup_and_execute(skip_group_testcase, TestSpec, Config). + +%%%----------------------------------------------------------------- +%%% + +topgroup(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_12_SUITE,test_group_2}, + {groups,TestDir,groups_12_SUITE,test_group_4}], + + setup_and_execute(topgroup, TestSpec, Config). + +%%%----------------------------------------------------------------- +%%% + +subgroup(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_12_SUITE,test_group_3}], + + setup_and_execute(subgroup, TestSpec, Config). + +skip_subgroup(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_12_SUITE,[test_group_4]}, + {skip_groups,TestDir,groups_12_SUITE, + test_group_8,"SKIPPED!"}], + + setup_and_execute(skip_subgroup, TestSpec, Config). + +%%%----------------------------------------------------------------- +%%% + +subgroup_all_testcases(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_12_SUITE, + test_group_5,{cases,all}}, + {groups,TestDir,groups_12_SUITE, + test_group_3,{cases,all}}], + + setup_and_execute(subgroup_all_testcases, TestSpec, Config). + +skip_subgroup_all_testcases(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_12_SUITE,test_group_4}, + {skip_groups,TestDir,groups_12_SUITE, + test_group_5,{cases,all},"SKIPPED!"}], + + setup_and_execute(skip_subgroup_all_testcases, TestSpec, Config). + +%%%----------------------------------------------------------------- +%%% + +subgroup_testcase(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_12_SUITE, + test_group_7,{cases,testcase_7a}}, + {groups,TestDir,groups_12_SUITE, + test_group_3,{cases,testcase_3b}}], + + setup_and_execute(subgroup_testcase, TestSpec, Config). + +skip_subgroup_testcase(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_12_SUITE,test_group_5}, + {skip_groups,TestDir,groups_12_SUITE, + test_group_7,{cases,[testcase_7a,testcase_7b]},"SKIPPED!"}], + + setup_and_execute(skip_subgroup_testcase, TestSpec, Config). + +%%%----------------------------------------------------------------- +%%% + +%%! +%%! Somewhat weird result from this one: +%%! +sub_skipped_by_top(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{groups,TestDir,groups_12_SUITE,test_group_5}, + {skip_groups,TestDir,groups_12_SUITE,test_group_4,"SKIPPED!"}], + + setup_and_execute(sub_skipped_by_top, TestSpec, Config). + +%%%----------------------------------------------------------------- +%%% + +testcase_in_multiple_groups(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + + TestDir = filename:join(DataDir, "groups_1"), + TestSpec = [{cases,TestDir,groups_12_SUITE,[testcase_1a,testcase_1b]}, + {skip_cases,TestDir,groups_12_SUITE,[testcase_1b],"SKIPPED!"}], + + setup_and_execute(testcase_in_multiple_groups, TestSpec, Config). + +%%%----------------------------------------------------------------- +%%% HELP FUNCTIONS +%%%----------------------------------------------------------------- + +setup_and_execute(TCName, TestSpec, Config) -> + SpecFile = create_spec_file(?config(priv_dir, Config), + TCName, TestSpec), + TestTerms = + case lists:keymember(label, 1, TestSpec) of + true -> [{spec,SpecFile}]; + false -> [{spec,SpecFile},{label,TCName}] + end, + {Opts,ERPid} = setup(TestTerms, Config), + ok = ct_test_support:run(Opts, Config), + TestSpec1 = [{logdir,proplists:get_value(logdir,Opts)}, + {label,proplists:get_value(label,TestTerms)} | TestSpec], + ok = ct_test_support:run(ct, run_testspec, [TestSpec1], Config), + Events = ct_test_support:get_events(ERPid, Config), + + ct_test_support:log_events(TCName, + reformat(Events, ?eh), + ?config(priv_dir, Config)), + + TestEvents = events_to_check(TCName), + ok = ct_test_support:verify_events(TestEvents, Events, Config). + +create_spec_file(SpecDir, TCName, TestSpec) -> + FileName = filename:join(SpecDir, + atom_to_list(TCName)++".spec"), + {ok,Dev} = file:open(FileName, [write]), + [io:format(Dev, "~p.~n", [Term]) || Term <- TestSpec], + file:close(Dev), + io:format("~nTest spec created here~n~n<a href=\"file://~s\">~s</a>~n", + [FileName,FileName]), + FileName. + +setup(Test, Config) when is_tuple(Test) -> + setup([Test], Config); +setup(Tests, Config) -> + Opts0 = ct_test_support:get_opts(Config), + Level = ?config(trace_level, Config), + EvHArgs = [{cbm,ct_test_support},{trace_level,Level}], + Opts = Opts0 ++ Tests ++ [{event_handler,{?eh,EvHArgs}}], + ERPid = ct_test_support:start_event_receiver(Config), + {Opts,ERPid}. + +reformat(Events, EH) -> + ct_test_support:reformat(Events, EH). +%reformat(Events, _EH) -> +% Events. + +%%%----------------------------------------------------------------- +%%% TEST EVENTS +%%%----------------------------------------------------------------- +events_to_check(Test) -> + %% 2 tests (ct:run_test + script_start) is default + events_to_check(Test, 2). + +events_to_check(_, 0) -> + []; +events_to_check(Test, N) -> + test_events(Test) ++ events_to_check(Test, N-1). + +test_events(_) -> + [ + ]. diff --git a/lib/common_test/test/ct_testspec_1_SUITE_data/groups_1/groups_11_SUITE.erl b/lib/common_test/test/ct_testspec_1_SUITE_data/groups_1/groups_11_SUITE.erl new file mode 100644 index 0000000000..4f11d8a0e8 --- /dev/null +++ b/lib/common_test/test/ct_testspec_1_SUITE_data/groups_1/groups_11_SUITE.erl @@ -0,0 +1,281 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2009-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(groups_11_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%==================================================================== +%% COMMON TEST CALLBACK FUNCTIONS +%%==================================================================== + +suite() -> + [{timetrap,{minutes,1}}]. + +groups() -> + [ + {test_group_1a, [testcase_1a,testcase_1b]}, + + {test_group_1b, [], [testcase_1a,testcase_1b]}, + + {test_group_2, [], [testcase_2a, + + {test_group_3, [], [testcase_3a, + testcase_3b]}, + testcase_2b]}, + + {test_group_4, [{test_group_5, [], [testcase_5a, + + {group, test_group_6}, + + testcase_5b]}]}, + {test_group_6, [{group, test_group_7}]}, + + {test_group_7, [testcase_7a,testcase_7b]} + ]. + +all() -> + [testcase_1, + {group, test_group_1a}, + {group, test_group_1b}, + testcase_2, + {group, test_group_2}, + testcase_3, + {group, test_group_4}]. + +%% this func only for internal test purposes +grs_and_tcs() -> + {[ + test_group_1a, test_group_1b, + test_group_2, test_group_3, + test_group_4, test_group_5, + test_group_6, test_group_7 + ], + [ + testcase_1, + testcase_1a, testcase_1b, + testcase_2, + testcase_2a, testcase_2b, + testcase_3a, testcase_3b, + testcase_3, + testcase_5a, testcase_5b, + testcase_7a, testcase_7b + ]}. + +%%-------------------------------------------------------------------- +%% Suite Configuration +%%-------------------------------------------------------------------- + +init_per_suite(Config) -> + [{suite,init}|Config]. + +end_per_suite(Config) -> + init = ?config(suite,Config). + +%%-------------------------------------------------------------------- +%% Group Configuration +%%-------------------------------------------------------------------- + +init_per_group(Group, Config) -> + [{name,Group}] = ?config(tc_group_properties,Config), + {Grs,_} = grs_and_tcs(), + case lists:member(Group, Grs) of + true -> + ct:comment(Group), + init = ?config(suite,Config), + [{Group,Group} | Config]; + false -> + ct:fail({bad_group,Group}) + end. + +end_per_group(Group, Config) -> + {Grs,_} = grs_and_tcs(), + case lists:member(Group, Grs) of + true -> + ct:comment(Group), + init = ?config(suite,Config), + Group = ?config(Group,Config), + ok; + false -> + ct:fail({bad_group,Group}) + end. + +%%-------------------------------------------------------------------- +%% Testcase Configuration +%%-------------------------------------------------------------------- + +init_per_testcase(TestCase, Config) -> + {_,TCs} = grs_and_tcs(), + case lists:member(TestCase, TCs) of + true -> + init = ?config(suite,Config), + [{TestCase,TestCase} | Config]; + false -> + ct:fail({unknown_testcase,TestCase}) + end. + +end_per_testcase(TestCase, Config) -> + {_,TCs} = grs_and_tcs(), + case lists:member(TestCase, TCs) of + true -> + init = ?config(suite,Config), + TestCase = ?config(TestCase,Config), + ok; + false -> + ct:fail({unknown_testcase,TestCase}) + end. + + +%%-------------------------------------------------------------------- +%% Testcases +%%-------------------------------------------------------------------- + +testcase_1() -> + []. +testcase_1(Config) -> + init = ?config(suite,Config), + testcase_1 = ?config(testcase_1,Config), + ok. + +testcase_1a() -> + []. +testcase_1a(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + testcase_1a = ?config(testcase_1a,Config), + ok. +testcase_1b() -> + []. +testcase_1b(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + undefined = ?config(testcase_1a,Config), + testcase_1b = ?config(testcase_1b,Config), + ok. + +testcase_2() -> + []. +testcase_2(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_1a,Config), + undefined = ?config(test_group_1b,Config), + testcase_2 = ?config(testcase_2,Config), + ok. + +testcase_2a() -> + []. +testcase_2a(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + testcase_2a = ?config(testcase_2a,Config), + ok. +testcase_2b() -> + []. +testcase_2b(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + undefined = ?config(testcase_2a,Config), + testcase_2b = ?config(testcase_2b,Config), + ok. + +testcase_3a() -> + []. +testcase_3a(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + test_group_3 = ?config(test_group_3,Config), + undefined = ?config(testcase_2b,Config), + testcase_3a = ?config(testcase_3a,Config), + ok. +testcase_3b() -> + []. +testcase_3b(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + test_group_3 = ?config(test_group_3,Config), + undefined = ?config(testcase_3a,Config), + testcase_3b = ?config(testcase_3b,Config), + ok. + +testcase_3() -> + []. +testcase_3(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_2,Config), + undefined = ?config(test_group_3,Config), + testcase_3 = ?config(testcase_3,Config), + ok. + +testcase_5a() -> + []. +testcase_5a(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_3,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + undefined = ?config(testcase_3,Config), + testcase_5a = ?config(testcase_5a,Config), + ok. +testcase_5b() -> + []. +testcase_5b(Config) -> + init = ?config(suite,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + undefined = ?config(testcase_5a,Config), + testcase_5b = ?config(testcase_5b,Config), + ok. + +testcase_7a() -> + []. +testcase_7a(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_3,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_7 = ?config(test_group_7,Config), + testcase_7a = ?config(testcase_7a,Config), + ok. +testcase_7b() -> + []. +testcase_7b(Config) -> + init = ?config(suite,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_7 = ?config(test_group_7,Config), + undefined = ?config(testcase_7a,Config), + testcase_7b = ?config(testcase_7b,Config), + ok. diff --git a/lib/common_test/test/ct_testspec_1_SUITE_data/groups_1/groups_12_SUITE.erl b/lib/common_test/test/ct_testspec_1_SUITE_data/groups_1/groups_12_SUITE.erl new file mode 100644 index 0000000000..69c06f9b83 --- /dev/null +++ b/lib/common_test/test/ct_testspec_1_SUITE_data/groups_1/groups_12_SUITE.erl @@ -0,0 +1,344 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2009-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(groups_12_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%==================================================================== +%% COMMON TEST CALLBACK FUNCTIONS +%%==================================================================== + +suite() -> + [{timetrap,{minutes,1}}]. + +groups() -> + [ + {test_group_1a, [shuffle], [testcase_1a,testcase_1b,testcase_1c]}, + + {test_group_1b, [parallel], [testcase_1a,testcase_1b]}, + + {test_group_2, [parallel], [testcase_2a, + + {test_group_3, [{repeat,2}], + [testcase_3a, testcase_3b]}, + + testcase_2b]}, + + {test_group_4, [{test_group_5, [parallel], [testcase_5a, + + {group, test_group_6}, + + testcase_5b]}]}, + + {test_group_6, [parallel], [{group, test_group_7}, + {group, test_group_8}]}, + + {test_group_7, [sequence], [testcase_7a,testcase_7b]}, + + {test_group_8, [shuffle,sequence], [testcase_8a,testcase_8b]} + ]. + +all() -> + [{group, test_group_1a}, + {group, test_group_1b}, + testcase_1, + testcase_2, + {group, test_group_2}, + testcase_3, + {group, test_group_4}]. + +%% this func only for internal test purposes +grs_and_tcs() -> + {[ + test_group_1a, test_group_1b, + test_group_2, test_group_3, + test_group_4, test_group_5, + test_group_6, test_group_7, + test_group_8 + ], + [ + testcase_1a, testcase_1b, testcase_1c, + testcase_1, + testcase_2, + testcase_2a, testcase_2b, + testcase_3a, testcase_3b, + testcase_3, + testcase_5a, testcase_5b, + testcase_7a, testcase_7b, + testcase_8a, testcase_8b + ]}. + +%%-------------------------------------------------------------------- +%% Suite Configuration +%%-------------------------------------------------------------------- + +init_per_suite(Config) -> + [{suite,init}|Config]. + +end_per_suite(Config) -> + init = ?config(suite,Config). + +%%-------------------------------------------------------------------- +%% Group Configuration +%%-------------------------------------------------------------------- + +init_per_group(Group, Config) -> + Cmt = + case {Group,?config(tc_group_properties,Config)} of + {test_group_1a,[{shuffle,S},{name,test_group_1a}]} -> + io_lib:format("shuffled, ~w", [S]); + {test_group_1b,[{name,test_group_1b},parallel]} -> "parallel"; + {test_group_2,[{name,test_group_2},parallel]} -> "parallel"; + {test_group_3,[{name,test_group_3},{repeat,2}]} -> "repeat 2"; + {test_group_3,[{name,test_group_3}]} -> "repeat 1"; + {test_group_4,[{name,test_group_4}]} -> ok; + {test_group_5,[{name,test_group_5},parallel]} -> "parallel"; + {test_group_6,[{name,test_group_6},parallel]} -> "parallel"; + {test_group_7,[{name,test_group_7},sequence]} -> "sequence"; + {test_group_8,[{shuffle,_},{name,test_group_8},sequence]} -> + "shuffle & sequence" + end, + {Grs,_} = grs_and_tcs(), + case lists:member(Group, Grs) of + true -> + init = ?config(suite,Config), + ct:comment(io_lib:format("~w, ~s", [Group,Cmt])), + [{Group,Group} | Config]; + false -> + ct:fail({bad_group,Group}) + end. + +end_per_group(Group, Config) -> + {Grs,_} = grs_and_tcs(), + case lists:member(Group, Grs) of + true -> + init = ?config(suite,Config), + Group = ?config(Group,Config), + ok; + false -> + ct:fail({bad_group,Group}) + end. + +%%-------------------------------------------------------------------- +%% Testcase Configuration +%%-------------------------------------------------------------------- + +init_per_testcase(TestCase, Config) -> + {_,TCs} = grs_and_tcs(), + case lists:member(TestCase, TCs) of + true -> + init = ?config(suite,Config), + [{TestCase,TestCase} | Config]; + false -> + ct:fail({unknown_testcase,TestCase}) + end. + +end_per_testcase(TestCase, Config) -> + {_,TCs} = grs_and_tcs(), + case lists:member(TestCase, TCs) of + true -> + init = ?config(suite,Config), + TestCase = ?config(TestCase,Config), + ok; + false -> + ct:fail({unknown_testcase,TestCase}) + end. + + +%%-------------------------------------------------------------------- +%% Testcases +%%-------------------------------------------------------------------- + +testcase_1a() -> + []. +testcase_1a(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + testcase_1a = ?config(testcase_1a,Config), + ok. +testcase_1b() -> + []. +testcase_1b(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + undefined = ?config(testcase_1a,Config), + testcase_1b = ?config(testcase_1b,Config), + ok. + +testcase_1c() -> + []. +testcase_1c(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + undefined = ?config(testcase_1b,Config), + testcase_1c = ?config(testcase_1c,Config), + ok. + +testcase_1() -> + []. +testcase_1(Config) -> + init = ?config(suite,Config), + testcase_1 = ?config(testcase_1,Config), + ok. + +testcase_2() -> + []. +testcase_2(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_1a,Config), + undefined = ?config(test_group_1b,Config), + testcase_2 = ?config(testcase_2,Config), + ok. + +testcase_2a() -> + []. +testcase_2a(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + testcase_2a = ?config(testcase_2a,Config), + ok. +testcase_2b() -> + []. +testcase_2b(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + undefined = ?config(testcase_2a,Config), + testcase_2b = ?config(testcase_2b,Config), + ok. + +testcase_3a() -> + []. +testcase_3a(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + test_group_3 = ?config(test_group_3,Config), + undefined = ?config(testcase_2b,Config), + testcase_3a = ?config(testcase_3a,Config), + ok. +testcase_3b() -> + []. +testcase_3b(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + test_group_3 = ?config(test_group_3,Config), + undefined = ?config(testcase_3a,Config), + testcase_3b = ?config(testcase_3b,Config), + ok. + +testcase_3() -> + []. +testcase_3(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_2,Config), + undefined = ?config(test_group_3,Config), + testcase_3 = ?config(testcase_3,Config), + ok. + +testcase_5a() -> + []. +testcase_5a(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_3,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + undefined = ?config(testcase_3,Config), + testcase_5a = ?config(testcase_5a,Config), + %% increase chance the done event will come + %% during execution of subgroup (could be + %% tricky to handle) + timer:sleep(3), + ok. +testcase_5b() -> + []. +testcase_5b(Config) -> + init = ?config(suite,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + undefined = ?config(testcase_5a,Config), + testcase_5b = ?config(testcase_5b,Config), + ok. + +testcase_7a() -> + []. +testcase_7a(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_3,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_7 = ?config(test_group_7,Config), + testcase_7a = ?config(testcase_7a,Config), + ok. +testcase_7b() -> + []. +testcase_7b(Config) -> + init = ?config(suite,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_7 = ?config(test_group_7,Config), + undefined = ?config(testcase_7a,Config), + testcase_7b = ?config(testcase_7b,Config), + ok. + +testcase_8a() -> + []. +testcase_8a(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_3,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_8 = ?config(test_group_8,Config), + testcase_8a = ?config(testcase_8a,Config), + ok. +testcase_8b() -> + []. +testcase_8b(Config) -> + init = ?config(suite,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_8 = ?config(test_group_8,Config), + undefined = ?config(testcase_8a,Config), + testcase_8b = ?config(testcase_8b,Config), + ok. diff --git a/lib/common_test/test/ct_testspec_1_SUITE_data/groups_2/groups_21_SUITE.erl b/lib/common_test/test/ct_testspec_1_SUITE_data/groups_2/groups_21_SUITE.erl new file mode 100644 index 0000000000..2533ac8e84 --- /dev/null +++ b/lib/common_test/test/ct_testspec_1_SUITE_data/groups_2/groups_21_SUITE.erl @@ -0,0 +1,281 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(groups_21_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%==================================================================== +%% COMMON TEST CALLBACK FUNCTIONS +%%==================================================================== + +suite() -> + [{timetrap,{minutes,1}}]. + +groups() -> + [ + {test_group_1a, [testcase_1a,testcase_1b]}, + + {test_group_1b, [], [testcase_1a,testcase_1b]}, + + {test_group_2, [], [testcase_2a, + + {test_group_3, [], [testcase_3a, + testcase_3b]}, + testcase_2b]}, + + {test_group_4, [{test_group_5, [], [testcase_5a, + + {group, test_group_6}, + + testcase_5b]}]}, + + {test_group_6, [{group, test_group_7}]}, + + {test_group_7, [testcase_7a,testcase_7b]} + ]. + +all() -> + [testcase_1, + {group, test_group_1a}, + {group, test_group_1b}, + testcase_2, + {group, test_group_2}, + testcase_3, + {group, test_group_4}]. + +%% this func only for internal test purposes +grs_and_tcs() -> + {[ + test_group_1a, test_group_1b, + test_group_2, test_group_3, + test_group_4, test_group_5, + test_group_6, test_group_7 + ], + [ + testcase_1, + testcase_1a, testcase_1b, + testcase_2, + testcase_2a, testcase_2b, + testcase_3a, testcase_3b, + testcase_3, + testcase_5a, testcase_5b, + testcase_7a, testcase_7b + ]}. + +%%-------------------------------------------------------------------- +%% Suite Configuration +%%-------------------------------------------------------------------- + +init_per_suite(Config) -> + [{suite,init}|Config]. + +end_per_suite(Config) -> + init = ?config(suite,Config). + +%%-------------------------------------------------------------------- +%% Group Configuration +%%-------------------------------------------------------------------- + +init_per_group(Group, Config) -> + [{name,Group}] = ?config(tc_group_properties,Config), + {Grs,_} = grs_and_tcs(), + case lists:member(Group, Grs) of + true -> + ct:comment(io_lib:format("~w", [Group])), + init = ?config(suite,Config), + [{Group,Group} | Config]; + false -> + ct:fail({bad_group,Group}) + end. + +end_per_group(Group, Config) -> + {Grs,_} = grs_and_tcs(), + case lists:member(Group, Grs) of + true -> + init = ?config(suite,Config), + Group = ?config(Group,Config), + ok; + false -> + ct:fail({bad_group,Group}) + end. + +%%-------------------------------------------------------------------- +%% Testcase Configuration +%%-------------------------------------------------------------------- + +init_per_testcase(TestCase, Config) -> + {_,TCs} = grs_and_tcs(), + case lists:member(TestCase, TCs) of + true -> + init = ?config(suite,Config), + [{TestCase,TestCase} | Config]; + false -> + ct:fail({unknown_testcase,TestCase}) + end. + +end_per_testcase(TestCase, Config) -> + {_,TCs} = grs_and_tcs(), + case lists:member(TestCase, TCs) of + true -> + init = ?config(suite,Config), + TestCase = ?config(TestCase,Config), + ok; + false -> + ct:fail({unknown_testcase,TestCase}) + end. + + +%%-------------------------------------------------------------------- +%% Testcases +%%-------------------------------------------------------------------- + +testcase_1() -> + []. +testcase_1(Config) -> + init = ?config(suite,Config), + testcase_1 = ?config(testcase_1,Config), + ok. + +testcase_1a() -> + []. +testcase_1a(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + testcase_1a = ?config(testcase_1a,Config), + ok. +testcase_1b() -> + []. +testcase_1b(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + undefined = ?config(testcase_1a,Config), + testcase_1b = ?config(testcase_1b,Config), + ok. + +testcase_2() -> + []. +testcase_2(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_1a,Config), + undefined = ?config(test_group_1b,Config), + testcase_2 = ?config(testcase_2,Config), + ok. + +testcase_2a() -> + []. +testcase_2a(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + testcase_2a = ?config(testcase_2a,Config), + ok. +testcase_2b() -> + []. +testcase_2b(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + undefined = ?config(testcase_2a,Config), + testcase_2b = ?config(testcase_2b,Config), + ok. + +testcase_3a() -> + []. +testcase_3a(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + test_group_3 = ?config(test_group_3,Config), + undefined = ?config(testcase_2b,Config), + testcase_3a = ?config(testcase_3a,Config), + ok. +testcase_3b() -> + []. +testcase_3b(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + test_group_3 = ?config(test_group_3,Config), + undefined = ?config(testcase_3a,Config), + testcase_3b = ?config(testcase_3b,Config), + ok. + +testcase_3() -> + []. +testcase_3(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_2,Config), + undefined = ?config(test_group_3,Config), + testcase_3 = ?config(testcase_3,Config), + ok. + +testcase_5a() -> + []. +testcase_5a(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_3,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + undefined = ?config(testcase_3,Config), + testcase_5a = ?config(testcase_5a,Config), + ok. +testcase_5b() -> + []. +testcase_5b(Config) -> + init = ?config(suite,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + undefined = ?config(testcase_5a,Config), + testcase_5b = ?config(testcase_5b,Config), + ok. + +testcase_7a() -> + []. +testcase_7a(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_3,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_7 = ?config(test_group_7,Config), + testcase_7a = ?config(testcase_7a,Config), + ok. +testcase_7b() -> + []. +testcase_7b(Config) -> + init = ?config(suite,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_7 = ?config(test_group_7,Config), + undefined = ?config(testcase_7a,Config), + testcase_7b = ?config(testcase_7b,Config), + ok. diff --git a/lib/common_test/test/ct_testspec_1_SUITE_data/groups_2/groups_22_SUITE.erl b/lib/common_test/test/ct_testspec_1_SUITE_data/groups_2/groups_22_SUITE.erl new file mode 100644 index 0000000000..cd517876df --- /dev/null +++ b/lib/common_test/test/ct_testspec_1_SUITE_data/groups_2/groups_22_SUITE.erl @@ -0,0 +1,314 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +-module(groups_22_SUITE). + +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%==================================================================== +%% COMMON TEST CALLBACK FUNCTIONS +%%==================================================================== + +suite() -> + [{timetrap,{minutes,1}}]. + +groups() -> + [ + {test_group_1a, [shuffle], [testcase_1a,testcase_1b,testcase_1c]}, + + {test_group_1b, [parallel], [testcase_1a,testcase_1b]}, + + {test_group_2, [parallel], [testcase_2a, + + {test_group_3, [{repeat,1}], + [testcase_3a, testcase_3b]}, + + testcase_2b]}, + + {test_group_4, [{test_group_5, [parallel], [testcase_5a, + + {group, test_group_6}, + + testcase_5b]}]}, + + {test_group_6, [parallel], [{group, test_group_7}]}, + + {test_group_7, [sequence], [testcase_7a,testcase_7b]} + ]. + +all() -> + [{group, test_group_1a}, + {group, test_group_1b}, + testcase_1, + testcase_2, + {group, test_group_2}, + testcase_3, + {group, test_group_4}]. + +%% this func only for internal test purposes +grs_and_tcs() -> + {[ + test_group_1a, test_group_1b, + test_group_2, test_group_3, + test_group_4, test_group_5, + test_group_6, test_group_7 + ], + [ + testcase_1a, testcase_1b, testcase_1c, + testcase_1, + testcase_2, + testcase_2a, testcase_2b, + testcase_3a, testcase_3b, + testcase_3, + testcase_5a, testcase_5b, + testcase_7a, testcase_7b + ]}. + +%%-------------------------------------------------------------------- +%% Suite Configuration +%%-------------------------------------------------------------------- + +init_per_suite(Config) -> + [{suite,init}|Config]. + +end_per_suite(Config) -> + init = ?config(suite,Config). + +%%-------------------------------------------------------------------- +%% Group Configuration +%%-------------------------------------------------------------------- + +init_per_group(Group, Config) -> + Cmt = + case {Group,?config(tc_group_properties,Config)} of + {test_group_1a,[{shuffle,S},{name,test_group_1a}]} -> + io_lib:format("shuffled, ~w", [S]); + {test_group_1b,[{name,test_group_1b},parallel]} -> "parallel"; + {test_group_2,[{name,test_group_2},parallel]} -> "parallel"; + {test_group_3,[{name,test_group_3},{repeat,1}]} -> "repeat 1"; + {test_group_3,[{name,test_group_3}]} -> "repeat 0"; + {test_group_4,[{name,test_group_4}]} -> ok; + {test_group_5,[{name,test_group_5},parallel]} -> "parallel"; + {test_group_6,[{name,test_group_6},parallel]} -> "parallel"; + {test_group_7,[{name,test_group_7},sequence]} -> "sequence" + end, + {Grs,_} = grs_and_tcs(), + case lists:member(Group, Grs) of + true -> + init = ?config(suite,Config), + ct:comment(io_lib:format("~w, ~s", [Group,Cmt])), + [{Group,Group} | Config]; + false -> + ct:fail({bad_group,Group}) + end. + +end_per_group(Group, Config) -> + {Grs,_} = grs_and_tcs(), + case lists:member(Group, Grs) of + true -> + init = ?config(suite,Config), + Group = ?config(Group,Config), + ok; + false -> + ct:fail({bad_group,Group}) + end. + +%%-------------------------------------------------------------------- +%% Testcase Configuration +%%-------------------------------------------------------------------- + +init_per_testcase(TestCase, Config) -> + {_,TCs} = grs_and_tcs(), + case lists:member(TestCase, TCs) of + true -> + init = ?config(suite,Config), + [{TestCase,TestCase} | Config]; + false -> + ct:fail({unknown_testcase,TestCase}) + end. + +end_per_testcase(TestCase, Config) -> + {_,TCs} = grs_and_tcs(), + case lists:member(TestCase, TCs) of + true -> + init = ?config(suite,Config), + TestCase = ?config(TestCase,Config), + ok; + false -> + ct:fail({unknown_testcase,TestCase}) + end. + + +%%-------------------------------------------------------------------- +%% Testcases +%%-------------------------------------------------------------------- + +testcase_1a() -> + []. +testcase_1a(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + testcase_1a = ?config(testcase_1a,Config), + ok. +testcase_1b() -> + []. +testcase_1b(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + undefined = ?config(testcase_1a,Config), + testcase_1b = ?config(testcase_1b,Config), + ok. + +testcase_1c() -> + []. +testcase_1c(Config) -> + init = ?config(suite,Config), + case ?config(test_group_1a,Config) of + test_group_1a -> ok; + _ -> + case ?config(test_group_1b,Config) of + test_group_1b -> ok; + _ -> ct:fail(no_group_data) + end + end, + undefined = ?config(testcase_1b,Config), + testcase_1c = ?config(testcase_1c,Config), + ok. + +testcase_1() -> + []. +testcase_1(Config) -> + init = ?config(suite,Config), + testcase_1 = ?config(testcase_1,Config), + ok. + +testcase_2() -> + []. +testcase_2(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_1a,Config), + undefined = ?config(test_group_1b,Config), + testcase_2 = ?config(testcase_2,Config), + ok. + +testcase_2a() -> + []. +testcase_2a(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + testcase_2a = ?config(testcase_2a,Config), + ok. +testcase_2b() -> + []. +testcase_2b(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + undefined = ?config(testcase_2a,Config), + testcase_2b = ?config(testcase_2b,Config), + ok. + +testcase_3a() -> + []. +testcase_3a(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + test_group_3 = ?config(test_group_3,Config), + undefined = ?config(testcase_2b,Config), + testcase_3a = ?config(testcase_3a,Config), + ok. +testcase_3b() -> + []. +testcase_3b(Config) -> + init = ?config(suite,Config), + test_group_2 = ?config(test_group_2,Config), + test_group_3 = ?config(test_group_3,Config), + undefined = ?config(testcase_3a,Config), + testcase_3b = ?config(testcase_3b,Config), + ok. + +testcase_3() -> + []. +testcase_3(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_2,Config), + undefined = ?config(test_group_3,Config), + testcase_3 = ?config(testcase_3,Config), + ok. + +testcase_5a() -> + []. +testcase_5a(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_3,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + undefined = ?config(testcase_3,Config), + testcase_5a = ?config(testcase_5a,Config), + %% increase chance the done event will come + %% during execution of subgroup (could be + %% tricky to handle) + timer:sleep(3), + ok. +testcase_5b() -> + []. +testcase_5b(Config) -> + init = ?config(suite,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + undefined = ?config(testcase_5a,Config), + testcase_5b = ?config(testcase_5b,Config), + ok. + +testcase_7a() -> + []. +testcase_7a(Config) -> + init = ?config(suite,Config), + undefined = ?config(test_group_3,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_7 = ?config(test_group_7,Config), + testcase_7a = ?config(testcase_7a,Config), + ok. +testcase_7b() -> + []. +testcase_7b(Config) -> + init = ?config(suite,Config), + test_group_4 = ?config(test_group_4,Config), + test_group_5 = ?config(test_group_5,Config), + test_group_6 = ?config(test_group_6,Config), + test_group_7 = ?config(test_group_7,Config), + undefined = ?config(testcase_7a,Config), + testcase_7b = ?config(testcase_7b,Config), + ok. diff --git a/lib/common_test/test/ct_testspec_1_SUITE_data/suites_1/simple_1_SUITE.erl b/lib/common_test/test/ct_testspec_1_SUITE_data/suites_1/simple_1_SUITE.erl new file mode 100644 index 0000000000..b789851134 --- /dev/null +++ b/lib/common_test/test/ct_testspec_1_SUITE_data/suites_1/simple_1_SUITE.erl @@ -0,0 +1,146 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(simple_1_SUITE). + +%% Note: This directive should only be used in test suites. +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% COMMON TEST CALLBACK FUNCTIONS +%%-------------------------------------------------------------------- + +%%-------------------------------------------------------------------- +%% Function: suite() -> Info +%% +%% Info = [tuple()] +%% List of key/value pairs. +%% +%% Description: Returns list of tuples to set default properties +%% for the suite. +%% +%% Note: The suite/0 function is only meant to be used to return +%% default data values, not perform any other operations. +%%-------------------------------------------------------------------- +suite() -> + [ + {timetrap,{seconds,10}} + ]. + +%%-------------------------------------------------------------------- +%% Function: init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% Reason = term() +%% The reason for skipping the suite. +%% +%% Description: Initialization before the suite. +%% +%% Note: This function is free to add any key/value pairs to the Config +%% variable, but should NOT alter/remove any existing entries. +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + [{ips,ips_data} | Config]. + +%%-------------------------------------------------------------------- +%% Function: end_per_suite(Config0) -> void() | {save_config,Config1} +%% +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% +%% Description: Cleanup after the suite. +%%-------------------------------------------------------------------- +end_per_suite(Config) -> + ips_data = ?config(ips, Config). + +%%-------------------------------------------------------------------- +%% Function: init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% +%% TestCase = atom() +%% Name of the test case that is about to run. +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% Reason = term() +%% The reason for skipping the test case. +%% +%% Description: Initialization before each test case. +%% +%% Note: This function is free to add any key/value pairs to the Config +%% variable, but should NOT alter/remove any existing entries. +%%-------------------------------------------------------------------- +init_per_testcase(TestCase, Config) -> + [{TestCase,{TestCase,data}} | Config]. + +%%-------------------------------------------------------------------- +%% Function: end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} +%% +%% TestCase = atom() +%% Name of the test case that is finished. +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% +%% Description: Cleanup after each test case. +%%-------------------------------------------------------------------- +end_per_testcase(TestCase, Config) -> + {TestCase,data} = ?config(TestCase, Config). + +%%-------------------------------------------------------------------- +%% Function: all() -> TestCases | {skip,Reason} +%% +%% TestCases = [TestCase | {sequence,SeqName}] +%% TestCase = atom() +%% Name of a test case. +%% SeqName = atom() +%% Name of a test case sequence. +%% Reason = term() +%% The reason for skipping all test cases. +%% +%% Description: Returns the list of test cases that are to be executed. +%%-------------------------------------------------------------------- +all() -> + [tc1, + tc2]. + + +%%-------------------------------------------------------------------- +%% TEST CASES +%%-------------------------------------------------------------------- + +tc1() -> + [{userdata,{info, "This is a testcase"}}]. + +tc1(Config) -> + ips_data = ?config(ips, Config), + {tc1,data} = ?config(tc1, Config), + ok. + +tc2() -> + [{timetrap,5000}]. + +tc2(Config) -> + ips_data = ?config(ips, Config), + undefined = ?config(tc1, Config), + {tc2,data} = ?config(tc2, Config), + ok. diff --git a/lib/common_test/test/ct_testspec_1_SUITE_data/suites_1/simple_2_SUITE.erl b/lib/common_test/test/ct_testspec_1_SUITE_data/suites_1/simple_2_SUITE.erl new file mode 100644 index 0000000000..eb7e9cdf7b --- /dev/null +++ b/lib/common_test/test/ct_testspec_1_SUITE_data/suites_1/simple_2_SUITE.erl @@ -0,0 +1,146 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(simple_2_SUITE). + +%% Note: This directive should only be used in test suites. +-compile(export_all). + +-include_lib("common_test/include/ct.hrl"). + +%%-------------------------------------------------------------------- +%% COMMON TEST CALLBACK FUNCTIONS +%%-------------------------------------------------------------------- + +%%-------------------------------------------------------------------- +%% Function: suite() -> Info +%% +%% Info = [tuple()] +%% List of key/value pairs. +%% +%% Description: Returns list of tuples to set default properties +%% for the suite. +%% +%% Note: The suite/0 function is only meant to be used to return +%% default data values, not perform any other operations. +%%-------------------------------------------------------------------- +suite() -> + [ + {timetrap,{seconds,10}} + ]. + +%%-------------------------------------------------------------------- +%% Function: init_per_suite(Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% Reason = term() +%% The reason for skipping the suite. +%% +%% Description: Initialization before the suite. +%% +%% Note: This function is free to add any key/value pairs to the Config +%% variable, but should NOT alter/remove any existing entries. +%%-------------------------------------------------------------------- +init_per_suite(Config) -> + [{ips,ips_data} | Config]. + +%%-------------------------------------------------------------------- +%% Function: end_per_suite(Config0) -> void() | {save_config,Config1} +%% +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% +%% Description: Cleanup after the suite. +%%-------------------------------------------------------------------- +end_per_suite(Config) -> + ips_data = ?config(ips, Config). + +%%-------------------------------------------------------------------- +%% Function: init_per_testcase(TestCase, Config0) -> +%% Config1 | {skip,Reason} | {skip_and_save,Reason,Config1} +%% +%% TestCase = atom() +%% Name of the test case that is about to run. +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% Reason = term() +%% The reason for skipping the test case. +%% +%% Description: Initialization before each test case. +%% +%% Note: This function is free to add any key/value pairs to the Config +%% variable, but should NOT alter/remove any existing entries. +%%-------------------------------------------------------------------- +init_per_testcase(TestCase, Config) -> + [{TestCase,{TestCase,data}} | Config]. + +%%-------------------------------------------------------------------- +%% Function: end_per_testcase(TestCase, Config0) -> +%% void() | {save_config,Config1} +%% +%% TestCase = atom() +%% Name of the test case that is finished. +%% Config0 = Config1 = [tuple()] +%% A list of key/value pairs, holding the test case configuration. +%% +%% Description: Cleanup after each test case. +%%-------------------------------------------------------------------- +end_per_testcase(TestCase, Config) -> + {TestCase,data} = ?config(TestCase, Config). + +%%-------------------------------------------------------------------- +%% Function: all() -> TestCases | {skip,Reason} +%% +%% TestCases = [TestCase | {sequence,SeqName}] +%% TestCase = atom() +%% Name of a test case. +%% SeqName = atom() +%% Name of a test case sequence. +%% Reason = term() +%% The reason for skipping all test cases. +%% +%% Description: Returns the list of test cases that are to be executed. +%%-------------------------------------------------------------------- +all() -> + [tc1, + tc2]. + + +%%-------------------------------------------------------------------- +%% TEST CASES +%%-------------------------------------------------------------------- + +tc1() -> + [{userdata,{info, "This is a testcase"}}]. + +tc1(Config) -> + ips_data = ?config(ips, Config), + {tc1,data} = ?config(tc1, Config), + ok. + +tc2() -> + [{timetrap,5000}]. + +tc2(Config) -> + ips_data = ?config(ips, Config), + undefined = ?config(tc1, Config), + {tc2,data} = ?config(tc2, Config), + ok. diff --git a/lib/common_test/test/ct_userconfig_callback.erl b/lib/common_test/test/ct_userconfig_callback.erl new file mode 100644 index 0000000000..ca51bf240b --- /dev/null +++ b/lib/common_test/test/ct_userconfig_callback.erl @@ -0,0 +1,32 @@ +%%-------------------------------------------------------------------- +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +-module(ct_userconfig_callback). + +-export([check_parameter/1, read_config/1]). + +read_config(Str) -> + KeyVals = string:tokens(Str, " "), + {ok,read_config1(KeyVals)}. + +read_config1([Key,Val | KeyVals]) -> + [{list_to_atom(Key),Val} | read_config1(KeyVals)]; +read_config1([]) -> + []. + +check_parameter(Str) -> + {ok,{config,Str}}. diff --git a/lib/common_test/vsn.mk b/lib/common_test/vsn.mk index cdb8e1f71c..413ef21df3 100644 --- a/lib/common_test/vsn.mk +++ b/lib/common_test/vsn.mk @@ -1,3 +1,3 @@ -COMMON_TEST_VSN = 1.4.8 +COMMON_TEST_VSN = 1.5.1 diff --git a/lib/compiler/doc/src/notes.xml b/lib/compiler/doc/src/notes.xml index 7ea000a895..c08839bc7b 100644 --- a/lib/compiler/doc/src/notes.xml +++ b/lib/compiler/doc/src/notes.xml @@ -31,6 +31,106 @@ <p>This document describes the changes made to the Compiler application.</p> +<section><title>Compiler 4.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Several problems in the inliner have been fixed.</p> + <p> + Own Id: OTP-8552</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The module binary from EEP31 (and EEP9) is implemented.</p> + <p> + Own Id: OTP-8217</p> + </item> + <item> + <p>Local and imported functions now override the + auto-imported BIFs when the names clash. The pre R14 + behaviour was that auto-imported BIFs would override + local functions. To avoid that old programs change + behaviour, the following will generate an error:</p> + <list><item><p>Doing a call without explicit module name + to a local function having a name clashing with the name + of an auto-imported BIF that was present (and + auto-imported) before OTP R14A</p></item> + <item><p>Explicitly importing a function having a name + clashing with the name of an autoimported BIF that was + present (and autoimported) before OTP R14A</p></item> + <item><p>Using any form of the old compiler directive + <c>nowarn_bif_clash</c></p></item> </list> <p>If the BIF + was added or auto-imported in OTP R14A or later, + overriding it with an import or a local function will + only result in a warning,</p> <p>To resolve clashes, you + can either use the explicit module name <c>erlang</c> to + call the BIF, or you can remove the auto-import of that + specific BIF by using the new compiler directive + <c>-compile({no_auto_import,[F/A]}).</c>, which makes all + calls to the local or imported function without explicit + module name pass without warnings or errors.</p> <p>The + change makes it possible to add auto-imported BIFs + without breaking or silently changing old code in the + future. However some current code ingeniously utilizing + the old behaviour or the <c>nowarn_bif_clash</c> compiler + directive, might need changing to be accepted by the + compiler.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8579</p> + </item> + <item> + <p>The undocumented, unsupport, and deprecated function + <c>lists:flat_length/1</c> has been removed.</p> + <p> + Own Id: OTP-8584</p> + </item> + <item> + <p>Nested records can now be accessed without + parenthesis. See the Reference Manual for examples. + (Thanks to YAMASHINA Hio and Tuncer Ayaz.)</p> + <p> + Own Id: OTP-8597</p> + </item> + <item> + <p>It is now possible to suppress the warning in code + such as "<c>list_to_integer(S), ok</c>" by assigning the + ignored value "_" like this: "<c>_ = list_to_integer(S), + ok</c>".</p> + <p> + Own Id: OTP-8602</p> + </item> + <item> + <p><c>receive</c> statements that can only read out a + newly created reference are now specially optimized so + that it will execute in constant time regardless of the + number of messages in the receive queue for the process. + That optimization will benefit calls to + <c>gen_server:call()</c>. (See <c>gen:do_call/4</c> for + an example of a receive statement that will be + optimized.)</p> + <p> + Own Id: OTP-8623</p> + </item> + <item> + <p>The compiler optimizes record operations better.</p> + <p> + Own Id: OTP-8668</p> + </item> + </list> + </section> + +</section> + <section><title>Compiler 4.6.5</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/compiler/src/beam_dict.erl b/lib/compiler/src/beam_dict.erl index 4ffe8bc606..9164259a94 100644 --- a/lib/compiler/src/beam_dict.erl +++ b/lib/compiler/src/beam_dict.erl @@ -33,7 +33,7 @@ exports = [] :: [{label(), arity(), label()}], locals = [] :: [{label(), arity(), label()}], imports = gb_trees:empty() :: gb_tree(), %{{M,F,A},Index} - strings = [] :: [string()], %String pool + strings = [] :: string(), %String pool lambdas = [], %[{...}] literals = dict:new() :: dict(), %Format: {Literal,Number} next_atom = 1 :: pos_integer(), @@ -219,7 +219,7 @@ my_term_to_binary(Term) -> %% Search for string Str in the string pool Pool. %% old_string(Str, Pool) -> none | Index --spec old_string(string(), [string()]) -> 'none' | pos_integer(). +-spec old_string(string(), string()) -> 'none' | pos_integer(). old_string([C|Str]=Str0, [C|Pool]) -> case lists:prefix(Str, Pool) of diff --git a/lib/compiler/src/compile.erl b/lib/compiler/src/compile.erl index 4642fb68b3..ed7a9144a8 100644 --- a/lib/compiler/src/compile.erl +++ b/lib/compiler/src/compile.erl @@ -41,8 +41,7 @@ -type option() :: atom() | {atom(), term()} | {'d', atom(), term()}. --type line() :: integer(). --type err_info() :: {line(), module(), term()}. %% ErrorDescriptor +-type err_info() :: {erl_scan:line(), module(), term()}. %% ErrorDescriptor -type errors() :: [{file:filename(), [err_info()]}]. -type warnings() :: [{file:filename(), [err_info()]}]. -type mod_ret() :: {'ok', module()} @@ -70,7 +69,7 @@ file(File) -> file(File, ?DEFAULT_OPTIONS). --spec file(module() | file:filename(), [option()]) -> comp_ret(). +-spec file(module() | file:filename(), [option()] | option()) -> comp_ret(). file(File, Opts) when is_list(Opts) -> do_compile({file,File}, Opts++env_default_opts()); @@ -88,6 +87,8 @@ forms(Forms, Opt) when is_atom(Opt) -> %% would have generated a Beam file, false otherwise (if only a binary or a %% listing file would have been generated). +-spec output_generated([option()]) -> boolean(). + output_generated(Opts) -> noenv_output_generated(Opts++env_default_opts()). @@ -96,6 +97,8 @@ output_generated(Opts) -> %% for default options. %% +-spec noenv_file(module() | file:filename(), [option()] | option()) -> comp_ret(). + noenv_file(File, Opts) when is_list(Opts) -> do_compile({file,File}, Opts); noenv_file(File, Opt) -> @@ -106,6 +109,8 @@ noenv_forms(Forms, Opts) when is_list(Opts) -> noenv_forms(Forms, Opt) when is_atom(Opt) -> noenv_forms(Forms, [Opt|?DEFAULT_OPTIONS]). +-spec noenv_output_generated([option()]) -> boolean(). + noenv_output_generated(Opts) -> any(fun ({save_binary,_F}) -> true; (_Other) -> false @@ -216,16 +221,16 @@ format_error({module_name,Mod,Filename}) -> [Mod,Filename]). %% The compile state record. --record(compile, {filename="", - dir="", - base="", - ifile="", - ofile="", +-record(compile, {filename="" :: file:filename(), + dir="" :: file:filename(), + base="" :: file:filename(), + ifile="" :: file:filename(), + ofile="" :: file:filename(), module=[], code=[], core_code=[], abstract_code=[], %Abstract code for debugger. - options=[], + options=[] :: [option()], errors=[], warnings=[]}). @@ -362,7 +367,7 @@ mpf(Ms) -> [{File,[M || {F,M} <- Ms, F =:= File]} || File <- lists:usort([F || {F,_} <- Ms])]. -%% passes(form|file, [Option]) -> [{Name,PassFun}] +%% passes(forms|file, [Option]) -> [{Name,PassFun}] %% Figure out which passes that need to be run. passes(forms, Opts) -> @@ -828,7 +833,6 @@ get_core_transforms(Opts) -> [M || {core_transform,M} <- Opts]. core_transforms(St) -> %% The options field holds the complete list of options at this - Ts = get_core_transforms(St#compile.options), foldl_core_transforms(St, Ts). @@ -1172,12 +1176,12 @@ write_binary(Name, Bin, St) -> %% report_errors(State) -> ok %% report_warnings(State) -> ok -report_errors(St) -> - case member(report_errors, St#compile.options) of +report_errors(#compile{options=Opts,errors=Errors}) -> + case member(report_errors, Opts) of true -> foreach(fun ({{F,_L},Eds}) -> list_errors(F, Eds); ({F,Eds}) -> list_errors(F, Eds) end, - St#compile.errors); + Errors); false -> ok end. diff --git a/lib/compiler/vsn.mk b/lib/compiler/vsn.mk index a5e6de7b5f..47feab5fe1 100644 --- a/lib/compiler/vsn.mk +++ b/lib/compiler/vsn.mk @@ -1 +1 @@ -COMPILER_VSN = 4.6.5 +COMPILER_VSN = 4.7 diff --git a/lib/crypto/doc/src/notes.xml b/lib/crypto/doc/src/notes.xml index 6b9d1f56f1..79798331ca 100644 --- a/lib/crypto/doc/src/notes.xml +++ b/lib/crypto/doc/src/notes.xml @@ -30,6 +30,43 @@ </header> <p>This document describes the changes made to the Crypto application.</p> +<section><title>Crypto 2.0</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + crypto application changed to use NIFs instead of driver.</p> + <p> + Own Id: OTP-8333</p> + </item> + <item> + <p> + des_ecb_encrypt/2 and des_ecb_decrypt/2 has been added to + the crypto module. The crypto:md4/1 function has been + documented.</p> + <p> + Own Id: OTP-8551</p> + </item> + <item> + <p>The undocumented, unsupport, and deprecated function + <c>lists:flat_length/1</c> has been removed.</p> + <p> + Own Id: OTP-8584</p> + </item> + <item> + <p> + New variants of <c>crypto:dss_sign</c> and + <c>crypto:dss_verify</c> with an extra argument to + control how the digest is calculated.</p> + <p> + Own Id: OTP-8700</p> + </item> + </list> + </section> + +</section> + <section><title>Crypto 1.6.4</title> <section><title>Improvements and New Features</title> diff --git a/lib/crypto/src/crypto.erl b/lib/crypto/src/crypto.erl index 39512d27e1..19fa495b7d 100644 --- a/lib/crypto/src/crypto.erl +++ b/lib/crypto/src/crypto.erl @@ -195,8 +195,8 @@ sha_final(_Context) -> ?nif_stub. %% %% MD5_MAC %% --spec md5_mac(iodata(), iodata()) -> binary. --spec md5_mac_96(iodata(), iodata()) -> binary. +-spec md5_mac(iodata(), iodata()) -> binary(). +-spec md5_mac_96(iodata(), iodata()) -> binary(). md5_mac(Key, Data) -> md5_mac_n(Key,Data,16). @@ -209,8 +209,8 @@ md5_mac_n(_Key,_Data,_MacSz) -> ?nif_stub. %% %% SHA_MAC %% --spec sha_mac(iodata(), iodata()) -> binary. --spec sha_mac_96(iodata(), iodata()) -> binary. +-spec sha_mac(iodata(), iodata()) -> binary(). +-spec sha_mac_96(iodata(), iodata()) -> binary(). sha_mac(Key, Data) -> sha_mac_n(Key,Data,20). diff --git a/lib/debugger/doc/src/notes.xml b/lib/debugger/doc/src/notes.xml index afd49e8593..c5f611b8f3 100644 --- a/lib/debugger/doc/src/notes.xml +++ b/lib/debugger/doc/src/notes.xml @@ -32,6 +32,30 @@ <p>This document describes the changes made to the Debugger application.</p> +<section><title>Debugger 3.2.3</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Warnings due to new autoimported BIFs removed</p> + <p> + Own Id: OTP-8674 Aux Id: OTP-8579 </p> + </item> + <item> + <p> + The predefined builtin type tid() has been removed. + Instead, ets:tid() should be used.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8687</p> + </item> + </list> + </section> + +</section> + <section><title>Debugger 3.2.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/debugger/src/dbg_icmd.erl b/lib/debugger/src/dbg_icmd.erl index 7ccb9793a3..a26b16c82d 100644 --- a/lib/debugger/src/dbg_icmd.erl +++ b/lib/debugger/src/dbg_icmd.erl @@ -94,7 +94,7 @@ break_p(Mod, Line, Le, Bs) -> Bool = case Cond of null -> true; {CM, CN} -> - try apply(CM, CN, [Bs]) of + try CM:CN(Bs) of true -> true; false -> false; _Term -> false @@ -245,7 +245,7 @@ handle_int_msg({attached, AttPid}, Status, _Bs, %% Tell attached process in which module evalution is located if - Le==1 -> + Le =:= 1 -> tell_attached({attached, undefined, -1, get(trace)}); true -> tell_attached({attached, M, Line, get(trace)}), @@ -269,7 +269,7 @@ handle_int_msg(detached, _Status, _Bs, _Ieval) -> handle_int_msg({old_code,Mod}, Status, Bs, #ieval{level=Le,module=M}=Ieval) -> if - Status==idle, Le==1 -> + Status =:= idle, Le =:= 1 -> erase([Mod|db]), put(cache, []); true -> @@ -352,9 +352,9 @@ set_stack_trace(true) -> set_stack_trace(all); set_stack_trace(Flag) -> if - Flag==false -> + Flag =:= false -> put(stack, []); - Flag==no_tail; Flag==all -> + Flag =:= no_tail; Flag =:= all -> ignore end, put(trace_stack, Flag), @@ -367,7 +367,7 @@ bindings(Bs, nostack) -> Bs; bindings(Bs, SP) -> case dbg_ieval:stack_level() of - Le when SP>Le -> + Le when SP > Le -> Bs; _ -> dbg_ieval:bindings(SP) @@ -377,7 +377,6 @@ messages() -> {messages, Msgs} = erlang:process_info(get(self), messages), Msgs. - %%==================================================================== %% Evaluating expressions within process context %%==================================================================== @@ -398,7 +397,7 @@ eval_restricted({From,_Mod,Cmd,SP}, Bs) -> From ! {self(), {eval_rsp, Rsp}} end. -eval_nonrestricted({From,Mod,Cmd,SP}, Bs, #ieval{level=Le}) when SP<Le-> +eval_nonrestricted({From,Mod,Cmd,SP}, Bs, #ieval{level=Le}) when SP < Le-> %% Evaluate in stack eval_restricted({From, Mod, Cmd, SP}, Bs), Bs; @@ -424,15 +423,15 @@ eval_nonrestricted({From, _Mod, Cmd, _SP}, Bs, eval_nonrestricted_1({match,_,{var,_,Var},Expr}, Bs, Ieval) -> {value,Res,Bs2} = dbg_ieval:eval_expr(Expr, Bs, Ieval#ieval{last_call=false}), - Bs3 = case lists:keysearch(Var, 1, Bs) of - {value, {Var,_Value}} -> + Bs3 = case lists:keyfind(Var, 1, Bs) of + {Var,_Value} -> lists:keyreplace(Var, 1, Bs2, {Var,Res}); false -> [{Var,Res} | Bs2] end, {Res,Bs3}; eval_nonrestricted_1({var,_,Var}, Bs, _Ieval) -> - Res = case lists:keysearch(Var, 1, Bs) of - {value, {Var, Value}} -> Value; + Res = case lists:keyfind(Var, 1, Bs) of + {Var, Value} -> Value; false -> unbound end, {Res,Bs}; @@ -458,7 +457,6 @@ parse_cmd(Cmd, LineNo) -> {ok,Forms} = erl_parse:parse_exprs(Tokens), Forms. - %%==================================================================== %% Library functions for attached process handling %%==================================================================== @@ -470,13 +468,12 @@ tell_attached(Msg) -> AttPid ! {self(), Msg} end. - %%==================================================================== %% get_binding/2 %%==================================================================== get_binding(Var, Bs) -> - case lists:keysearch(Var, 1, Bs) of - {value, {Var, Value}} -> {value, Value}; + case lists:keyfind(Var, 1, Bs) of + {Var, Value} -> {value, Value}; false -> unbound end. diff --git a/lib/debugger/src/dbg_ieval.erl b/lib/debugger/src/dbg_ieval.erl index c13fda7ac1..476dfd8796 100644 --- a/lib/debugger/src/dbg_ieval.erl +++ b/lib/debugger/src/dbg_ieval.erl @@ -120,9 +120,9 @@ check_exit_msg({'EXIT', Int, Reason}, _Bs, #ieval{level=Le}) -> %% This *must* be interpreter which has terminated, %% we are not linked to anyone else if - Le==1 -> + Le =:= 1 -> exit(Reason); - Le>1 -> + Le > 1 -> exit({Int, Reason}) end; check_exit_msg({'DOWN',_,_,_,Reason}, Bs, @@ -139,9 +139,9 @@ check_exit_msg({'DOWN',_,_,_,Reason}, Bs, %% importance in this case %% If we don't save them, however, post-mortem analysis %% of the process isn't possible - undefined when Le==1 -> % died outside interpreted code + undefined when Le =:= 1 -> % died outside interpreted code {}; - undefined when Le>1 -> + undefined when Le > 1 -> StackBin = term_to_binary(get(stack)), {{Mod, Li}, Bs, StackBin}; @@ -152,9 +152,9 @@ check_exit_msg({'DOWN',_,_,_,Reason}, Bs, dbg_iserver:cast(get(int), {set_exit_info,self(),ExitInfo}), if - Le==1 -> + Le =:= 1 -> exit(Reason); - Le>1 -> + Le > 1 -> exit({get(self), Reason}) end; check_exit_msg(_Msg, _Bs, _Ieval) -> @@ -271,7 +271,7 @@ meta_loop(Debugged, Bs, #ieval{level=Le} = Ieval) -> end; %% Re-entry to Meta from non-interpreted code - {re_entry, Debugged, {eval,{M,F,As}}} when Le==1 -> + {re_entry, Debugged, {eval,{M,F,As}}} when Le =:= 1 -> %% Reset process dictionary %% This is really only necessary if the process left %% interpreted code at a call level > 1 @@ -346,7 +346,7 @@ push(MFA, Bs, #ieval{level=Le,module=Cm,line=Li,last_call=Lc}) -> [] -> put(stack, [Entry]); [_Entry|Entries] -> put(stack, [Entry|Entries]) end; - _ -> % all | no_tail when Lc==false + _ -> % all | no_tail when Lc =:= false put(stack, [Entry|get(stack)]) end. @@ -413,10 +413,10 @@ sublist(L, Start, Length) -> lists:sublist(L, Start, Length). fix_stacktrace2([{_,{{M,F,As1},_,_}}, {_,{{M,F,As2},_,_}}|_]) - when length(As1)==length(As2) -> + when length(As1) =:= length(As2) -> [{M,F,As1}]; fix_stacktrace2([{_,{{Fun,As1},_,_}}, {_,{{Fun,As2},_,_}}|_]) - when length(As1)==length(As2) -> + when length(As1) =:= length(As2) -> [{Fun,As1}]; fix_stacktrace2([{_,{MFA,_,_}}|Entries]) -> [MFA|fix_stacktrace2(Entries)]; @@ -465,9 +465,9 @@ stack_frame(SP, Dir, [{SP, _}|Stack]) -> case Stack of [{Le, {_MFA,Where,Bs}}|_] -> {Le, Where, Bs}; - [] when Dir==up -> + [] when Dir =:= up -> top; - [] when Dir==down -> + [] when Dir =:= down -> bottom end; stack_frame(SP, Dir, [_Entry|Stack]) -> @@ -509,7 +509,7 @@ trace(What, Args, true) -> end, io_lib:format(" (~w) receive " ++ Tail, [Le]); - received when Args==null -> + received when Args =:= null -> io_lib:format("~n", []); received -> % Args=Msg io_lib:format("~n<== ~p~n", [Args]); @@ -586,8 +586,8 @@ eval_mfa(Debugged, M, F, As, Ieval) -> end. eval_function(Mod, Fun, As0, Bs0, _Called, Ieval) when is_function(Fun); - Mod==?MODULE, - Fun==eval_fun -> + Mod =:= ?MODULE, + Fun =:= eval_fun -> #ieval{level=Le, line=Li, last_call=Lc} = Ieval, case lambda(Fun, As0) of {Cs,Module,Name,As,Bs} -> @@ -657,7 +657,7 @@ eval_function(Mod, Name, As0, Bs0, Called, Ieval) -> lambda(eval_fun, [Cs,As,Bs,{Mod,Name}=F]) -> %% Fun defined in interpreted code, called from outside if - length(element(3,hd(Cs))) == length(As) -> + length(element(3,hd(Cs))) =:= length(As) -> db_ref(Mod), %% Adds ref between module and process {Cs,Mod,Name,As,Bs}; true -> @@ -672,7 +672,7 @@ lambda(Fun, As) when is_function(Fun) -> {env, [{Mod,Name},Bs,Cs]} = erlang:fun_info(Fun, env), {arity, Arity} = erlang:fun_info(Fun, arity), if - length(As) == Arity -> + length(As) =:= Arity -> db_ref(Mod), %% Adds ref between module and process {Cs,Mod,Name,As,Bs}; true -> @@ -731,7 +731,7 @@ db_ref(Mod) -> ModDb -> Node = node(get(int)), DbRef = if - Node/=node() -> {Node,ModDb}; + Node =/= node() -> {Node,ModDb}; true -> ModDb end, put([Mod|db], DbRef), @@ -741,13 +741,12 @@ db_ref(Mod) -> DbRef end. - cache(Key, Data) -> put(cache, lists:sublist([{Key,Data}|get(cache)], 5)). cached(Key) -> - case lists:keysearch(Key, 1, get(cache)) of - {value,{Key,Data}} -> Data; + case lists:keyfind(Key, 1, get(cache)) of + {Key,Data} -> Data; false -> false end. @@ -844,7 +843,7 @@ expr({'try',Line,Es,CaseCs,CatchCs,[]}, Bs0, Ieval0) -> case_clauses(Val, CaseCs, Bs, try_clause, Ieval) end catch - Class:Reason when CatchCs=/=[] -> + Class:Reason when CatchCs =/= [] -> catch_clauses({Class,Reason,[]}, CatchCs, Bs0, Ieval) end; expr({'try',Line,Es,CaseCs,CatchCs,As}, Bs0, Ieval0) -> @@ -1498,10 +1497,7 @@ guard(Gs, Bs) -> or_guard(Gs, Bs). or_guard([G|Gs], Bs) -> %% Short-circuit OR. - case and_guard(G, Bs) of - true -> true; - false -> or_guard(Gs, Bs) - end; + and_guard(G, Bs) orelse or_guard(Gs, Bs); or_guard([], _) -> false. and_guard([G|Gs], Bs) -> @@ -1598,8 +1594,7 @@ match1({bin,_,Fs}, B, Bs0, BBs0) when is_bitstring(B) -> try eval_bits:match_bits(Fs, B, Bs1, BBs, fun(L, R, Bs) -> match1(L, R, Bs, BBs) end, fun(E, Bs) -> expr(E, Bs, #ieval{}) end, - false) of - Match -> Match + false) catch _:_ -> throw(nomatch) end; @@ -1687,7 +1682,7 @@ merge_bindings([{Name,V}|B1s], B2s, Ieval) -> case binding(Name, B2s) of {value,V} -> % Already there, and the same merge_bindings(B1s, B2s, Ieval); - {value,_} when Name=='_' -> % Already there, but anonymous + {value,_} when Name =:= '_' -> % Already there, but anonymous B2s1 = lists:keydelete('_', 1, B2s), [{Name,V}|merge_bindings(B1s, B2s1, Ieval)]; {value,_} -> % Already there, but different => badmatch diff --git a/lib/debugger/src/dbg_iload.erl b/lib/debugger/src/dbg_iload.erl index ec54c646c8..2ae0c333da 100644 --- a/lib/debugger/src/dbg_iload.erl +++ b/lib/debugger/src/dbg_iload.erl @@ -26,7 +26,7 @@ %%-------------------------------------------------------------------- %% load_mod(Mod, File, Binary, Db) -> {ok, Mod} -%% Mod = atom() +%% Mod = module() %% File = string() Source file (including path) %% Binary = binary() %% Db = ETS identifier @@ -408,8 +408,7 @@ expr({call,Line,{remote,_,{atom,_,erlang},{atom,_,fault}},[_]=As}) -> {dbg,Line,fault,expr_list(As)}; expr({call,Line,{remote,_,{atom,_,erlang},{atom,_,exit}},[_]=As}) -> {dbg,Line,exit,expr_list(As)}; -expr({call,Line,{remote,_,{atom,_,erlang},{atom,_,apply}},As0}) - when length(As0) == 3 -> +expr({call,Line,{remote,_,{atom,_,erlang},{atom,_,apply}},[_,_,_]=As0}) -> As = expr_list(As0), {apply,Line,As}; expr({call,Line,{remote,_,{atom,_,Mod},{atom,_,Func}},As0}) -> @@ -517,15 +516,9 @@ expr(Other) -> %% here as sys_pre_expand has transformed source. is_guard_test({op,_,Op,L,R}) -> - case erl_internal:comp_op(Op, 2) of - true -> is_gexpr_list([L,R]); - false -> false - end; + erl_internal:comp_op(Op, 2) andalso is_gexpr_list([L,R]); is_guard_test({call,_,{remote,_,{atom,_,erlang},{atom,_,Test}},As}) -> - case erl_internal:type_test(Test, length(As)) of - true -> is_gexpr_list(As); - false -> false - end; + erl_internal:type_test(Test, length(As)) andalso is_gexpr_list(As); is_guard_test({atom,_,true}) -> true; is_guard_test(_) -> false. @@ -542,22 +535,12 @@ is_gexpr({call,_,{remote,_,{atom,_,erlang},{atom,_,F}},As}) -> Ar = length(As), case erl_internal:guard_bif(F, Ar) of true -> is_gexpr_list(As); - false -> - case erl_internal:arith_op(F, Ar) of - true -> is_gexpr_list(As); - false -> false - end + false -> erl_internal:arith_op(F, Ar) andalso is_gexpr_list(As) end; is_gexpr({op,_,Op,A}) -> - case erl_internal:arith_op(Op, 1) of - true -> is_gexpr(A); - false -> false - end; + erl_internal:arith_op(Op, 1) andalso is_gexpr(A); is_gexpr({op,_,Op,A1,A2}) -> - case erl_internal:arith_op(Op, 2) of - true -> is_gexpr_list([A1,A2]); - false -> false - end; + erl_internal:arith_op(Op, 2) andalso is_gexpr_list([A1,A2]); is_gexpr(_) -> false. is_gexpr_list(Es) -> lists:all(fun (E) -> is_gexpr(E) end, Es). diff --git a/lib/debugger/src/dbg_iserver.erl b/lib/debugger/src/dbg_iserver.erl index 4c1e9ccb7b..59188d83a2 100644 --- a/lib/debugger/src/dbg_iserver.erl +++ b/lib/debugger/src/dbg_iserver.erl @@ -155,11 +155,8 @@ handle_call(get_stack_trace, _From, State) -> %% Retrieving information handle_call(snapshot, _From, State) -> - Reply = lists:map(fun(Proc) -> - {Proc#proc.pid, Proc#proc.function, - Proc#proc.status, Proc#proc.info} - end, - State#state.procs), + Reply = [{Proc#proc.pid, Proc#proc.function, + Proc#proc.status, Proc#proc.info} || Proc <- State#state.procs], {reply, Reply, State}; handle_call({get_meta, Pid}, _From, State) -> Reply = case get_proc({pid, Pid}, State#state.procs) of @@ -181,21 +178,21 @@ handle_call({get_attpid, Pid}, _From, State) -> %% Breakpoint handling handle_call({new_break, Point, Options}, _From, State) -> - case lists:keysearch(Point, 1, State#state.breaks) of + case lists:keymember(Point, 1, State#state.breaks) of false -> Break = {Point, Options}, send_all([subscriber, meta, attached], {new_break, Break}, State), Breaks = keyinsert(Break, 1, State#state.breaks), {reply, ok, State#state{breaks=Breaks}}; - {value, _Break} -> + true -> {reply, {error, break_exists}, State} end; handle_call(all_breaks, _From, State) -> {reply, State#state.breaks, State}; handle_call({all_breaks, Mod}, _From, State) -> Reply = lists:filter(fun({{M,_L}, _Options}) -> - if M==Mod -> true; true -> false end + M =/= Mod end, State#state.breaks), {reply, Reply, State}; @@ -276,7 +273,7 @@ handle_call({contents, Mod, Pid}, _From, State) -> Db = State#state.db, [{{Mod, refs}, ModDbs}] = ets:lookup(Db, {Mod, refs}), ModDb = if - Pid==any -> hd(ModDbs); + Pid =:= any -> hd(ModDbs); true -> lists:foldl(fun(T, not_found) -> [{T, Pids}] = ets:lookup(Db, T), @@ -295,7 +292,7 @@ handle_call({raw_contents, Mod, Pid}, _From, State) -> Db = State#state.db, [{{Mod, refs}, ModDbs}] = ets:lookup(Db, {Mod, refs}), ModDb = if - Pid==any -> hd(ModDbs); + Pid =:= any -> hd(ModDbs); true -> lists:foldl(fun(T, not_found) -> [{T, Pids}] = ets:lookup(Db, T), @@ -360,15 +357,15 @@ handle_cast({set_stack_trace, Flag}, State) -> %% Retrieving information handle_cast(clear, State) -> Procs = lists:filter(fun(#proc{status=Status}) -> - if Status==exit -> false; true -> true end + Status =/= exit end, State#state.procs), {noreply, State#state{procs=Procs}}; %% Breakpoint handling handle_cast({delete_break, Point}, State) -> - case lists:keysearch(Point, 1, State#state.breaks) of - {value, _Break} -> + case lists:keymember(Point, 1, State#state.breaks) of + true -> send_all([subscriber, meta, attached], {delete_break, Point}, State), Breaks = lists:keydelete(Point, 1, State#state.breaks), @@ -377,8 +374,8 @@ handle_cast({delete_break, Point}, State) -> {noreply, State} end; handle_cast({break_option, Point, Option, Value}, State) -> - case lists:keysearch(Point, 1, State#state.breaks) of - {value, {Point, Options}} -> + case lists:keyfind(Point, 1, State#state.breaks) of + {Point, Options} -> N = case Option of status -> 1; action -> 2; @@ -399,7 +396,7 @@ handle_cast(no_break, State) -> handle_cast({no_break, Mod}, State) -> send_all([subscriber, meta, attached], {no_break, Mod}, State), Breaks = lists:filter(fun({{M, _L}, _O}) -> - if M==Mod -> false; true -> true end + M =/= Mod end, State#state.breaks), {noreply, State#state{breaks=Breaks}}; @@ -409,7 +406,7 @@ handle_cast({set_status, Meta, Status, Info}, State) -> {true, Proc} = get_proc({meta, Meta}, State#state.procs), send_all(subscriber, {new_status, Proc#proc.pid, Status, Info}, State), if - Status==break -> + Status =:= break -> auto_attach(break, State#state.auto, Proc); true -> ignore end, @@ -526,11 +523,10 @@ code_change(_OldVsn, State, _Extra) -> %% Internal functions %%==================================================================== -auto_attach(Why, Auto, Proc) when is_record(Proc, proc) -> - case Proc#proc.attpid of - AttPid when is_pid(AttPid) -> ignore; - undefined -> - auto_attach(Why, Auto, Proc#proc.pid) +auto_attach(Why, Auto, #proc{attpid = Attpid, pid = Pid}) -> + case Attpid of + undefined -> auto_attach(Why, Auto, Pid); + _ when is_pid(Attpid) -> ignore end; auto_attach(Why, Auto, Pid) when is_pid(Pid) -> case Auto of @@ -545,7 +541,7 @@ auto_attach(Why, Auto, Pid) when is_pid(Pid) -> keyinsert(Tuple1, N, [Tuple2|Tuples]) -> if - element(N, Tuple1)<element(N, Tuple2) -> + element(N, Tuple1) < element(N, Tuple2) -> [Tuple1, Tuple2|Tuples]; true -> [Tuple2 | keyinsert(Tuple1, N, Tuples)] @@ -576,7 +572,7 @@ send_all([], _Msg, _State) -> ok; send_all(subscriber, Msg, State) -> send_all(State#state.subs, Msg); send_all(meta, Msg, State) -> - Metas = lists:map(fun(Proc) -> Proc#proc.meta end, State#state.procs), + Metas = [Proc#proc.meta || Proc <- State#state.procs], send_all(Metas, Msg); send_all(attached, Msg, State) -> AttPids= mapfilter(fun(Proc) -> @@ -600,7 +596,7 @@ get_proc({Type, Pid}, Procs) -> meta -> #proc.meta; attpid -> #proc.attpid end, - case lists:keysearch(Pid, Index, Procs) of - {value, Proc} -> {true, Proc}; - false -> false + case lists:keyfind(Pid, Index, Procs) of + false -> false; + Proc -> {true, Proc} end. diff --git a/lib/debugger/src/dbg_ui_break_win.erl b/lib/debugger/src/dbg_ui_break_win.erl index a56fe36828..0c1e25e703 100644 --- a/lib/debugger/src/dbg_ui_break_win.erl +++ b/lib/debugger/src/dbg_ui_break_win.erl @@ -268,12 +268,7 @@ handle_event({gs, _Id, click, _Data, ["Ok"|_]}, WinInfo) -> IndexL -> Funcs = WinInfo#winInfo.funcs, Breaks = - lists:map(fun(Index) -> - Func = lists:nth(Index+1, - Funcs), - [Mod | Func] - end, - IndexL), + [[Mod|lists:nth(Index+1, Funcs)] || Index <- IndexL], {break, Breaks, enable} end end; diff --git a/lib/debugger/src/dbg_ui_filedialog_win.erl b/lib/debugger/src/dbg_ui_filedialog_win.erl index f7d76076a5..79ccf20946 100644 --- a/lib/debugger/src/dbg_ui_filedialog_win.erl +++ b/lib/debugger/src/dbg_ui_filedialog_win.erl @@ -202,8 +202,7 @@ handle_event({gs, 'Files', doubleclick, _Data, _Arg}, WinInfo) -> handle_event({gs, _Id, click, select, _Arg}, _WinInfo) -> {select, gs:read('Selection', text)}; handle_event({gs, _Id, click, multiselect, _Arg}, WinInfo) -> - Files = lists:map(fun(File) -> untag(File) end, - gs:read('Files', items)), + Files = [untag(File) || File <- gs:read('Files', items)], {multiselect, WinInfo#winInfo.cwd, Files}; handle_event({gs, _Id, click, filter, _Arg}, WinInfo) -> {Cwd, Pattern} = update_win(gs:read('Filter', text), @@ -286,7 +285,7 @@ max_existing([Name | Names]) -> max_existing(Dir, [Name | Names]) -> Dir2 = filename:join(Dir, Name), case filelib:is_file(Dir2, erl_prim_loader) of - true when Names==[] -> {Dir2, []}; + true when Names =:= [] -> {Dir2, []}; true -> max_existing(Dir2, Names); false -> {Dir, [Name | Names]} end. @@ -309,11 +308,8 @@ extra_filter([], _Dir, _Fun) -> []. get_subdirs(Dir) -> case erl_prim_loader:list_dir(Dir) of {ok, FileNames} -> - X = lists:filter(fun(FileName) -> - File = filename:join(Dir, FileName), - filelib:is_dir(File, erl_prim_loader) - end, - FileNames), + X = [FN || FN <- FileNames, + filelib:is_dir(filename:join(Dir, FN), erl_prim_loader)], lists:sort(X); _Error -> [] @@ -335,4 +331,3 @@ compare([], [$/|File]) -> File; compare(_, _) -> error. - diff --git a/lib/debugger/src/dbg_ui_mon.erl b/lib/debugger/src/dbg_ui_mon.erl index 8888075124..82fe210968 100644 --- a/lib/debugger/src/dbg_ui_mon.erl +++ b/lib/debugger/src/dbg_ui_mon.erl @@ -162,7 +162,7 @@ init2(CallingPid, Mode, SFile, GS) -> CallingPid ! {initialization_complete, self()}, if - SFile==default -> + SFile =:= default -> loop(State3); true -> loop(load_settings(SFile, State3)) @@ -226,7 +226,7 @@ loop(State) -> gui_cmd(stopped, State); %% From the GUI - GuiEvent when is_tuple(GuiEvent), element(1, GuiEvent)==gs -> + GuiEvent when is_tuple(GuiEvent), element(1, GuiEvent) =:= gs -> Cmd = dbg_ui_mon_win:handle_event(GuiEvent,State#state.win), State2 = gui_cmd(Cmd, State), loop(State2); @@ -269,7 +269,7 @@ gui_cmd(ignore, State) -> State; gui_cmd(stopped, State) -> if - State#state.starter==true -> int:stop(); + State#state.starter =:= true -> int:stop(); true -> int:auto_attach(false) end, exit(stop); @@ -413,9 +413,9 @@ gui_cmd({'Trace Window', TraceWin}, State) -> State2; gui_cmd({'Auto Attach', When}, State) -> if - When==[] -> int:auto_attach(false); + When =:= [] -> int:auto_attach(false); true -> - Flags = lists:map(fun(Name) -> map(Name) end, When), + Flags = [map(Name) || Name <- When], int:auto_attach(Flags, trace_function(State)) end, State; @@ -676,7 +676,7 @@ load_settings2(Settings, State) -> Break, int:break(Mod, Line), if - Status==inactive -> + Status =:= inactive -> int:disable_break(Mod, Line); true -> ignore end, @@ -700,12 +700,8 @@ save_settings(SFile, State) -> int:auto_attach(), int:stack_trace(), State#state.backtrace, - lists:map(fun(Mod) -> - int:file(Mod) - end, - int:interpreted()), + [int:file(Mod) || Mod <- int:interpreted()], int:all_breaks()}, - Binary = term_to_binary({debugger_settings, Settings}), case file:write_file(SFile, Binary) of ok -> @@ -720,13 +716,12 @@ save_settings(SFile, State) -> %%==================================================================== registered_name(Pid) -> - %% Yield in order to give Pid more time to register its name timer:sleep(200), Node = node(Pid), if - Node==node() -> + Node =:= node() -> case erlang:process_info(Pid, registered_name) of {registered_name, Name} -> Name; _ -> undefined diff --git a/lib/debugger/src/dbg_ui_mon_win.erl b/lib/debugger/src/dbg_ui_mon_win.erl index ba2f94c550..66e59a822a 100644 --- a/lib/debugger/src/dbg_ui_mon_win.erl +++ b/lib/debugger/src/dbg_ui_mon_win.erl @@ -224,8 +224,7 @@ select(MenuItem, Bool) -> %%-------------------------------------------------------------------- add_module(WinInfo, Menu, Mod) -> Modules = WinInfo#winInfo.modules, - case lists:keysearch(Mod, #moduleInfo.module, Modules) of - {value, _ModInfo} -> WinInfo; + case lists:keymember(Mod, #moduleInfo.module, Modules) of false -> %% Create a menu for the module Font = dbg_ui_win:font(normal), @@ -244,7 +243,8 @@ add_module(WinInfo, Menu, Mod) -> gs:config(WinInfo#winInfo.listbox, {add, Mod}), ModInfo = #moduleInfo{module=Mod, menubtn=MenuBtn}, - WinInfo#winInfo{modules=[ModInfo | Modules]} + WinInfo#winInfo{modules=[ModInfo | Modules]}; + true -> WinInfo end. %%-------------------------------------------------------------------- @@ -491,14 +491,13 @@ handle_event({gs, _Id, click, autoattach, _Arg}, WinInfo) -> %% Process grid handle_event({gs, _Id, keypress, _Data, [Key|_]}, WinInfo) when - Key=='Up'; Key=='Down' -> - Dir = if Key=='Up' -> up; Key=='Down' -> down end, + Key =:= 'Up'; Key =:= 'Down' -> + Dir = if Key =:= 'Up' -> up; Key =:= 'Down' -> down end, Row = move(WinInfo, Dir), - if Row>1 -> WinInfo2 = highlight(WinInfo, Row), - {value, #procInfo{pid=Pid}} = - lists:keysearch(Row, #procInfo.row, WinInfo#winInfo.processes), + #procInfo{pid=Pid} = + lists:keyfind(Row, #procInfo.row, WinInfo#winInfo.processes), {focus, Pid, WinInfo2}; true -> ignore @@ -515,10 +514,9 @@ handle_event(_GSEvent, _WinInfo) -> move(WinInfo, Dir) -> Row = WinInfo#winInfo.focus, Last = WinInfo#winInfo.row, - if - Dir==up, Row>1 -> Row-1; - Dir==down, Row<Last -> Row+1; + Dir =:= up, Row > 1 -> Row-1; + Dir =:= down, Row < Last -> Row+1; true -> Row end. @@ -533,7 +531,6 @@ highlight(WinInfo, Row) -> GridLine2 = gs:read(Grid, {obj_at_row, Row}), gs:config(GridLine2, {fg, white}), WinInfo#winInfo{focus=Row}. - %%==================================================================== %% Internal functions @@ -545,7 +542,7 @@ configure(WinInfo, {W, H}) -> Dx = NewW - gs:read(Grid, width), Dy = H-42 - gs:read(Grid, height), if - (Dx+Dy)=/=0 -> + (Dx+Dy) =/= 0 -> gs:config(Grid, [{width, NewW}, {height, H-30}]), Cols = calc_columnwidths(NewW), gs:config(Grid, Cols); @@ -555,10 +552,9 @@ configure(WinInfo, {W, H}) -> calc_columnwidths(Width) -> W = if - Width=<?Wg -> ?Wg; + Width =< ?Wg -> ?Wg; true -> Width end, - First = lists:map(fun (X) -> round(X) end, - [0.13*W, 0.27*W, 0.18*W, 0.18*W]), + First = [round(X) || X <- [0.13*W, 0.27*W, 0.18*W, 0.18*W]], Last = W - lists:sum(First) - 30, {columnwidths, First++[Last]}. diff --git a/lib/debugger/src/dbg_ui_trace_win.erl b/lib/debugger/src/dbg_ui_trace_win.erl index c6f041a63d..82d4199630 100644 --- a/lib/debugger/src/dbg_ui_trace_win.erl +++ b/lib/debugger/src/dbg_ui_trace_win.erl @@ -147,17 +147,17 @@ configure(WinInfo, TraceWin) -> H = gs:read(Win, height), H2 = if - Bu1==close, Bu2==open -> + Bu1 =:= close, Bu2 =:= open -> resize_button_area(open, width, W-4), gs:config('ButtonArea', {height, 30}), H+30; - Bu1==open, Bu2==close -> + Bu1 =:= open, Bu2 =:= close -> gs:config('ButtonArea', [{width, 0}, {height, 0}]), H-30; true -> H end, H3 = if - Ev1==close, Ev2==open, Bi1==open -> + Ev1 =:= close, Ev2 =:= open, Bi1 =:= open -> Wnew1 = round((W-10-4)/2), % W = window/2 - rb - pads Hbi1 = gs:read('BindArea', height), % H = bind area h resize_eval_area(open, width, Wnew1), @@ -167,25 +167,25 @@ configure(WinInfo, TraceWin) -> resize_bind_area(open, width, Wnew1-gs:read('BindArea', width)), H2; - Ev1==close, Ev2==open, Bi1==close -> + Ev1 =:= close, Ev2 =:= open, Bi1 =:= close -> resize_eval_area(open, width, W-4), resize_eval_area(open, height, 200), H2+200; - Ev1==open, Ev2==close, Bi1==open -> + Ev1 =:= open, Ev2 =:= close, Bi1 =:= open -> gs:config('EvalArea', [{width,0}, {height,0}]), gs:config('RB3', [{width, 0}, {height, 0}]), Wnew2 = W-4, resize_bind_area(open, width, Wnew2-gs:read('BindArea', width)), H2; - Ev1==open, Ev2==close, Bi1==close -> + Ev1 =:= open, Ev2 =:= close, Bi1 =:= close -> Hs1 = gs:read('EvalArea', height), gs:config('EvalArea', [{width, 0}, {height, 0}]), H2-Hs1; true -> H2 end, H4 = if - Bi1==close, Bi2==open, Ev2==open -> + Bi1 =:= close, Bi2 =:= open, Ev2 =:= open -> Wnew3 = round((W-10-4)/2), % W = window/2 - rb - pads Hs2 = gs:read('EvalArea', height), % H = eval area h resize_bind_area(open, width, Wnew3), @@ -194,29 +194,29 @@ configure(WinInfo, TraceWin) -> resize_eval_area(open, width, Wnew3-gs:read('EvalArea', width)), H3; - Bi1==close, Bi2==open, Ev2==close -> + Bi1 =:= close, Bi2 =:= open, Ev2 =:= close -> resize_bind_area(open, width, W-4), resize_bind_area(open, height, 200), H3+200; - Bi1==open, Bi2==close, Ev2==open -> + Bi1 =:= open, Bi2 =:= close, Ev2 =:= open -> gs:config('BindArea', [{width, 0}, {height, 0}]), gs:config('RB3', [{width, 0}, {height, 0}]), Wnew4 = W-4, resize_eval_area(open, width, Wnew4-gs:read('EvalArea', width)), H3; - Bi1==open, Bi2==close, Ev2==close -> + Bi1 =:= open, Bi2 =:= close, Ev2 =:= close -> Hbi2 = gs:read('BindArea', height), gs:config('BindArea', [{width, 0}, {height, 0}]), H3-Hbi2; true -> H3 end, H5 = if - Tr1==close, Tr2==open -> + Tr1 =:= close, Tr2 =:= open -> resize_trace_area(open, width, W-4), resize_trace_area(open, height, 200), H4+200; - Tr1==open, Tr2==close -> + Tr1 =:= open, Tr2 =:= close -> Hf = gs:read('TraceArea', height), gs:config('TraceArea', [{width, 0}, {height, 0}]), H4-Hf; @@ -226,10 +226,10 @@ configure(WinInfo, TraceWin) -> RB1old = rb1(OldFlags), RB1new = rb1(NewFlags), if - RB1old==close, RB1new==open -> + RB1old =:= close, RB1new =:= open -> gs:config('RB1', [{width, W-4}, {height, 10}]), gs:config(Win, {height, gs:read(Win, height)+10}); - RB1old==open, RB1new==close -> + RB1old =:= open, RB1new =:= close -> gs:config('RB1', [{width, 0}, {height, 0}, lower]), gs:config(Win, {height, gs:read(Win, height)-10}); true -> ignore @@ -237,10 +237,10 @@ configure(WinInfo, TraceWin) -> RB2old = rb2(OldFlags), RB2new = rb2(NewFlags), if - RB2old==close, RB2new==open -> + RB2old =:= close, RB2new =:= open -> gs:config('RB2', [{width, W-4}, {height, 10}]), gs:config(Win, {height,gs:read(Win, height)+10}); - RB2old==open, RB2new==close -> + RB2old =:= open, RB2new =:= close -> gs:config('RB2', [{width, 0}, {height, 0}, lower]), gs:config(Win, {height, gs:read(Win, height)-10}); true -> ignore @@ -301,15 +301,15 @@ select(MenuItem, Bool) -> %% Cond = null | {Mod, Func} %%-------------------------------------------------------------------- add_break(WinInfo, Menu, {{Mod,Line},[Status|_Options]}=Break) -> - case lists:keysearch(Mod, 1, WinInfo#winInfo.editors) of - {value, {Mod, Editor}} -> + case lists:keyfind(Mod, 1, WinInfo#winInfo.editors) of + {Mod, Editor} -> add_break_to_code(Editor, Line, Status); false -> ignore end, add_break_to_menu(WinInfo, Menu, Break). add_break_to_code(Editor, Line, Status) -> - Color = if Status==active -> red; Status==inactive -> blue end, + Color = if Status =:= active -> red; Status =:= inactive -> blue end, config_editor(Editor, [{overwrite,{{Line,0},"-@- "}}, {fg,{{{Line,0},{Line,lineend}}, Color}}]). @@ -330,8 +330,8 @@ add_break_to_menu(WinInfo, Menu, {Point, [Status|_Options]=Options}) -> %% Cond = null | {Mod, Func} %%-------------------------------------------------------------------- update_break(WinInfo, {{Mod,Line},[Status|_Options]}=Break) -> - case lists:keysearch(Mod, 1, WinInfo#winInfo.editors) of - {value, {Mod, Editor}} -> + case lists:keyfind(Mod, 1, WinInfo#winInfo.editors) of + {Mod, Editor} -> add_break_to_code(Editor, Line, Status); false -> ignore end, @@ -352,8 +352,8 @@ update_break_in_menu(WinInfo, {Point, [Status|_Options]=Options}) -> %% Point = {Mod, Line} %%-------------------------------------------------------------------- delete_break(WinInfo, {Mod,Line}=Point) -> - case lists:keysearch(Mod, 1, WinInfo#winInfo.editors) of - {value, {Mod, Editor}} -> delete_break_from_code(Editor, Line); + case lists:keyfind(Mod, 1, WinInfo#winInfo.editors) of + {Mod, Editor} -> delete_break_from_code(Editor, Line); false -> ignore end, delete_break_from_menu(WinInfo, Point). @@ -379,11 +379,11 @@ clear_breaks(WinInfo) -> clear_breaks(WinInfo, all). clear_breaks(WinInfo, Mod) -> Remove = if - Mod==all -> WinInfo#winInfo.breaks; + Mod =:= all -> WinInfo#winInfo.breaks; true -> lists:filter(fun(#breakInfo{point={Mod2,_L}}) -> if - Mod2==Mod -> true; + Mod2 =:= Mod -> true; true -> false end end, @@ -450,8 +450,8 @@ display(Arg) -> %% Note: remove_code/2 should not be used for currently shown module. %%-------------------------------------------------------------------- is_shown(WinInfo, Mod) -> - case lists:keysearch(Mod, 1, WinInfo#winInfo.editors) of - {value, {Mod, Editor}} -> + case lists:keyfind(Mod, 1, WinInfo#winInfo.editors) of + {Mod, Editor} -> gs:config(Editor, raise), {true, WinInfo#winInfo{editor={Mod, Editor}}}; false -> false @@ -459,24 +459,22 @@ is_shown(WinInfo, Mod) -> show_code(WinInfo, Mod, Contents) -> Editors = WinInfo#winInfo.editors, - {Flag, Editor} = case lists:keysearch(Mod, 1, Editors) of - {value, {Mod, Ed}} -> {existing, Ed}; + {Flag, Editor} = case lists:keyfind(Mod, 1, Editors) of + {Mod, Ed} -> {existing, Ed}; false -> {new, code_editor()} end, - %% Insert code and update breakpoints, if any config_editor(Editor, [raise, clear]), show_code(Editor, Contents), lists:foreach(fun(BreakInfo) -> case BreakInfo#breakInfo.point of - {Mod2, Line} when Mod2==Mod -> + {Mod2, Line} when Mod2 =:= Mod -> Status = BreakInfo#breakInfo.status, add_break_to_code(Editor, Line,Status); _Point -> ignore end end, WinInfo#winInfo.breaks), - case Flag of existing -> WinInfo#winInfo{editor={Mod, Editor}}; @@ -485,7 +483,7 @@ show_code(WinInfo, Mod, Contents) -> editors=[{Mod, Editor} | Editors]} end. -show_code(Editor, Text) when length(Text)>1500 -> +show_code(Editor, Text) when length(Text) > 1500 -> %% Add some text at a time so that other processes may get scheduled Str = string:sub_string(Text, 1, 1500), config_editor(Editor, {insert,{'end', Str}}), @@ -494,21 +492,19 @@ show_code(Editor, Text) -> config_editor(Editor, {insert,{'end',Text}}). show_no_code(WinInfo) -> - {value, {'$top', Editor}} = - lists:keysearch('$top', 1, WinInfo#winInfo.editors), + {'$top', Editor} = lists:keyfind('$top', 1, WinInfo#winInfo.editors), gs:config(Editor, raise), WinInfo#winInfo{editor={'$top', Editor}}. remove_code(WinInfo, Mod) -> Editors = WinInfo#winInfo.editors, - case lists:keysearch(Mod, 1, Editors) of - {value, {Mod, Editor}} -> + case lists:keyfind(Mod, 1, Editors) of + {Mod, Editor} -> gs:destroy(Editor), WinInfo#winInfo{editors=lists:keydelete(Mod, 1, Editors)}; false -> WinInfo end. - %%-------------------------------------------------------------------- %% mark_line(WinInfo, Line, How) -> WinInfo @@ -522,7 +518,7 @@ mark_line(WinInfo, Line, How) -> mark_line2(Editor, WinInfo#winInfo.marked_line, false), mark_line2(Editor, Line, How), if - Line/=0 -> config_editor(Editor, {vscrollpos, Line-5}); + Line =/= 0 -> config_editor(Editor, {vscrollpos, Line-5}); true -> ignore end, WinInfo#winInfo{marked_line=Line}. @@ -537,7 +533,7 @@ mark_line2(Editor, Line, How) -> false -> " " end, Font = if - How==false -> dbg_ui_win:font(normal); + How =:= false -> dbg_ui_win:font(normal); true -> dbg_ui_win:font(bold) end, config_editor(Editor, [{overwrite, {{Line,5}, Prefix}}, @@ -558,10 +554,10 @@ select_line(WinInfo, Line) -> %% help window, it must be checked that it is correct Size = gs:read(Editor, size), if - Line==0 -> + Line =:= 0 -> select_line(Editor, WinInfo#winInfo.selected_line, false), WinInfo#winInfo{selected_line=0}; - Line<Size -> + Line < Size -> select_line(Editor, Line, true), config_editor(Editor, {vscrollpos, Line-5}), WinInfo#winInfo{selected_line=Line}; @@ -712,10 +708,10 @@ handle_event({gs, Editor, buttonpress, code_editor, _Arg}, WinInfo) -> {Row, _} -> {Mod, _Editor} = WinInfo#winInfo.editor, Point = {Mod, Row}, - case lists:keysearch(Point, #breakInfo.point, + case lists:keymember(Point, #breakInfo.point, WinInfo#winInfo.breaks) of - {value, _BreakInfo} -> {break, Point, delete}; - false -> {break, Point, add} + false -> {break, Point, add}; + true -> {break, Point, delete} end; {Row2, _} -> select_line(Editor, Row2, true), @@ -776,7 +772,7 @@ code_editor() -> code_editor(Name, W, H) -> Editor = if - Name==null -> gs:editor('CodeArea', []); + Name =:= null -> gs:editor('CodeArea', []); true -> gs:editor(Name, 'CodeArea', []) end, gs:config(Editor, [{x,5}, {y,30}, {width,W}, {height,H}, @@ -814,8 +810,8 @@ buttons() -> {'Where','WhereButton'}, {'Up','UpButton'}, {'Down','DownButton'}]. is_button(Name) -> - case lists:keysearch(Name, 1, buttons()) of - {value, {Name, Button}} -> {true, Button}; + case lists:keyfind(Name, 1, buttons()) of + {Name, Button} -> {true, Button}; false -> false end. @@ -847,7 +843,7 @@ resize_button_area(open, width, Diff) -> eval_area({Ev,Bi}, X, Y, FrameOpts, Win) -> {W,H} = if - Ev==open -> {289,200}; + Ev =:= open -> {289,200}; true -> {0,0} end, Font = dbg_ui_win:font(normal), @@ -870,7 +866,7 @@ eval_area({Ev,Bi}, X, Y, FrameOpts, Win) -> {font_style,{{{1,0},'end'},Font}}]), gs:config('EvalEditor', {enable, false}), if - Ev==open, Bi==close -> resize_eval_area(Ev, width, 257); + Ev =:= open, Bi =:= close -> resize_eval_area(Ev, width, 257); true -> ignore end. @@ -891,7 +887,7 @@ resize_eval_area(open, Key, Diff) -> bind_area({Ev,Bi}, X, Y, FrameOpts, Win) -> {W,H} = if - Bi==open -> {249,200}; + Bi =:= open -> {249,200}; true -> {0,0} end, gs:frame('BindArea', Win, @@ -908,7 +904,7 @@ bind_area({Ev,Bi}, X, Y, FrameOpts, Win) -> {text,{1,"Name"}}, {text,{2,"Value"}}, {font,Font}]), gs:config('BindGrid', {rows,{1,1}}), if - Bi==open, Ev==close -> resize_bind_area(Bi, width, 297); + Bi =:= open, Ev =:= close -> resize_bind_area(Bi, width, 297); true -> ignore end. @@ -993,15 +989,15 @@ resizebar(Flag, Name, X, Y, W, H, Obj) -> rb1({_Bu,Ev,Bi,Tr}) -> if - Ev==close, Bi==close, Tr==close -> close; + Ev =:= close, Bi =:= close, Tr =:= close -> close; true -> open end. rb2({_Bu,Ev,Bi,Tr}) -> if - Tr==open -> + Tr =:= open -> if - Ev==close, Bi==close -> close; + Ev =:= close, Bi =:= close -> close; true -> open end; true -> close @@ -1009,7 +1005,7 @@ rb2({_Bu,Ev,Bi,Tr}) -> rb3({_Bu,Ev,Bi,_Tr}) -> if - Ev==open, Bi==open -> open; + Ev =:= open, Bi =:= open -> open; true -> close end. @@ -1067,7 +1063,7 @@ configure(WinInfo, NewW, NewH) -> %% Adjust width unless it is unchanged or less than minimum width if - OldW/=NewW -> + OldW =/= NewW -> {Dcode,Deval,Dbind} = configure_widths(OldW,NewW,Flags), resize_code_area(WinInfo, width, Dcode), case rb1(Flags) of @@ -1090,7 +1086,7 @@ configure(WinInfo, NewW, NewH) -> %% Adjust height unless it is unchanged or less than minimum height if - OldH/=NewH -> + OldH =/= NewH -> {Dcode2,Deval2,Dtrace} = configure_heights(OldH,NewH,Flags), resize_code_area(WinInfo, height, Dcode2), resize_eval_area(Ev, height, Deval2), @@ -1117,11 +1113,11 @@ configure_widths(OldW, NewW, Flags) -> %% Check how much the frames can be resized in reality Limits = if %% Window larger - NewW>OldW -> + NewW > OldW -> if - Ev==open,Bi==open -> {0,Diff,Diff}; - Ev==open -> {0,Diff,0}; - Bi==open -> {0,0,Diff}; + Ev =:= open, Bi =:= open -> {0,Diff,Diff}; + Ev =:= open -> {0,Diff,0}; + Bi =:= open -> {0,0,Diff}; true -> {Diff,0,0} end; @@ -1129,12 +1125,12 @@ configure_widths(OldW, NewW, Flags) -> %% and current size OldW>NewW -> if - Ev==open,Bi==open -> + Ev =:= open, Bi =:= open -> {0, gs:read('EvalArea',width)-204, gs:read('BindArea',width)-112}; - Ev==open -> {0,Diff,0}; - Bi==open -> {0,0,Diff}; + Ev =:= open -> {0,Diff,0}; + Bi =:= open -> {0,0,Diff}; true -> {Diff,0,0} end end, @@ -1142,13 +1138,13 @@ configure_widths(OldW, NewW, Flags) -> case Limits of %% No Shell or Bind frame, larger window - {T,0,0} when NewW>OldW -> {T,0,0}; + {T,0,0} when NewW > OldW -> {T,0,0}; %% No Shell or Bind frame, smaller window - {T,0,0} when OldW>NewW -> {-T,0,0}; + {T,0,0} when OldW > NewW -> {-T,0,0}; %% Window larger; divide Diff among the frames and return result - {_,Sf,B} when NewW>OldW -> + {_,Sf,B} when NewW > OldW -> {_,Sf2,B2} = divide([{0,0},{0,Sf},{0,B}],Diff), {Sf2+B2,Sf2,B2}; @@ -1171,33 +1167,33 @@ configure_heights(OldH, NewH, Flags) -> %% Check how much the frames can be resized in reality {T,Sf,Ff} = if %% Window larger - NewH>OldH -> + NewH > OldH -> {Diff, if - Ev==close, Bi==close -> 0; + Ev =:= close, Bi =:= close -> 0; true -> Diff end, if - Tr==open -> Diff; + Tr =:= open -> Diff; true -> 0 end}; %% Window smaller; get difference between min size %% and current size - OldH>NewH -> + OldH > NewH -> {gs:read('CodeArea',height)-100, if - Ev==close, Bi==close -> 0; + Ev =:= close, Bi =:= close -> 0; true -> if - Ev==open -> + Ev =:= open -> gs:read('EvalArea',height)-100; - Bi==open -> + Bi =:= open -> gs:read('BindArea',height)-100 end end, if - Tr==open -> gs:read('TraceArea',height)-100; + Tr =:= open -> gs:read('TraceArea',height)-100; true -> 0 end} end, @@ -1251,10 +1247,8 @@ divide(L, Diff) -> if %% All of Diff has been distributed - D==0 -> {T,S,F}; - + D =:= 0 -> {T,S,F}; true -> - %% For each element, try to add as much as possible of D {NewT,Dt} = divide2(D,T,Tmax), {NewS,Ds} = divide2(D,S,Smax), @@ -1296,25 +1290,25 @@ resize(WinInfo, ResizeBar) -> rblimits('RB2',W,H), rblimits('RB3',W,H)). -resizeloop(WI, RB, Prev, {Min1,Max1},{Min2,Max2},{Min3,Max3}) -> +resizeloop(WI, RB, Prev, {Min1,Max1}, {Min2,Max2}, {Min3,Max3}) -> receive - {gs,_,motion,_,[_,Y]} when RB=='RB1', Y>Min1,Y<Max1 -> + {gs,_,motion,_,[_,Y]} when RB =:= 'RB1', Y > Min1, Y < Max1 -> gs:config('RB1', {y,Y}), - resizeloop(WI, RB, Y, {Min1,Max1},{Min2,Max2},{Min3,Max3}); - {gs,_,motion,_,_} when RB=='RB1' -> - resizeloop(WI, RB, Prev,{Min1,Max1},{Min2,Max2},{Min3,Max3}); + resizeloop(WI, RB, Y, {Min1,Max1}, {Min2,Max2}, {Min3,Max3}); + {gs,_,motion,_,_} when RB =:= 'RB1' -> + resizeloop(WI, RB, Prev, {Min1,Max1}, {Min2,Max2}, {Min3,Max3}); - {gs,_,motion,_,[_,Y]} when RB=='RB2', Y>Min2,Y<Max2 -> + {gs,_,motion,_,[_,Y]} when RB =:= 'RB2', Y > Min2, Y < Max2 -> gs:config('RB2', {y,Y}), - resizeloop(WI, RB, Y, {Min1,Max1},{Min2,Max2},{Min3,Max3}); - {gs,_,motion,_,_} when RB=='RB2' -> - resizeloop(WI, RB, Prev,{Min1,Max1},{Min2,Max2},{Min3,Max3}); + resizeloop(WI, RB, Y, {Min1,Max1}, {Min2,Max2}, {Min3,Max3}); + {gs,_,motion,_,_} when RB =:= 'RB2' -> + resizeloop(WI, RB, Prev, {Min1,Max1}, {Min2,Max2}, {Min3,Max3}); - {gs,_,motion,_,[X,_]} when RB=='RB3', X>Min3,X<Max3 -> + {gs,_,motion,_,[X,_]} when RB =:= 'RB3', X > Min3, X < Max3 -> gs:config('RB3', {x,X}), - resizeloop(WI, RB, X, {Min1,Max1},{Min2,Max2},{Min3,Max3}); - {gs,_,motion,_,_} when RB=='RB3' -> - resizeloop(WI, RB, Prev,{Min1,Max1},{Min2,Max2},{Min3,Max3}); + resizeloop(WI, RB, X, {Min1,Max1}, {Min2,Max2}, {Min3,Max3}); + {gs,_,motion,_,_} when RB =:= 'RB3' -> + resizeloop(WI, RB, Prev, {Min1,Max1}, {Min2,Max2}, {Min3,Max3}); {gs,_,buttonrelease,_,_} -> resize_win(WI, RB, Prev) @@ -1329,7 +1323,7 @@ resize_win(WinInfo, 'RB1', Y) -> %% Resize Code, Evaluator and Binding areas resize_code_area(WinInfo, height, -Diff), if - S==close, Bi==close, F==open -> + S =:= close, Bi =:= close, F =:= open -> resize_trace_area(open, height, Diff); true -> resize_eval_area(S, height, Diff), @@ -1388,7 +1382,7 @@ rblimits('RB1',_W,H) -> RB2 = gs:read('RB2',height), FF = gs:read('TraceArea',height), Max = case RB2 of - 0 when FF/=0 -> + 0 when FF =/= 0 -> H-112; _ -> Y = gs:read('RB2',y), @@ -1397,18 +1391,14 @@ rblimits('RB1',_W,H) -> {Min,Max}; rblimits('RB2',_W,H) -> - - %% TraceFrame should not have height <100 + %% TraceFrame should not have height < 100 Max = H-112, - %% Min is decided by a minimum distance to 'RB1' Y = gs:read('RB1',y), Min = erlang:min(Max,Y+140), - {Min,Max}; rblimits('RB3',W,_H) -> - %% Neither CodeArea nor BindArea should occupy %% less than 1/3 of the total window width and EvalFrame should %% be at least 289 pixels wide @@ -1484,7 +1474,7 @@ helpwin_action(gotoline, default, AttPid, _Editor, Data, Win) -> end, Data; helpwin_action(search, case_sensitive, _AttPid, _Ed, {Pos, CS}, _Win) -> - Bool = if CS==true -> false; CS==false -> true end, + Bool = if CS =:= true -> false; CS =:= false -> true end, {Pos, Bool}; helpwin_action(search, default, _AttPid, Editor, {Pos, CS}, Win) -> gs:config(lbl(Win), {label, {text, ""}}), @@ -1517,13 +1507,9 @@ search(Str, Editor, Max, {Row, Col}, CS) -> lowercase(true, Str) -> Str; lowercase(false, Str) -> - lists:map(fun(Char) -> - if - Char>=$A, Char=<$Z -> Char+32; - true -> Char - end - end, - Str). + [if Char >= $A, Char =< $Z -> Char+32; + true -> Char + end || Char <- Str]. mark_string(Editor, {Row, Col}, Str) -> Between = {{Row,Col}, {Row,Col+length(Str)}}, @@ -1540,10 +1526,9 @@ unmark_string(Editor, {Row, Col}) -> {fg, {Between, black}}]). helpwin(Type, GS, {X, Y}) -> - W = 200, Pad=10, Wbtn = 50, + W = 200, Pad = 10, Wbtn = 50, - Title = - case Type of search -> "Search"; gotoline -> "Go To Line" end, + Title = case Type of search -> "Search"; gotoline -> "Go To Line" end, Win = gs:window(GS, [{title, Title}, {x, X}, {y, Y}, {width, W}, {destroy, true}]), diff --git a/lib/debugger/src/dbg_ui_view.erl b/lib/debugger/src/dbg_ui_view.erl index 075275f196..7350a830a8 100644 --- a/lib/debugger/src/dbg_ui_view.erl +++ b/lib/debugger/src/dbg_ui_view.erl @@ -21,9 +21,6 @@ %% External exports -export([start/2]). -%% Internal exports --export([init/3]). - -record(state, {gs, % term() Graphics system id win, % term() Attach process window data coords, % {X,Y} Mouse point position @@ -42,7 +39,7 @@ start(GS, Mod) -> Title = "View Module " ++ atom_to_list(Mod), case dbg_ui_winman:is_started(Title) of true -> ignore; - false -> spawn(?MODULE, init, [GS, Mod, Title]) + false -> spawn(fun () -> init(GS, Mod, Title) end) end. @@ -51,7 +48,6 @@ start(GS, Mod) -> %%==================================================================== init(GS, Mod, Title) -> - %% Subscribe to messages from the interpreter int:subscribe(), diff --git a/lib/debugger/src/dbg_ui_win.erl b/lib/debugger/src/dbg_ui_win.erl index 74ff2503ab..9bed6a1ec5 100644 --- a/lib/debugger/src/dbg_ui_win.erl +++ b/lib/debugger/src/dbg_ui_win.erl @@ -24,7 +24,6 @@ create_menus/2, select/2, selected/1, add_break/2, update_break/2, delete_break/1, motion/2 - ]). -record(break, {mb, smi, emi, dimi, demi}). @@ -49,11 +48,11 @@ init() -> font(Style) -> GS = init(), Style2 = if - Style==normal -> []; + Style =:= normal -> []; true -> [Style] end, case gs:read(GS, {choose_font, {screen,Style2,12}}) of - Font when element(1, Font)==screen -> + Font when element(1, Font) =:= screen -> Font; _ -> gs:read(GS, {choose_font, {courier,Style2,12}}) @@ -168,8 +167,7 @@ select(MenuItem, Bool) -> %%-------------------------------------------------------------------- selected(Menu) -> Children = gs:read(Menu, children), - Selected = lists:filter(fun(Child) -> gs:read(Child, select) end, - Children), + Selected = [gs:read(Child, select) || Child <- Children], lists:map(fun(Child) -> {text, Name} = gs:read(Child, label), list_to_atom(Name) diff --git a/lib/debugger/src/dbg_ui_winman.erl b/lib/debugger/src/dbg_ui_winman.erl index 71023cd0d6..398735a7ca 100644 --- a/lib/debugger/src/dbg_ui_winman.erl +++ b/lib/debugger/src/dbg_ui_winman.erl @@ -120,9 +120,9 @@ init(_Arg) -> {ok, #state{}}. handle_call({is_started, Title}, _From, State) -> - Reply = case lists:keysearch(Title, #win.title, State#state.wins) of - {value, Win} -> {true, Win#win.win}; - false -> false + Reply = case lists:keyfind(Title, #win.title, State#state.wins) of + false -> false; + Win -> {true, Win#win.win} end, {reply, Reply, State}. @@ -134,8 +134,8 @@ handle_cast({insert, Pid, Title, Win}, State) -> handle_cast({clear_process, Title}, State) -> OldWins = State#state.wins, - Wins = case lists:keysearch(Title, #win.title, OldWins) of - {value, #win{owner=Pid}} -> + Wins = case lists:keyfind(Title, #win.title, OldWins) of + #win{owner=Pid} -> Msg = {dbg_ui_winman, destroy}, Pid ! Msg, lists:keydelete(Title, #win.title, OldWins); @@ -147,7 +147,7 @@ handle_cast({clear_process, Title}, State) -> handle_info({'EXIT', Pid, _Reason}, State) -> [Mon | _Wins] = State#state.wins, if - Pid==Mon#win.owner -> {stop, normal, State}; + Pid =:= Mon#win.owner -> {stop, normal, State}; true -> Wins2 = lists:keydelete(Pid, #win.owner, State#state.wins), inform_all(Wins2), diff --git a/lib/debugger/src/dbg_wx_break_win.erl b/lib/debugger/src/dbg_wx_break_win.erl index 5dafb0fbe6..78733c98c8 100644 --- a/lib/debugger/src/dbg_wx_break_win.erl +++ b/lib/debugger/src/dbg_wx_break_win.erl @@ -206,11 +206,7 @@ handle_event(#wx{id=OKorListBox, event=#wxCommand{type=OkorDoubleClick}}, OkorDoubleClick =:= command_listbox_doubleclicked -> Mod = wxComboBox:getValue(Text), {_, IndexL} = wxListBox:getSelections(LB), - Breaks = lists:map(fun(Index) -> - Func = lists:nth(Index+1, Funcs), - [list_to_atom(Mod) | Func] - end, - IndexL), + Breaks = [[list_to_atom(Mod)|lists:nth(Index+1, Funcs)] || Index <- IndexL], wxDialog:destroy(Win), {break, Breaks, enable}; handle_event(#wx{id=?wxID_OK},#winInfo{win=Win,text=Text, entries=Es, trigger=Trigger}) -> diff --git a/lib/debugger/src/dbg_wx_interpret.erl b/lib/debugger/src/dbg_wx_interpret.erl index f711ba679d..ffcfbcf36b 100644 --- a/lib/debugger/src/dbg_wx_interpret.erl +++ b/lib/debugger/src/dbg_wx_interpret.erl @@ -115,11 +115,11 @@ interpret_all(Dir, [File0|Files], Mode, Window, Errors) -> interpret_all(_Dir, [], _Mode, _Window, []) -> true; interpret_all(Dir, [], _Mode, Window, Errors) -> - Msg = lists:map(fun(Name) -> - File = filename:join(Dir, Name), - Error = format_error(int:interpretable(File)), - ["\n ",Name,": ",Error] - end, Errors), + Msg = [begin + File = filename:join(Dir, Name), + Error = format_error(int:interpretable(File)), + ["\n ",Name,": ",Error] + end || Name <- Errors], All = ["Error when interpreting: ", Msg], dbg_wx_win:confirm(Window, lists:flatten(All)), true. diff --git a/lib/debugger/src/dbg_wx_mon.erl b/lib/debugger/src/dbg_wx_mon.erl index 3f55c38d35..6bdec994b1 100644 --- a/lib/debugger/src/dbg_wx_mon.erl +++ b/lib/debugger/src/dbg_wx_mon.erl @@ -170,7 +170,7 @@ init2(CallingPid, Mode, SFile, GS) -> CallingPid ! {initialization_complete, self()}, if - SFile==default -> + SFile =:= default -> loop(State3); true -> loop(load_settings(SFile, State3)) @@ -268,7 +268,7 @@ gui_cmd(ignore, State) -> State; gui_cmd(stopped, State) -> if - State#state.starter==true -> int:stop(); + State#state.starter =:= true -> int:stop(); true -> int:auto_attach(false) end, exit(stop); @@ -420,9 +420,9 @@ gui_cmd({'Trace Window', TraceWin}, State) -> State2; gui_cmd({'Auto Attach', When}, State) -> if - When==[] -> int:auto_attach(false); + When =:= [] -> int:auto_attach(false); true -> - Flags = lists:map(fun(Name) -> map(Name) end, When), + Flags = [map(Name) || Name <- When], int:auto_attach(Flags, trace_function(State)) end, State; @@ -691,12 +691,12 @@ load_settings2(Settings, State) -> Break, int:break(Mod, Line), if - Status==inactive -> + Status =:= inactive -> int:disable_break(Mod, Line); true -> ignore end, if - Action/=enable -> + Action =/= enable -> int:action_at_break(Mod,Line,Action); true -> ignore end, @@ -715,10 +715,7 @@ save_settings(SFile, State) -> int:auto_attach(), int:stack_trace(), State#state.backtrace, - lists:map(fun(Mod) -> - int:file(Mod) - end, - int:interpreted()), + [int:file(Mod) || Mod <- int:interpreted()], int:all_breaks()}, Binary = term_to_binary({debugger_settings, Settings}), @@ -741,7 +738,7 @@ registered_name(Pid) -> Node = node(Pid), if - Node==node() -> + Node =:= node() -> case erlang:process_info(Pid, registered_name) of {registered_name, Name} -> Name; _ -> undefined diff --git a/lib/debugger/src/dbg_wx_mon_win.erl b/lib/debugger/src/dbg_wx_mon_win.erl index 8ad4f4213f..04c3501b8c 100644 --- a/lib/debugger/src/dbg_wx_mon_win.erl +++ b/lib/debugger/src/dbg_wx_mon_win.erl @@ -266,8 +266,7 @@ select(MenuItem, Bool) -> add_module(WinInfo, MenuName, Mod) -> Win = WinInfo#winInfo.window, Modules = WinInfo#winInfo.modules, - case lists:keysearch(Mod, #moduleInfo.module, Modules) of - {value, _ModInfo} -> WinInfo; + case lists:keymember(Mod, #moduleInfo.module, Modules) of false -> %% Create a menu for the module Menu = get(MenuName), @@ -284,8 +283,9 @@ add_module(WinInfo, MenuName, Mod) -> wxListBox:append(WinInfo#winInfo.listbox, atom_to_list(Mod)), ModInfo = #moduleInfo{module=Mod, menubtn={Menu,MenuBtn}}, - WinInfo#winInfo{modules=[ModInfo | Modules]} - end. + WinInfo#winInfo{modules=[ModInfo | Modules]}; + true -> WinInfo + end. %%-------------------------------------------------------------------- %% delete_module(WinInfo, Mod) -> WinInfo @@ -559,8 +559,7 @@ handle_event(#wx{event=#wxCommand{type=command_checkbox_clicked}}, handle_event(#wx{event=#wxList{type=command_list_item_selected, itemIndex=Row}}, WinInfo) -> #winInfo{processes=Pids} = WinInfo, - {value, #procInfo{pid=Pid}} = - lists:keysearch(Row, #procInfo.row, Pids), + #procInfo{pid=Pid} = lists:keyfind(Row, #procInfo.row, Pids), {focus, Pid, WinInfo#winInfo{focus=Row}}; handle_event(#wx{event=#wxList{type=command_list_item_activated}}, _WinInfo) -> diff --git a/lib/debugger/src/dbg_wx_trace.erl b/lib/debugger/src/dbg_wx_trace.erl index f9fdf593c4..6675ea33e7 100644 --- a/lib/debugger/src/dbg_wx_trace.erl +++ b/lib/debugger/src/dbg_wx_trace.erl @@ -180,12 +180,12 @@ init_contents(Breaks, State) -> State#state{win=Win}. -loop(#state{meta=Meta} = State) -> +loop(#state{meta=Meta, win=Win} = State) -> receive %% From the GUI main window - GuiEvent when element(1, GuiEvent)==wx -> + GuiEvent when element(1, GuiEvent) =:= wx -> Cmd = wx:batch(fun() -> - dbg_wx_trace_win:handle_event(GuiEvent,State#state.win) + dbg_wx_trace_win:handle_event(GuiEvent,Win) end), State2 = gui_cmd(Cmd, State), loop(State2); @@ -211,11 +211,11 @@ loop(#state{meta=Meta} = State) -> %% From the dbg_wx_winman process (Debugger window manager) {dbg_ui_winman, update_windows_menu, Data} -> - Window = dbg_wx_trace_win:get_window(State#state.win), + Window = dbg_wx_trace_win:get_window(Win), dbg_wx_winman:update_windows_menu(Window,Data), loop(State); {dbg_ui_winman, destroy} -> - dbg_wx_trace_win:stop(State#state.win), + dbg_wx_trace_win:stop(Win), exit(stop) end. @@ -269,7 +269,7 @@ gui_cmd('Continue', State) -> int:meta(State#state.meta, continue), {Status, Mod, Line} = State#state.status, if - Status==wait_break -> + Status =:= wait_break -> Win = dbg_wx_trace_win:unmark_line(State#state.win), gui_enable_functions(wait_running), State#state{win=Win, status={wait_running,Mod,Line}}; @@ -291,7 +291,7 @@ gui_cmd('Stop', State) -> int:meta(State#state.meta, stop), {Status, Mod, Line} = State#state.status, if - Status==wait_running -> + Status =:= wait_running -> Win = dbg_wx_trace_win:mark_line(State#state.win, Line, break), gui_enable_functions(wait_break), @@ -421,7 +421,7 @@ gui_cmd('Function Break...', State) -> gui_cmd('Enable All', State) -> Breaks = int:all_breaks(), ThisMod = State#state.cm, - lists:foreach(fun ({{Mod, Line}, _Options}) when Mod==ThisMod -> + lists:foreach(fun ({{Mod, Line}, _Options}) when Mod =:= ThisMod -> int:enable_break(Mod, Line); (_Break) -> ignore @@ -431,7 +431,7 @@ gui_cmd('Enable All', State) -> gui_cmd('Disable All', State) -> Breaks = int:all_breaks(), ThisMod = State#state.cm, - lists:foreach(fun ({{Mod, Line}, _Options}) when Mod==ThisMod -> + lists:foreach(fun ({{Mod, Line}, _Options}) when Mod =:= ThisMod -> int:disable_break(Mod, Line); (_Break) -> ignore @@ -458,7 +458,7 @@ gui_cmd({'Trace Window', TraceWin}, State) -> Win = dbg_wx_trace_win:configure(State#state.win, TraceWin), {Status,_,_} = State#state.status, if - Status==break; Status==wait_break -> + Status =:= break; Status =:= wait_break -> gui_enable_btrace(Trace, State#state.stack_trace); true -> ignore end, @@ -467,7 +467,7 @@ gui_cmd({'Stack Trace', [Name]}, State) -> int:meta(State#state.meta, stack_trace, map(Name)), {Status,_,_} = State#state.status, if - Status==break; Status==wait_break -> + Status =:= break; Status =:= wait_break -> gui_enable_btrace(State#state.trace, map(Name)); true -> ignore end, @@ -490,9 +490,9 @@ gui_cmd('Debugger', State) -> gui_cmd({user_command, Cmd}, State) -> {Status, _Mod, _Line} = State#state.status, if - Status==break; - Status==wait_break; - Status==wait_running -> + Status =:= break; + Status =:= wait_break; + Status =:= wait_running -> Cm = State#state.cm, Arg = case State#state.stack of {Cur, Max} when Cur<Max -> {Cm, Cmd, Cur}; @@ -531,14 +531,14 @@ add_break(WI, Coords, Type, Mod, Line) -> int_cmd({interpret, Mod}, State) -> if - Mod==State#state.cm -> + Mod =:= State#state.cm -> State#state{cm_obsolete=true}; true -> State end; int_cmd({no_interpret, Mod}, State) -> if - Mod==State#state.cm -> + Mod =:= State#state.cm -> State#state{cm_obsolete=true}; true -> Win = dbg_wx_trace_win:remove_code(State#state.win, Mod), @@ -584,7 +584,7 @@ meta_cmd({re_entry, dbg_ieval, eval_fun}, State) -> meta_cmd({re_entry, Mod, _Func}, State) -> Obs = State#state.cm_obsolete, case State#state.cm of - Mod when Obs==true -> + Mod when Obs =:= true -> Win = gui_load_module(State#state.win, Mod,State#state.pid), State#state{win=Win, cm_obsolete=false}; Mod -> State; @@ -630,11 +630,11 @@ meta_cmd({func_at, Mod, Line, Cur}, State) -> gui_enable_functions(idle), dbg_wx_trace_win:display(State#state.win, idle), State#state{win=Win, cm=Mod, status={idle,Mod,Line}, stack=Stack}; -meta_cmd({wait_at, Mod, Line, Cur}, #state{status={Status,_,_}}=State) - when Status/=init, Status/=break -> +meta_cmd({wait_at, Mod, Line, Cur}, #state{status={Status,_,_}, win=Win}=State) + when Status =/= init, Status =/= break -> Stack = {Cur,Cur}, gui_enable_functions(wait_running), - dbg_wx_trace_win:display(State#state.win, {wait,Mod,Line}), + dbg_wx_trace_win:display(Win, {wait,Mod,Line}), State#state{status={wait_running,Mod,Line}, stack=Stack}; meta_cmd({wait_at, Mod, Line, Cur}, State) -> Stack = {Cur,Cur}, @@ -675,7 +675,7 @@ meta_cmd({stack_trace, Flag}, State) -> gui_enable_updown(Flag, State#state.stack), {Status,_,_} = State#state.status, if - Status==break; Status==wait_break -> + Status =:= break; Status =:= wait_break -> gui_enable_btrace(State#state.trace, Flag); true -> ignore end, @@ -813,7 +813,7 @@ gui_enable_functions(Status) -> gui_enable_updown(Flag, Stack) -> {Enable, Disable} = if - Flag==false -> {[], ['Up', 'Down']}; + Flag =:= false -> {[], ['Up', 'Down']}; true -> case Stack of {1,1} -> {[], ['Up', 'Down']}; @@ -826,14 +826,14 @@ gui_enable_updown(Flag, Stack) -> dbg_wx_trace_win:enable(Enable, true), dbg_wx_trace_win:enable(Disable, false), if - Enable==[] -> dbg_wx_trace_win:enable(['Where'], false); + Enable =:= [] -> dbg_wx_trace_win:enable(['Where'], false); true -> dbg_wx_trace_win:enable(['Where'], true) end. gui_enable_btrace(Trace, StackTrace) -> Bool = if - Trace==false -> false; - StackTrace==false -> false; + Trace =:= false -> false; + StackTrace =:= false -> false; true -> true end, dbg_wx_trace_win:enable(['Back Trace'], Bool). diff --git a/lib/debugger/src/dbg_wx_trace_win.erl b/lib/debugger/src/dbg_wx_trace_win.erl index 2b4a1164ad..720b913024 100755 --- a/lib/debugger/src/dbg_wx_trace_win.erl +++ b/lib/debugger/src/dbg_wx_trace_win.erl @@ -410,11 +410,11 @@ clear_breaks(WinInfo) -> clear_breaks(WinInfo, all). clear_breaks(WinInfo, Mod) -> Remove = if - Mod==all -> WinInfo#winInfo.breaks; + Mod =:= all -> WinInfo#winInfo.breaks; true -> lists:filter(fun(#breakInfo{point={Mod2,_L}}) -> if - Mod2==Mod -> true; + Mod2 =:= Mod -> true; true -> false end end, @@ -481,8 +481,8 @@ display(#winInfo{window=Win, sb=Sb},Arg) -> %% Note: remove_code/2 should not be used for currently shown module. %%-------------------------------------------------------------------- is_shown(WinInfo, Mod) -> - case lists:keysearch(Mod, 1, WinInfo#winInfo.editors) of - {value, {Mod, Editor}} -> + case lists:keyfind(Mod, 1, WinInfo#winInfo.editors) of + {Mod, Editor} -> gs:config(Editor, raise), %% BUGBUG {true, WinInfo#winInfo{editor={Mod, Editor}}}; false -> false @@ -494,7 +494,7 @@ show_code(WinInfo = #winInfo{editor={_, Ed}}, Mod, Contents) -> lists:foreach(fun(BreakInfo) -> case BreakInfo#breakInfo.point of - {Mod2, Line} when Mod2==Mod -> + {Mod2, Line} when Mod2 =:= Mod -> Status = BreakInfo#breakInfo.status, dbg_wx_code:add_break_to_code(Ed, Line,Status); _Point -> ignore @@ -540,10 +540,10 @@ select_line(WinInfo, Line) -> %% help window, it must be checked that it is correct Size = dbg_wx_code:get_no_lines(Ed), if - Line==0 -> + Line =:= 0 -> dbg_wx_code:goto_line(Ed,1), WinInfo#winInfo{selected_line=0}; - Line<Size -> + Line < Size -> dbg_wx_code:goto_line(Ed,Line), WinInfo#winInfo{selected_line=Line}; true -> @@ -764,8 +764,8 @@ handle_event(#wx{event=#wxStyledText{type=stc_doubleclick}}, WinInfo = #winInfo{editor={Mod,Ed}}) -> Line = wxStyledTextCtrl:getCurrentLine(Ed), Point = {Mod, Line+1}, - case lists:keysearch(Point, #breakInfo.point,WinInfo#winInfo.breaks) of - {value, _BreakInfo} -> {break, Point, delete}; + case lists:keymember(Point, #breakInfo.point, WinInfo#winInfo.breaks) of + true -> {break, Point, delete}; false -> {break, Point, add} end; @@ -837,7 +837,7 @@ handle_event(#wx{id=?SEARCH_ENTRY, event=#wxCommand{cmdString=Str}}, %% Button area handle_event(#wx{id=ID, event=#wxCommand{type=command_button_clicked}},_Wi) -> - {value, {Button, _}} = lists:keysearch(ID, 2, buttons()), + {Button, _} = lists:keyfind(ID, 2, buttons()), Button; %% Evaluator area @@ -908,8 +908,8 @@ buttons() -> {'Where',?WhereButton}, {'Up',?UpButton}, {'Down',?DownButton}]. is_button(Name) -> - case lists:keysearch(Name, 1, buttons()) of - {value, {Name, Button}} -> {true, Button}; + case lists:keyfind(Name, 1, buttons()) of + {Name, Button} -> {true, Button}; false -> false end. diff --git a/lib/debugger/src/dbg_wx_view.erl b/lib/debugger/src/dbg_wx_view.erl index 6d34e5650c..8ff89a4847 100644 --- a/lib/debugger/src/dbg_wx_view.erl +++ b/lib/debugger/src/dbg_wx_view.erl @@ -23,9 +23,6 @@ %% External exports -export([start/2]). -%% Internal exports --export([init/4]). - -record(state, {gs, % term() Graphics system id win, % term() Attach process window data coords, % {X,Y} Mouse point position @@ -46,7 +43,7 @@ start(GS, Mod) -> true -> ignore; false -> Env = wx:get_env(), - spawn_link(?MODULE, init, [GS, Env, Mod, Title]) + spawn_link(fun () -> init(GS, Env, Mod, Title) end) end. @@ -84,7 +81,7 @@ loop(State) -> receive %% From the GUI main window - GuiEvent when element(1, GuiEvent)==wx -> + GuiEvent when element(1, GuiEvent) =:= wx -> Cmd = wx:batch(fun() -> dbg_wx_trace_win:handle_event(GuiEvent, State#state.win) end), @@ -167,7 +164,7 @@ gui_cmd('Function Break...', State) -> gui_cmd('Enable All', State) -> Breaks = int:all_breaks(), ThisMod = State#state.mod, - lists:foreach(fun ({{Mod, Line}, _Options}) when Mod==ThisMod -> + lists:foreach(fun ({{Mod, Line}, _Options}) when Mod =:= ThisMod -> int:enable_break(Mod, Line); (_Break) -> ignore @@ -177,7 +174,7 @@ gui_cmd('Enable All', State) -> gui_cmd('Disable All', State) -> Breaks = int:all_breaks(), ThisMod = State#state.mod, - lists:foreach(fun ({{Mod, Line}, _Options}) when Mod==ThisMod -> + lists:foreach(fun ({{Mod, Line}, _Options}) when Mod =:= ThisMod -> int:disable_break(Mod, Line); (_Break) -> ignore @@ -214,21 +211,19 @@ add_break(GS, Coords, Type, Mod, Line) -> %%--Commands from the interpreter------------------------------------- -int_cmd({new_break, {{Mod,_Line},_Options}=Break}, #state{mod=Mod}=State) -> - Win = dbg_wx_trace_win:add_break(State#state.win, 'Break', Break), - State#state{win=Win}; -int_cmd({delete_break, {Mod,_Line}=Point}, #state{mod=Mod}=State) -> - Win = dbg_wx_trace_win:delete_break(State#state.win, Point), - State#state{win=Win}; -int_cmd({break_options, {{Mod,_Line},_Options}=Break}, #state{mod=Mod}=State) -> - Win = dbg_wx_trace_win:update_break(State#state.win, Break), - State#state{win=Win}; -int_cmd(no_break, State) -> - Win = dbg_wx_trace_win:clear_breaks(State#state.win), - State#state{win=Win}; -int_cmd({no_break, _Mod}, State) -> - Win = dbg_wx_trace_win:clear_breaks(State#state.win), - State#state{win=Win}; +int_cmd({new_break, {{Mod,_Line},_Options}=Break}, + #state{mod = Mod, win = Win}=State) -> + State#state{win = dbg_wx_trace_win:add_break(Win, 'Break', Break)}; +int_cmd({delete_break, {Mod,_Line}=Point}, + #state{mod = Mod, win = Win}=State) -> + State#state{win = dbg_wx_trace_win:delete_break(Win, Point)}; +int_cmd({break_options, {{Mod,_Line},_Options}=Break}, + #state{mod = Mod, win = Win}=State) -> + State#state{win = dbg_wx_trace_win:update_break(Win, Break)}; +int_cmd(no_break, #state{win = Win}=State) -> + State#state{win = dbg_wx_trace_win:clear_breaks(Win)}; +int_cmd({no_break, _Mod}, #state{win = Win}=State) -> + State#state{win = dbg_wx_trace_win:clear_breaks(Win)}; int_cmd(_, State) -> State. diff --git a/lib/debugger/src/dbg_wx_winman.erl b/lib/debugger/src/dbg_wx_winman.erl index 1daabe3435..d0ddfeb51a 100755 --- a/lib/debugger/src/dbg_wx_winman.erl +++ b/lib/debugger/src/dbg_wx_winman.erl @@ -118,9 +118,9 @@ init([]) -> {ok, #state{}}. handle_call({is_started, Title}, _From, State) -> - Reply = case lists:keysearch(Title, #win.title, State#state.wins) of - {value, Win} -> {true, Win#win.win}; - false -> false + Reply = case lists:keyfind(Title, #win.title, State#state.wins) of + false -> false; + Win -> {true, Win#win.win} end, {reply, Reply, State}. @@ -132,8 +132,8 @@ handle_cast({insert, Pid, Title, Win}, State) -> handle_cast({clear_process, Title}, State) -> OldWins = State#state.wins, - Wins = case lists:keysearch(Title, #win.title, OldWins) of - {value, #win{owner=Pid}} -> + Wins = case lists:keyfind(Title, #win.title, OldWins) of + #win{owner=Pid} -> Msg = {dbg_ui_winman, destroy}, Pid ! Msg, lists:keydelete(Title, #win.title, OldWins); @@ -145,7 +145,7 @@ handle_cast({clear_process, Title}, State) -> handle_info({'EXIT', Pid, _Reason}, State) -> [Mon | _Wins] = State#state.wins, if - Pid==Mon#win.owner -> {stop, normal, State}; + Pid =:= Mon#win.owner -> {stop, normal, State}; true -> Wins2 = lists:keydelete(Pid, #win.owner, State#state.wins), inform_all(Wins2), diff --git a/lib/debugger/src/i.erl b/lib/debugger/src/i.erl index 7c2fb22946..476a53482e 100644 --- a/lib/debugger/src/i.erl +++ b/lib/debugger/src/i.erl @@ -31,7 +31,6 @@ iv() -> Vsn = string:substr(filename:basename(code:lib_dir(debugger)), 10), list_to_atom(Vsn). - %% ------------------------------------------- %% Start a new graphical monitor. @@ -288,10 +287,9 @@ ia(X,Y,Z) -> %% ------------------------------------------- ia(Pid,Fnk) -> - case lists:keysearch(Pid, 1, int:snapshot()) of - {value, _PidTuple} -> - int:attach(Pid,Fnk); - false -> no_proc + case lists:keymember(Pid, 1, int:snapshot()) of + false -> no_proc; + true -> int:attach(Pid,Fnk) end. ia(X,Y,Z,Fnk) -> diff --git a/lib/debugger/src/int.erl b/lib/debugger/src/int.erl index eeb4df4a8e..9ee2102a19 100644 --- a/lib/debugger/src/int.erl +++ b/lib/debugger/src/int.erl @@ -261,7 +261,7 @@ del_break_in(Mod, Func, Arity) when is_atom(Mod), is_atom(Func), is_integer(Arit end. first_lines(Clauses) -> - lists:map(fun(Clause) -> first_line(Clause) end, Clauses). + [first_line(Clause) || Clause <- Clauses]. first_line({clause,_L,_Vars,_,Exprs}) -> first_line(Exprs); @@ -469,10 +469,10 @@ contents(Mod, Pid) -> %% Arity = integer() %%-------------------------------------------------------------------- functions(Mod) -> - lists:filter(fun([module_info, _Arity]) -> false; - (_Func) -> true - end, - dbg_iserver:call({functions, Mod})). + [F || F <- dbg_iserver:call({functions, Mod}), functions_1(F)]. + +functions_1([module_info, _Arity]) -> false; +functions_1(_Func) -> true. %%==================================================================== diff --git a/lib/debugger/vsn.mk b/lib/debugger/vsn.mk index 5ce37a6bde..f33d66b5cf 100644 --- a/lib/debugger/vsn.mk +++ b/lib/debugger/vsn.mk @@ -1 +1 @@ -DEBUGGER_VSN = 3.2.2 +DEBUGGER_VSN = 3.2.3 diff --git a/lib/dialyzer/doc/src/notes.xml b/lib/dialyzer/doc/src/notes.xml index b6106b928a..856af01525 100755 --- a/lib/dialyzer/doc/src/notes.xml +++ b/lib/dialyzer/doc/src/notes.xml @@ -31,6 +31,41 @@ <p>This document describes the changes made to the Dialyzer application.</p> +<section><title>Dialyzer 2.3.0</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Various changes to dialyzer-related files for R14.</p> + <p> + - Dialyzer properly supports the new attribute + -export_type and checks that remote types only refer to + exported types. A warning is produced if some + files/applications refer to types defined in modules + which are neither in the PLT nor in the analyzed + applications.</p> + <p> + - Support for detecting data races involving whereis/1 + and unregister/1.</p> + <p> + - More precise identification of the reason(s) why a + record construction violates the types declared for its + fields.</p> + <p> + - Fixed bug in the handling of the 'or' guard.</p> + <p> + - Better handling of the erlang:element/2 BIF.</p> + <p> + - Complete handling of Erlang BIFs.</p> + <p> + Own Id: OTP-8699</p> + </item> + </list> + </section> + +</section> + <section><title>Dialyzer 2.2.0</title> <section><title>Improvements and New Features</title> diff --git a/lib/docbuilder/src/docb_edoc_xml_cb.erl b/lib/docbuilder/src/docb_edoc_xml_cb.erl index 4dba843341..f5cfc0fe18 100644 --- a/lib/docbuilder/src/docb_edoc_xml_cb.erl +++ b/lib/docbuilder/src/docb_edoc_xml_cb.erl @@ -478,31 +478,29 @@ otp_xmlify_a_href("#"++_ = Marker, Es0) -> % <seealso marker="#what"> otp_xmlify_a_href("http:"++_ = URL, Es0) -> % external URL {URL, Es0}; otp_xmlify_a_href("OTPROOT"++AppRef, Es0) -> % <.. marker="App:FileRef - case split(AppRef, "/") of - [AppS, "doc", FileRef1] -> - FileRef = AppS++":"++otp_xmlify_a_fileref(FileRef1, AppS), - [#xmlText{value=Str0} = T] = Es0, - Str = case split(Str0, "/") of - %% //Application - [AppS2] -> - %% AppS2 can differ from AppS - %% Example: xmerl/XMerL - AppS2; - [_AppS,ModRef] -> - case split(ModRef, ":") of - %% //Application/Module - [Module] -> - Module++"(3)"; - %% //Application/Module:Type() - [_Module,_Type] -> - ModRef - end; - %% //Application/Module:Function/Arity - [_AppS,ModFunc,Arity] -> - ModFunc++"/"++Arity - end, - {FileRef, [T#xmlText{value=Str}]} - end; + [AppS, "doc", FileRef1] = split(AppRef, "/"), + FileRef = AppS++":"++otp_xmlify_a_fileref(FileRef1, AppS), + [#xmlText{value=Str0} = T] = Es0, + Str = case split(Str0, "/") of + %% //Application + [AppS2] -> + %% AppS2 can differ from AppS + %% Example: xmerl/XMerL + AppS2; + [_AppS,ModRef] -> + case split(ModRef, ":") of + %% //Application/Module + [Module] -> + Module++"(3)"; + %% //Application/Module:Type() + [_Module,_Type] -> + ModRef + end; + %% //Application/Module:Function/Arity + [_AppS,ModFunc,Arity] -> + ModFunc++"/"++Arity + end, + {FileRef, [T#xmlText{value=Str}]}; otp_xmlify_a_href("../"++File, Es0) -> %% Special case: This kind of relative path is used on some %% places within i.e. EDoc and refers to a file within the same @@ -639,13 +637,13 @@ otp_xmlify_table([#xmlText{} = E|Es]) -> [E | otp_xmlify_table(Es)]; otp_xmlify_table([#xmlElement{name=tbody} = E|Es]) -> otp_xmlify_table(E#xmlElement.content)++otp_xmlify_table(Es); -otp_xmlify_table([#xmlElement{name=tr} = E|Es]) -> +otp_xmlify_table([#xmlElement{name=tr, content=Content}|Es]) -> %% Insert newlines between table rows - otp_xmlify_table(E#xmlElement.content)++[{br,[]}]++otp_xmlify_table(Es); -otp_xmlify_table([#xmlElement{name=th} = E|Es]) -> - [{em, E#xmlElement.content} | otp_xmlify_table(Es)]; -otp_xmlify_table([#xmlElement{name=td} = E|Es]) -> - otp_xmlify_e(E#xmlElement.content) ++ otp_xmlify_table(Es); + otp_xmlify_table(Content)++[{br,[]}]++otp_xmlify_table(Es); +otp_xmlify_table([#xmlElement{name=th, content=Content}|Es]) -> + [{em, Content} | otp_xmlify_table(Es)]; +otp_xmlify_table([#xmlElement{name=td, content=Content}|Es]) -> + otp_xmlify_e(Content) ++ otp_xmlify_table(Es); otp_xmlify_table([]) -> []. @@ -1155,8 +1153,8 @@ get_text(#xmlElement{content=[E]}) -> %% text_only(Es) -> Ts %% Takes a list of xmlElement and xmlText and return a lists of xmlText. -text_only([#xmlElement{} = E |Es]) -> - text_only(E#xmlElement.content) ++ text_only(Es); +text_only([#xmlElement{content = Content}|Es]) -> + text_only(Content) ++ text_only(Es); text_only([#xmlText{} = E |Es]) -> [E | text_only(Es)]; text_only([]) -> diff --git a/lib/docbuilder/src/docb_html.erl b/lib/docbuilder/src/docb_html.erl index 9aea4c8a66..bdfc5ea876 100644 --- a/lib/docbuilder/src/docb_html.erl +++ b/lib/docbuilder/src/docb_html.erl @@ -283,16 +283,16 @@ rule([term|_], {_, [ID], _}, Opts) -> ID, "</strong></em> "]}, Opts}; TermList -> - case lists:keysearch(ID, 1, TermList) of + case lists:keyfind(ID, 1, TermList) of false -> {{drop, ["<em><strong>", ID, "</strong></em> "]}, Opts}; - {value, {ID, Name, _Description, _Resp}} -> + {ID, Name, _Description, _Resp} -> {{drop, ["<em><strong>", Name, "</strong></em> "]}, Opts}; - {value, {ID, Name, _Description}} -> + {ID, Name, _Description} -> {{drop, [ "<em><strong>", Name, "</strong></em> "]}, Opts} @@ -306,16 +306,16 @@ rule([term|_], {_, [ID], _}, Opts) -> TermList -> PartApplication = docb_util:lookup_option(part_application, Opts), - case lists:keysearch(ID, 1, TermList) of + case lists:keyfind(ID, 1, TermList) of false -> {{drop, ["<a href=\"", PartApplication, "_term.html#", ID, "\">", ID, "</a> "]}, Opts}; - {value, {ID, Name, _Description, _Resp}} -> + {ID, Name, _Description, _Resp} -> {{drop, ["<a href=\"", PartApplication, "_term.html#", ID, "\">", Name, "</a> "]}, Opts}; - {value, {ID, Name, _Description}} -> + {ID, Name, _Description} -> {{drop, ["<a href=\"", PartApplication, "_term.html#", ID, "\">", Name, "</a> "]}, Opts} @@ -331,17 +331,16 @@ rule([cite|_], {_, [ID], _}, Opts) -> {{drop, ["<em><strong>", ID, "</strong></em> "]}, Opts}; CiteList -> - case lists:keysearch(ID, 1, CiteList) of + case lists:keyfind(ID, 1, CiteList) of false -> {{drop, ["<em><strong>", ID, "</strong></em> "]}, Opts}; - - {value, {ID, Name, _Description, _Resp}} -> + {ID, Name, _Description, _Resp} -> {{drop, ["<em><strong>", Name, "</strong></em> "]}, Opts}; - {value, {ID, Name, _Description}} -> + {ID, Name, _Description} -> {{drop, ["<em><strong>", Name, "</strong></em> "]}, Opts} @@ -355,18 +354,18 @@ rule([cite|_], {_, [ID], _}, Opts) -> CiteList -> PartApp = docb_util:lookup_option(part_application, Opts), - case lists:keysearch(ID, 1, CiteList) of + case lists:keyfind(ID, 1, CiteList) of false -> {{drop, ["<a href=\"", PartApp, "_cite.html#", ID, "\">", ID, "</a> "]}, Opts}; - {value, {ID, Name, _Description, _Resp}} -> + {ID, Name, _Description, _Resp} -> {{drop, ["<a href=\"", PartApp, "_cite.html#", ID, "\">", Name, "</a> "]}, Opts}; - {value, {ID, Name, _Description}} -> + {ID, Name, _Description} -> {{drop, ["<a href=\"", PartApp, "_cite.html#", ID, "\">", Name, "</a> "]}, @@ -376,7 +375,7 @@ rule([cite|_], {_, [ID], _}, Opts) -> end; rule([code|_], {_, [Type], [{pcdata, _, Code}]}, Opts) -> - case lists:member(Type,["ERL","C","NONE"]) of + case lists:member(Type, ["ERL","C","NONE"]) of true -> {{drop, ["\n<div class=\"example\"><pre>\n", docb_html_util:element_cdata_to_html(Code), "\n</pre></div>\n"]}, Opts}; diff --git a/lib/docbuilder/src/docb_html_util.erl b/lib/docbuilder/src/docb_html_util.erl index b2951706ea..02ce8b52a7 100644 --- a/lib/docbuilder/src/docb_html_util.erl +++ b/lib/docbuilder/src/docb_html_util.erl @@ -136,7 +136,6 @@ copy_pics(Src, Dest, Opts) -> Dir = code:lib_dir(docbuilder), InFile = filename:join([Dir, "etc", Src]), OutFile = docb_util:outfile(Dest, "", Opts), - case filelib:last_modified(OutFile) of 0 -> % File doesn't exist file:copy(InFile, OutFile); @@ -156,10 +155,10 @@ copy_pics(Src, Dest, Opts) -> %%--Resolve header data------------------------------------------------- extract_header_data(Key, {header, [], List}) -> - case lists:keysearch(Key, 1, List) of - {value, {Key, [], []}} -> + case lists:keyfind(Key, 1, List) of + {Key, [], []} -> ""; - {value, {Key, [], [{pcdata, [], Value}]}} -> + {Key, [], [{pcdata, [], Value}]} -> pcdata_to_html(Value); false -> "" @@ -253,7 +252,7 @@ make_anchor_href(HRef) -> {ok, [HRef]} -> %% No `#' in HRef, i.e. only path make_anchor_href(HRef, ""); - {ok, [Path, Fragment]}-> + {ok, [Path, Fragment]} -> make_anchor_href(Path, Fragment) end. @@ -398,10 +397,10 @@ count_sections([]) -> %%--Make a ToC---------------------------------------------------------- format_toc(Toc) -> - lists:map(fun({Number, Title}) -> - [Number, " <a href = \"#", Number, - "\">", Title, "</a><br/>\n"] - end, Toc). + [format_toc1(T) || T <- Toc]. + +format_toc1({Number, Title}) -> + [Number, " <a href = \"#", Number, "\">", Title, "</a><br/>\n"]. %%--Convert HTML ISO Latin 1 characters to ordinary characters---------- diff --git a/lib/docbuilder/src/docb_main.erl b/lib/docbuilder/src/docb_main.erl index ef21f65557..87a1401a02 100644 --- a/lib/docbuilder/src/docb_main.erl +++ b/lib/docbuilder/src/docb_main.erl @@ -55,7 +55,7 @@ process(File, Opts) -> %% If no target format is specified, assume HTML: Tos = if - Tos0==[] -> [html]; + Tos0 =:= [] -> [html]; true -> Tos0 end, @@ -327,12 +327,8 @@ verify(Tree) -> verify(Tree, [], 1). verify({pcdata, Optional, _}, Path, Level) -> verify_optional(Optional, Path, Level); verify({Tag, Optional, Args}, Path, Level) when is_list(Args) -> - case verify_optional(Optional, Path, Level) of - true -> - verify_list(Args, [Tag|Path], Level); - false -> - false - end; + verify_optional(Optional, Path, Level) + andalso verify_list(Args, [Tag|Path], Level); verify(Other, Path, Level) -> verify_error(Other, Path, Level). @@ -342,12 +338,7 @@ verify_error(X, Path, Level) -> false. verify_list([H|T], Path, Level) -> - case verify(H, Path, Level) of - true -> - verify_list(T, Path, Level +1); - false -> - false - end; + verify(H, Path, Level) andalso verify_list(T, Path, Level + 1); verify_list([], _, _) -> true. @@ -419,7 +410,7 @@ edit1_list(_Tag, [], _Op) -> %% Actual transformation of tree structure to desired format. transform(From, To, Opts, File, Tree) -> Filter = if - To==html; To==kwic -> + To =:= html; To =:= kwic -> list_to_atom("docb_tr_" ++ atom_to_list(From) ++ [$2|atom_to_list(To)]); true -> @@ -427,7 +418,7 @@ transform(From, To, Opts, File, Tree) -> [$2|atom_to_list(To)]) end, - case catch apply(Filter, transform, [File, Tree, Opts]) of + case catch Filter:transform(File, Tree, Opts) of %% R5C {'EXIT', {undef, [{Filter, transform, [File, Tree, Opts]}|_]}}-> @@ -459,9 +450,9 @@ transform(From, To, Opts, File, Tree) -> finish_transform(Tree, File, Opts, Filter) -> {Str, NewOpts} = pp(Tree, [], 1, Filter, Opts), Extension = - case catch apply(Filter, extension, [NewOpts]) of + case catch Filter:extension(NewOpts) of {'EXIT', _} -> - apply(Filter, extension, []); + Filter:extension(); Others -> Others end, @@ -606,7 +597,7 @@ include_all(Fd) -> eof -> []; ListOfChars -> - lists:append(ListOfChars, include_all(Fd)) + ListOfChars ++ include_all(Fd) end. extract(File, Fd, StartTag, StopTag, State) -> diff --git a/lib/docbuilder/src/docb_pretty_format.erl b/lib/docbuilder/src/docb_pretty_format.erl index 0c4fb0507b..25dcd8987b 100644 --- a/lib/docbuilder/src/docb_pretty_format.erl +++ b/lib/docbuilder/src/docb_pretty_format.erl @@ -47,7 +47,7 @@ term(Term) -> %% the next line to need an "extra" indent!). term([], Indent) -> {Indent, [$[,$]]}; -term(L, Indent) when list(L) -> +term(L, Indent) when is_list(L) -> case is_string(L) of true -> {Indent, io_lib:write_string(L)}; @@ -59,7 +59,7 @@ term(L, Indent) when list(L) -> write_simple_list(L, Indent) end end; -term(T, Indent) when tuple(T) -> +term(T, Indent) when is_tuple(T) -> case complex_tuple(T) of true -> write_complex_tuple(T, Indent); diff --git a/lib/docbuilder/src/docb_tr_application2html.erl b/lib/docbuilder/src/docb_tr_application2html.erl index 4084cfe6ba..d8cb214d0a 100644 --- a/lib/docbuilder/src/docb_tr_application2html.erl +++ b/lib/docbuilder/src/docb_tr_application2html.erl @@ -119,8 +119,8 @@ transform(File, {application, _Attrs, [Header|Rest]}, Opts0) -> case docb_main:parse1("fascicules", Opts0) of {ok, Parse} -> FascData = get_fasc_data(Parse), - case lists:keysearch(File, 1, FascData) of - {value, {_, _, "YES", _}} -> + case lists:keyfind(File, 1, FascData) of + {_, _, "YES", _} -> OrigFile = docb_util:outfile(File++"_frame", ".html", Opts0), @@ -167,7 +167,7 @@ concat_files([File|Rest], Body, Opts) -> %% Remove the reference manual header [{Ref, [], [_Hdr| NewBody]}] = NewParse, RefParse = [{Ref, [], NewBody}], - lists:append(Body, concat_files(Rest, RefParse, Opts)); + Body ++ concat_files(Rest, RefParse, Opts); errors -> errors end; @@ -216,7 +216,7 @@ make_toc([{lib, Attrs, More}|Rest]) -> make_toc([{com, Attrs, More}|Rest]) -> [{com, Attrs, More}|make_toc(Rest)]; make_toc([{_Tag, _Attrs, More}|Rest]) -> - lists:append(make_toc(More), make_toc(Rest)). + make_toc(More) ++ make_toc(Rest). rule([module|_], {_, [File], _}) -> {"<small><a target=\"document\" href=\"" ++ @@ -280,9 +280,7 @@ get_fasc_data({fascicules, _, Fascs}) -> Fascs). get_avals(Atts) -> - lists:map(fun(Tuple) -> - element(3, Tuple) end, - Atts). + [element(3, Tuple) || Tuple <- Atts]. get_pc_text([{pcdata, _, Text}]) -> Text. diff --git a/lib/docbuilder/src/docb_tr_cite2html.erl b/lib/docbuilder/src/docb_tr_cite2html.erl index 4ecbfa4e91..77f1c4e636 100644 --- a/lib/docbuilder/src/docb_tr_cite2html.erl +++ b/lib/docbuilder/src/docb_tr_cite2html.erl @@ -39,24 +39,22 @@ purge_body([], _) -> purge_body([{pcdata,_Attrs,_More}|Rest], CiteList) -> purge_body(Rest, CiteList); purge_body([{cite,[{"ID","CDATA",ID}],More}|Rest], CiteList) -> - case lists:keysearch(ID, 1, CiteList) of + case lists:keyfind(ID, 1, CiteList) of false -> [{cite, [{"NAME","CDATA",ID}, {"ID","CDATA",ID}], More}| purge_body(Rest, CiteList)]; - {value, {ID, Name, _Description, _Responsible}} -> + {ID, Name, _Description, _Responsible} -> [{cite, [{"NAME","CDATA",Name}, {"ID","CDATA",ID}], More}| purge_body(Rest, CiteList)]; - {value, {ID, Name, _Description}} -> + {ID, Name, _Description} -> [{cite, [{"NAME","CDATA",Name}, {"ID","CDATA",ID}], More}| purge_body(Rest, CiteList)] end; purge_body([{_Tag,_Attrs,More}|Rest], CiteList) -> - lists:append(purge_body(More, CiteList), - purge_body(Rest, CiteList)). + purge_body(More, CiteList) ++ purge_body(Rest, CiteList). rule([header|_], _) -> {drop, ""}; - rule(_, _) -> {drop, ""}. @@ -91,19 +89,19 @@ rule([cite|_], {_,[Name,ID], [{citedef,[],[{pcdata,[],Def}]}]}, Opts) -> false -> []; Value -> Value end, - case lists:keysearch(ID, 1, CiteList) of + case lists:keyfind(ID, 1, CiteList) of false -> {{drop,"\n<dt><a name=\"" ++ ID ++ "\">" ++ "<strong>" ++ ID ++ "</strong></a></dt>\n<dd>" ++ docb_html_util:pcdata_to_html(Def) ++ "\n</dd>\n"}, Opts}; - {value, {ID, Name, Description, _Responsible}} -> + {ID, Name, Description, _Responsible} -> docb_util:message(warning, "Global cite ~s overriding local", [ID]), {{drop,"\n<dt><a name=\"" ++ ID ++ "\">" ++ "<strong>" ++ Name ++ "</strong></a></dt>\n<dd>" ++ docb_html_util:pcdata_to_html(Description) ++ "\n</dd>\n"}, Opts}; - {value, {ID, Name, Description}} -> + {ID, Name, Description} -> docb_util:message(warning, "Global cite ~s overriding local", [ID]), {{drop,"\n<dt><a name=\"" ++ ID ++ "\">" ++ @@ -117,19 +115,19 @@ rule([cite|_], {_,[Name,ID],_}, Opts) -> false -> []; Value -> Value end, - case lists:keysearch(ID, 1, CiteList) of + case lists:keyfind(ID, 1, CiteList) of false -> docb_util:message(error, "The cite ~s has no definition", [ID]), {{drop,"\n<dt><a name=\"" ++ ID ++ "\">" ++ "<strong>" ++ ID ++ "</strong></a></dt>\n<dd>" ++ "??" ++ "\n</dd>\n"}, Opts}; - {value, {ID, Name, Description, _Responsible}} -> + {ID, Name, Description, _Responsible} -> {{drop,"\n<dt><a name=\"" ++ ID ++ "\">" ++ "<strong>" ++ Name ++ "</strong></a></dt>\n<dd>" ++ docb_html_util:pcdata_to_html(Description) ++ "\n</dd>\n"}, Opts}; - {value, {ID, Name, Description}} -> + {ID, Name, Description} -> {{drop,"\n<dt><a name=\"" ++ ID ++ "\">" ++ "<strong>" ++ Name ++ "</strong></a></dt>\n<dd>" ++ docb_html_util:pcdata_to_html(Description) ++ "\n</dd>\n"}, Opts} diff --git a/lib/docbuilder/src/docb_tr_index2html.erl b/lib/docbuilder/src/docb_tr_index2html.erl index bbf419f3ef..312342add2 100644 --- a/lib/docbuilder/src/docb_tr_index2html.erl +++ b/lib/docbuilder/src/docb_tr_index2html.erl @@ -73,9 +73,7 @@ prune_flat([], _) -> []. keep_pcdata(Trees) -> - lists:filter(fun({pcdata, _, _}) -> true; - (_) -> false - end, Trees). + [T || T = {pcdata, _, _} <- Trees]. new_trees(FileFuncs) -> Files0 = [{File, RefType} || {File, RefType, _} <- FileFuncs], diff --git a/lib/docbuilder/src/docb_tr_part2html.erl b/lib/docbuilder/src/docb_tr_part2html.erl index dd44c4a8df..30befe8432 100644 --- a/lib/docbuilder/src/docb_tr_part2html.erl +++ b/lib/docbuilder/src/docb_tr_part2html.erl @@ -93,8 +93,8 @@ transform(File, {part, _Attrs, [Header| Rest]}, Opts0) -> case docb_main:parse1("fascicules", Opts0) of {ok, Parse} -> FascData = get_fasc_data(Parse), - case lists:keysearch(File, 1, FascData) of - {value, {_, _, "YES", _}} -> + case lists:keyfind(File, 1, FascData) of + {_, _, "YES", _} -> OrigFile = docb_util:outfile(File++"_frame", ".html", Opts0), @@ -137,7 +137,7 @@ concat_files([File | Rest], Body, ChLevel, Opts, TP, TOpts, Ext) -> case docb_main:parse1(File, Opts) of {ok, Parse} -> {TopTag, Attrs, [Header = {header, _, HeaderContents} | More]} = Parse, - {value,{title,_,Title}} = lists:keysearch(title,1,HeaderContents), + {title,_,Title} = lists:keyfind(title,1,HeaderContents), NewMore = [{section, [], [{title, [], Title}| More]}], NewParse = {TopTag, Attrs, [Header| NewMore]}, if @@ -156,9 +156,8 @@ concat_files([File | Rest], Body, ChLevel, Opts, TP, TOpts, Ext) -> docb_html_util:number(NewParse, integer_to_list(ChLevel), File), {_, [], [_| NewBody]} = NumberTree, - lists:append(Body, - concat_files(Rest, NewBody, ChLevel+1, Opts, - TP, TOpts, Ext)); + Body ++ concat_files(Rest, NewBody, ChLevel+1, Opts, + TP, TOpts, Ext); errors -> throw({error,"Parse error when building chapter "++File}) end; @@ -232,9 +231,7 @@ get_fasc_data({fascicules, _, Fascs}) -> Fascs). get_avals(Atts) -> - lists:map(fun(Tuple) -> - element(3, Tuple) end, - Atts). + [element(3, Tuple) || Tuple <- Atts]. get_pc_text([{pcdata, _, Text}]) -> Text. diff --git a/lib/docbuilder/src/docb_tr_term2html.erl b/lib/docbuilder/src/docb_tr_term2html.erl index 0a993cebb1..a3c4a5312a 100644 --- a/lib/docbuilder/src/docb_tr_term2html.erl +++ b/lib/docbuilder/src/docb_tr_term2html.erl @@ -39,24 +39,22 @@ purge_body([], _) -> purge_body([{pcdata,_Attrs,_More}|Rest], TermList) -> purge_body(Rest, TermList); purge_body([{term,[{"ID","CDATA",ID}],More}|Rest], TermList) -> - case lists:keysearch(ID, 1, TermList) of + case lists:keyfind(ID, 1, TermList) of false -> [{term,[{"NAME","CDATA",ID},{"ID","CDATA",ID}],More}| purge_body(Rest, TermList)]; - {value, {ID, Name, _Description, _Responsible}} -> + {ID, Name, _Description, _Responsible} -> [{term,[{"NAME","CDATA",Name},{"ID","CDATA",ID}],More}| purge_body(Rest, TermList)]; - {value, {ID, Name, _Description}} -> + {ID, Name, _Description} -> [{term,[{"NAME","CDATA",Name},{"ID","CDATA",ID}],More}| purge_body(Rest, TermList)] end; purge_body([{_Tag,_Attrs,More}|Rest], TermList) -> - lists:append(purge_body(More, TermList), - purge_body(Rest, TermList)). + purge_body(More, TermList) ++ purge_body(Rest, TermList). rule([header|_], _) -> {drop, ""}; - rule(_, _) -> {drop, ""}. @@ -82,19 +80,19 @@ rule([term|_], {_,[Name,ID],[{termdef,[],[{pcdata,[],Def}]}]}, Opts) -> false -> []; Value -> Value end, - case lists:keysearch(ID, 1, TermList) of + case lists:keyfind(ID, 1, TermList) of false -> {{drop,"\n<dt><a name=\"" ++ ID ++ "\">" ++ "<strong>" ++ ID ++ "</strong></a>\n</dt>\n<dd>" ++ docb_html_util:pcdata_to_html(Def) ++ "\n</dd>\n"}, Opts}; - {value, {ID, Name, Description, _Responsible}} -> + {ID, Name, Description, _Responsible} -> docb_util:message(warning, "Global term ~s overriding local", [ID]), {{drop,"\n<dt><a name=\"" ++ ID ++ "\">" ++ "<strong>" ++ Name ++ "</strong></a></dt>\n<dd>" ++ docb_html_util:pcdata_to_html(Description) ++ "\n</dd>\n"}, Opts}; - {value, {ID, Name, Description}} -> + {ID, Name, Description} -> docb_util:message(warning, "Global term ~s overriding local", [ID]), {{drop, "\n<dt><a name=\"" ++ ID ++ "\">" ++ @@ -107,19 +105,19 @@ rule([term|_], {_,[Name,ID],_}, Opts) -> false -> []; Value -> Value end, - case lists:keysearch(ID, 1, TermList) of + case lists:keyfind(ID, 1, TermList) of false -> docb_util:message(error, "The term ~s has no definition", [ID]), {{drop, "\n<dt><a name=\"" ++ ID ++ "\">" ++ "<strong>" ++ ID ++ "</strong></a></dt>\n<dd>" ++ "??" ++ "\n</dd>\n"}, Opts}; - {value, {ID, Name, Description, _Responsible}} -> + {ID, Name, Description, _Responsible} -> {{drop, "\n<dt><a name=\"" ++ ID ++ "\">" ++ "<strong>" ++ Name ++ "</strong></a></dt>\n<dd>" ++ docb_html_util:pcdata_to_html(Description) ++ "\n</dd>\n"}, Opts}; - {value, {ID, Name, Description}} -> + {ID, Name, Description} -> {{drop, "\n<dt><a name=\"" ++ ID ++ "\">" ++ "<strong>" ++ Name ++ "</strong></a></dt>\n<dd>" ++ docb_html_util:pcdata_to_html(Description) ++ "\n</dd>\n"}, Opts} diff --git a/lib/docbuilder/src/docb_transform.erl b/lib/docbuilder/src/docb_transform.erl index a432038adf..9c7561b07b 100644 --- a/lib/docbuilder/src/docb_transform.erl +++ b/lib/docbuilder/src/docb_transform.erl @@ -135,8 +135,8 @@ parse([Opt | _RawOpts], _Opts) -> get_defs(Type, File, Opts) -> Key = {defs,Type}, {PrevDefs, Opts2} = - case lists:keysearch(Key, 1, Opts) of - {value, {_, Defs0}} -> + case lists:keyfind(Key, 1, Opts) of + {_, Defs0} -> {Defs0, lists:keydelete(Key, 1, Opts)}; false -> {[], Opts} diff --git a/lib/docbuilder/src/docb_util.erl b/lib/docbuilder/src/docb_util.erl index 59673ef3a4..9b2eec7733 100644 --- a/lib/docbuilder/src/docb_util.erl +++ b/lib/docbuilder/src/docb_util.erl @@ -61,7 +61,7 @@ html_snippet(What, Opts) -> case lookup_option(html_mod, Opts) of false -> ""; Module -> - case catch apply(Module, What, []) of + case catch Module:What() of HTML when is_list(HTML) -> HTML; {'EXIT', {undef, _}} -> @@ -82,7 +82,7 @@ html_snippet(What, Arg, Opts) -> case lookup_option(html_mod, Opts) of false -> ""; Module -> - case catch apply(Module, What, [Arg]) of + case catch Module:What(Arg) of HTML when is_list(HTML) -> HTML; {'EXIT', {undef, _}} -> @@ -106,8 +106,8 @@ html_snippet(What, Arg, Opts) -> %% lookup_option(Opt, Opts) -> Value | false lookup_option(Opt, Opts) -> - case lists:keysearch(Opt, 1, Opts) of - {value, {Opt,Value}} -> Value; + case lists:keyfind(Opt, 1, Opts) of + {Opt,Value} -> Value; false -> false end. diff --git a/lib/docbuilder/src/docb_xmerl_tree_cb.erl b/lib/docbuilder/src/docb_xmerl_tree_cb.erl index d57f55bff8..bc62069230 100644 --- a/lib/docbuilder/src/docb_xmerl_tree_cb.erl +++ b/lib/docbuilder/src/docb_xmerl_tree_cb.erl @@ -188,8 +188,8 @@ attrs(DTD, Tag, GivenAttrs) -> merge_attrs(Tag, default_attrs(DTD, Tag), GivenAttrs). merge_attrs(Tag, [{NameA, Type, DefVal}|Default], GivenAttrs) -> - Val = case lists:keysearch(NameA, #xmlAttribute.name, GivenAttrs) of - {value, #xmlAttribute{value=Val0}} -> Val0; + Val = case lists:keyfind(NameA, #xmlAttribute.name, GivenAttrs) of + #xmlAttribute{value=Val0} -> Val0; false -> DefVal end, Attr = {attr_name(NameA), Type, attr_val(Type, Val)}, diff --git a/lib/edoc/src/edoc_lib.erl b/lib/edoc/src/edoc_lib.erl index 47e61f7932..5b7fb1e0d2 100644 --- a/lib/edoc/src/edoc_lib.erl +++ b/lib/edoc/src/edoc_lib.erl @@ -472,7 +472,7 @@ uri_get_file(File0) -> uri_get_http(URI) -> %% Try using option full_result=false - case catch {ok, http:request(get, {URI,[]}, [], + case catch {ok, httpc:request(get, {URI,[]}, [], [{full_result, false}])} of {'EXIT', _} -> uri_get_http_r10(URI); @@ -482,7 +482,7 @@ uri_get_http(URI) -> uri_get_http_r10(URI) -> %% Try most general form of request - Result = (catch {ok, http:request(get, {URI,[]}, [], [])}), + Result = (catch {ok, httpc:request(get, {URI,[]}, [], [])}), uri_get_http_1(Result, URI). uri_get_http_1(Result, URI) -> diff --git a/lib/erl_docgen/doc/src/notes.xml b/lib/erl_docgen/doc/src/notes.xml index ee74e598d9..ce6ba3d923 100644 --- a/lib/erl_docgen/doc/src/notes.xml +++ b/lib/erl_docgen/doc/src/notes.xml @@ -29,7 +29,25 @@ <file>notes.xml</file> </header> <p>This document describes the changes made to the erl_docgen application.</p> - <section><title>Erl_Docgen 0.2</title> + <section><title>erl_docgen 0.2.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The text within code examples (CODE and PRE tags) will not be + space normalized for man pages. + </p> + <p> + Own Id: OTP-8476 + </p> + </item> + </list> + </section> + + </section> + + <section><title>erl_docgen 0.2</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/erl_docgen/priv/xsl/db_man.xsl b/lib/erl_docgen/priv/xsl/db_man.xsl index a2b1e755e2..71c4a66707 100644 --- a/lib/erl_docgen/priv/xsl/db_man.xsl +++ b/lib/erl_docgen/priv/xsl/db_man.xsl @@ -336,6 +336,16 @@ <xsl:text>></xsl:text> </xsl:template> + <!-- Do not noramlize any text within pre and code tags. --> + <xsl:template match="pre/text()"> + <xsl:value-of select="."/> + </xsl:template> + + <xsl:template match="code/text()"> + <xsl:value-of select="."/> + </xsl:template> + + <!-- Replace ' by \&' ans . by \&. --> <xsl:template match="text()"> <xsl:variable name="startstring"> diff --git a/lib/erl_docgen/vsn.mk b/lib/erl_docgen/vsn.mk index dd74ae9ddd..5b14051034 100644 --- a/lib/erl_docgen/vsn.mk +++ b/lib/erl_docgen/vsn.mk @@ -17,6 +17,6 @@ # %CopyrightEnd% # -ERL_DOCGEN_VSN = 0.2 +ERL_DOCGEN_VSN = 0.2.1 TICKETS = diff --git a/lib/erl_interface/doc/src/ei_connect.xml b/lib/erl_interface/doc/src/ei_connect.xml index abf705f9e2..927395d1bf 100644 --- a/lib/erl_interface/doc/src/ei_connect.xml +++ b/lib/erl_interface/doc/src/ei_connect.xml @@ -54,7 +54,7 @@ the operation, if the primitive does not complete within the time specified, the function will return an error and <c><![CDATA[erl_errno]]></c> will be set to <c><![CDATA[ETIMEDOUT]]></c>. With - communication primitive is ment an operation on the socket, like + communication primitive is meant an operation on the socket, like <c><![CDATA[connect]]></c>, <c><![CDATA[accept]]></c>, <c><![CDATA[recv]]></c> or <c><![CDATA[send]]></c>.</p> <p>Obviously the timeouts are for implementing fault tolerance, not to keep hard realtime promises. The <c><![CDATA[_tmo]]></c> functions diff --git a/lib/erl_interface/doc/src/notes.xml b/lib/erl_interface/doc/src/notes.xml index 14aec4a4d9..848da9cb23 100644 --- a/lib/erl_interface/doc/src/notes.xml +++ b/lib/erl_interface/doc/src/notes.xml @@ -30,6 +30,45 @@ </header> <p>This document describes the changes made to the Erl_interface application.</p> +<section><title>Erl_Interface 3.7</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>compact IEEE 754 double encoding in external binary + format for ei</p> <list><item><p>Implement the compact + IEEE 754 double encoding in external binary format for + ei. Encoding for ei now always produces the NEW_FLOAT_EXT + format. Decoding and term printing handle both the old + ERL_FLOAT_EXT encoding and the new NEW_FLOAT_EXT + encoding. </p></item> <item><p>Legacy erl_interface code + also handles the new encoding, but still produces the + ERL_FLOAT_EXT encoding by default.</p></item> + <item><p>Also enable the DFLAG_NEW_FLOATS distribution + flag.</p></item> <item><p>ei_get_type() will return + ERL_FLOAT_EXT regardless if the external format is + encoded with ERL_FLOAT_EXT or NEW_FLOAT_EXT for + doubles.</p></item> <item><p>Reduce the number of copies + of the code for encoding and decoding doubles throughout + ei and erl_interface by instead calling the ei encoding + and decoding functions wherever possible.</p></item> + <item><p>Restore commented-out float tests in + ei_decode_SUITE and ei_encode_SUITE in + lib/erl_interface/test. Modify them to make them match + the style of other tests in the same suites.</p></item> + </list> <p>These changes are based on an ei float patch + from Serge Aleynikov originally submitted against R12B-2 + in July 2008 and reworked by Steve Vinoski May 2010.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8684</p> + </item> + </list> + </section> + +</section> + <section><title>Erl_Interface 3.6.5</title> <section><title>Improvements and New Features</title> diff --git a/lib/erl_interface/include/ei.h b/lib/erl_interface/include/ei.h index 729b9fc367..466d84bb99 100644 --- a/lib/erl_interface/include/ei.h +++ b/lib/erl_interface/include/ei.h @@ -719,11 +719,9 @@ int ei_x_encode_bignum(ei_x_buff *x, mpz_t obj); #define EI_LONGLONG __int64 #define EI_ULONGLONG unsigned __int64 #else -#ifndef VXWORKS #define EI_LONGLONG long long #define EI_ULONGLONG unsigned long long #endif -#endif #ifndef VXWORKS int ei_decode_longlong(const char *buf, int *index, EI_LONGLONG *p); diff --git a/lib/erl_interface/src/legacy/erl_marshal.c b/lib/erl_interface/src/legacy/erl_marshal.c index c57c552b90..18315bfbd3 100644 --- a/lib/erl_interface/src/legacy/erl_marshal.c +++ b/lib/erl_interface/src/legacy/erl_marshal.c @@ -26,6 +26,7 @@ #include <ctype.h> #include <sys/types.h> #include <string.h> +#include <limits.h> #include "erl_interface.h" #include "erl_marshal.h" @@ -178,26 +179,14 @@ int erl_encode_it(ETERM *ep, unsigned char **ext, int dist) return 0; case ERL_INTEGER: - i = ep->uval.ival.i; - /* ERL_SMALL_BIG */ - if ((i > ERL_MAX) || (i < ERL_MIN)) { - *(*ext)++ = ERL_SMALL_BIG_EXT; - *(*ext)++ = 4; /* four bytes */ - if ((*(*ext)++ = ((i>>31) & 0x01))) /* sign byte */ - i = -i; - *(*ext)++ = i & 0xff; /* LSB first */ - *(*ext)++ = (i >> 8) & 0xff; - *(*ext)++ = (i >> 16) & 0xff; - *(*ext)++ = (i >> 24) & 0x7f; /* Don't include the sign bit */ - return 0; - } + i = ep->uval.ival.i; /* SMALL_INTEGER */ if ((i < 256) && (i >= 0)) { *(*ext)++ = ERL_SMALL_INTEGER_EXT; *(*ext)++ = i & 0xff; return 0; } - /* INTEGER */ + /* R14B: Use all 32 bits of INTEGER_EXT */ *(*ext)++ = ERL_INTEGER_EXT; *(*ext)++ = (i >> 24) & 0xff; *(*ext)++ = (i >> 16) & 0xff; @@ -208,23 +197,23 @@ int erl_encode_it(ETERM *ep, unsigned char **ext, int dist) case ERL_U_INTEGER: u = ep->uval.uival.u; /* ERL_U_SMALL_BIG */ - if (u > ERL_MAX) { - *(*ext)++ = ERL_SMALL_BIG_EXT; - *(*ext)++ = 4; /* four bytes */ - *(*ext)++ = 0; /* sign byte */ - *(*ext)++ = u & 0xff; /* LSB first */ - *(*ext)++ = (u >> 8) & 0xff; - *(*ext)++ = (u >> 16) & 0xff; - *(*ext)++ = (u >> 24) & 0xff; - return 0; + if ((int)u < 0) { + *(*ext)++ = ERL_SMALL_BIG_EXT; + *(*ext)++ = 4; /* four bytes */ + *(*ext)++ = 0; /* sign byte */ + *(*ext)++ = u & 0xff; /* LSB first */ + *(*ext)++ = (u >> 8) & 0xff; + *(*ext)++ = (u >> 16) & 0xff; + *(*ext)++ = (u >> 24) & 0xff; + return 0; } /* SMALL_INTEGER */ - if ((u < 256) && (u >= 0)) { + if (u < 256) { *(*ext)++ = ERL_SMALL_INTEGER_EXT; *(*ext)++ = u & 0xff; return 0; } - /* INTEGER */ + /* R14B: Use all 32 bits of INTEGER_EXT */ *(*ext)++ = ERL_INTEGER_EXT; *(*ext)++ = (u >> 24) & 0xff; *(*ext)++ = (u >> 16) & 0xff; @@ -234,29 +223,28 @@ int erl_encode_it(ETERM *ep, unsigned char **ext, int dist) case ERL_LONGLONG: l = ep->uval.llval.i; /* ERL_SMALL_BIG */ - if ((l > ((long long) ERL_MAX)) || - (l < ((long long) ERL_MIN))) { - *(*ext)++ = ERL_SMALL_BIG_EXT; - *(*ext)++ = 8; /* eight bytes */ - if ((*(*ext)++ = ((l>>63) & 0x01))) /* sign byte */ + if (l > ((long long) INT_MAX) || l < ((long long) INT_MIN)) { + *(*ext)++ = ERL_SMALL_BIG_EXT; + *(*ext)++ = 8; + if ((*(*ext)++ = (l<0))) /* sign byte */ l = -l; - *(*ext)++ = l & 0xff; /* LSB first */ + *(*ext)++ = l & 0xff; /* LSB first */ *(*ext)++ = (l >> 8) & 0xff; *(*ext)++ = (l >> 16) & 0xff; - *(*ext)++ = (l >> 24) & 0xff; - *(*ext)++ = (l >> 32) & 0xff; - *(*ext)++ = (l >> 40) & 0xff; - *(*ext)++ = (l >> 48) & 0xff; - *(*ext)++ = (l >> 56) & 0x7f; /* Don't include the sign bit */ + *(*ext)++ = (l >> 24) & 0xff; + *(*ext)++ = (l >> 32) & 0xff; + *(*ext)++ = (l >> 40) & 0xff; + *(*ext)++ = (l >> 48) & 0xff; + *(*ext)++ = (l >> 56) & 0xff; return 0; } /* SMALL_INTEGER */ if ((l < 256) && (l >= 0)) { - *(*ext)++ = ERL_SMALL_INTEGER_EXT; - *(*ext)++ = l & 0xff; - return 0; + *(*ext)++ = ERL_SMALL_INTEGER_EXT; + *(*ext)++ = l & 0xff; + return 0; } - /* INTEGER */ + /* R14B: Use all 32 bits of INTEGER_EXT */ *(*ext)++ = ERL_INTEGER_EXT; *(*ext)++ = (l >> 24) & 0xff; *(*ext)++ = (l >> 16) & 0xff; @@ -267,7 +255,7 @@ int erl_encode_it(ETERM *ep, unsigned char **ext, int dist) case ERL_U_LONGLONG: ul = ep->uval.ullval.u; /* ERL_U_SMALL_BIG */ - if (ul > ((unsigned long long) ERL_MAX)) { + if (ul > ((unsigned long long) INT_MAX)) { *(*ext)++ = ERL_SMALL_BIG_EXT; *(*ext)++ = 8; /* eight bytes */ *(*ext)++ = 0; /* sign byte */ @@ -287,7 +275,7 @@ int erl_encode_it(ETERM *ep, unsigned char **ext, int dist) *(*ext)++ = ul & 0xff; return 0; } - /* INTEGER */ + /* R14B: Use all 32 bits of INTEGER_EXT */ *(*ext)++ = ERL_INTEGER_EXT; *(*ext)++ = (ul >> 24) & 0xff; *(*ext)++ = (ul >> 16) & 0xff; @@ -732,11 +720,6 @@ static ETERM *erl_decode_it(unsigned char **ext) if (arity > 8) goto big_truncate; - if (arity == 8 && ((*ext)[7] & 0x80) && sign) { - /* MSB already occupied ! */ - goto big_truncate; - } - if (arity == 4 && ((*ext)[3] & 0x80) && !sign) { /* It will fit into an unsigned int !! */ u = (((*ext)[3] << 24)|((*ext)[2])<< 16|((*ext)[1]) << 8 |(**ext)); @@ -747,14 +730,10 @@ static ETERM *erl_decode_it(unsigned char **ext) return ep; } else if (arity == 4 && !((*ext)[3] & 0x80)) { /* It will fit into an int !! - * Note: It comes in "one's-complement notation" */ - if (sign) - i = (int) (~(((*ext)[3] << 24) | ((*ext)[2])<< 16 | - ((*ext)[1]) << 8 | (**ext)) | (unsigned int) sign); - else - i = (int) (((*ext)[3] << 24) | ((*ext)[2])<< 16 | - ((*ext)[1]) << 8 | (**ext)); + i = (int) (((*ext)[3] << 24) | ((*ext)[2])<< 16 | + ((*ext)[1]) << 8 | (**ext)); + if (sign) i = -i; ERL_TYPE(ep) = ERL_INTEGER; ep->uval.ival.i = i; *ext += arity; @@ -780,8 +759,10 @@ static ETERM *erl_decode_it(unsigned char **ext) for(x = 0 ; x < arity ; x++) { l |= ((long long)(*ext)[x]) << ((long long)(8*x)); } - - if (sign) l = (long long) (~l | (unsigned long long) sign); + if (sign) { + l = -l; + if (l > 0) goto big_truncate; + } ERL_TYPE(ep) = ERL_LONGLONG; ep->uval.llval.i = l; diff --git a/lib/erl_interface/src/misc/ei_decode_term.c b/lib/erl_interface/src/misc/ei_decode_term.c index ddcbfa5a9a..75c5dc9460 100644 --- a/lib/erl_interface/src/misc/ei_decode_term.c +++ b/lib/erl_interface/src/misc/ei_decode_term.c @@ -34,7 +34,6 @@ int ei_decode_ei_term(const char* buf, int* index, ei_term* term) const char* s = buf + *index, * s0 = s; int len, i, n, sign; char c; - double f; if (term == NULL) return -1; c = term->ei_type = get8(s); diff --git a/lib/erl_interface/src/prog/erl_call.c b/lib/erl_interface/src/prog/erl_call.c index f0d638324d..93b84cbb36 100644 --- a/lib/erl_interface/src/prog/erl_call.c +++ b/lib/erl_interface/src/prog/erl_call.c @@ -812,7 +812,8 @@ static int get_module(char **mbuf, char **mname) *mname = (char *) calloc(i+1, sizeof(char)); memcpy(*mname, start, i); } - free(mbuf); /* Allocated in read_stdin() */ + if (*mbuf) + free(*mbuf); /* Allocated in read_stdin() */ return len; diff --git a/lib/erl_interface/test/ei_decode_SUITE.erl b/lib/erl_interface/test/ei_decode_SUITE.erl index c6858b45ad..09a37409f2 100644 --- a/lib/erl_interface/test/ei_decode_SUITE.erl +++ b/lib/erl_interface/test/ei_decode_SUITE.erl @@ -222,14 +222,16 @@ send_integers(P) -> ?line send_term_as_binary(P,256), % INTEGER_EXT smallest pos (*) ?line send_term_as_binary(P,-1), % INTEGER_EXT largest neg - ?line send_term_as_binary(P, 16#07ffffff), % INTEGER_EXT largest (28 bits) - ?line send_term_as_binary(P,-16#08000000), % INTEGER_EXT smallest - ?line send_term_as_binary(P, 16#08000000), % SMALL_BIG_EXT smallest pos(*) - ?line send_term_as_binary(P,-16#08000001), % SMALL_BIG_EXT largest neg (*) - - ?line send_term_as_binary(P, 16#7fffffff), % SMALL_BIG_EXT largest i32 - ?line send_term_as_binary(P,-16#80000000), % SMALL_BIG_EXT smallest i32 - + ?line send_term_as_binary(P, 16#07ffffff), % INTEGER_EXT old largest (28 bits) + ?line send_term_as_binary(P,-16#08000000), % INTEGER_EXT old smallest + ?line send_term_as_binary(P, 16#08000000), % SMALL_BIG_EXT old smallest pos(*) + ?line send_term_as_binary(P,-16#08000001), % SMALL_BIG_EXT old largest neg (*) + + ?line send_term_as_binary(P, 16#7fffffff), % INTEGER_EXT new largest (32 bits) + ?line send_term_as_binary(P,-16#80000000), % INTEGER_EXT new smallest (32 bis) + ?line send_term_as_binary(P, 16#80000000), % SMALL_BIG_EXT new smallest pos(*) + ?line send_term_as_binary(P,-16#80000001), % SMALL_BIG_EXT new largest neg (*) + case erlang:system_info(wordsize) of 4 -> ?line send_term_as_binary(P, 16#80000000),% SMALL_BIG_EXT u32 @@ -266,15 +268,17 @@ send_integers2(P) -> ?line send_term_as_binary(P,255), % SMALL_INTEGER_EXT largest ?line send_term_as_binary(P,256), % INTEGER_EXT smallest pos (*) ?line send_term_as_binary(P,-1), % INTEGER_EXT largest neg + + ?line send_term_as_binary(P, 16#07ffffff), % INTEGER_EXT old largest (28 bits) + ?line send_term_as_binary(P,-16#08000000), % INTEGER_EXT old smallest + ?line send_term_as_binary(P, 16#08000000), % SMALL_BIG_EXT old smallest pos(*) + ?line send_term_as_binary(P,-16#08000001), % SMALL_BIG_EXT old largest neg (*) - ?line send_term_as_binary(P, 16#07ffffff), % INTEGER_EXT largest (28 bits) - ?line send_term_as_binary(P,-16#08000000), % INTEGER_EXT smallest - ?line send_term_as_binary(P, 16#08000000), % SMALL_BIG_EXT smallest pos(*) - ?line send_term_as_binary(P,-16#08000001), % SMALL_BIG_EXT largest neg (*) + ?line send_term_as_binary(P, 16#7fffffff), % INTEGER_EXT new largest (32 bits) + ?line send_term_as_binary(P,-16#80000000), % INTEGER_EXT new smallest + ?line send_term_as_binary(P, 16#80000000), % SMALL_BIG_EXT new smallest pos(*) + ?line send_term_as_binary(P,-16#80000001), % SMALL_BIG_EXT new largest neg (*) - ?line send_term_as_binary(P, 16#7fffffff), % SMALL_BIG_EXT largest i32 - ?line send_term_as_binary(P,-16#80000000), % SMALL_BIG_EXT smallest i32 - ?line send_term_as_binary(P, 16#80000000),% SMALL_BIG_EXT u32 ?line send_term_as_binary(P, 16#ffffffff),% SMALL_BIG_EXT largest u32 ?line send_term_as_binary(P, 16#7fffffffffff), % largest i48 diff --git a/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c b/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c index 5447e2deb3..b349138ae9 100644 --- a/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c +++ b/lib/erl_interface/test/ei_decode_SUITE_data/ei_decode_test.c @@ -246,13 +246,23 @@ TESTCASE(test_ei_decode_long) EI_DECODE_2 (decode_long, 5, long, 256); EI_DECODE_2 (decode_long, 5, long, -1); + /* Old 28 bit limits for INTEGER_EXT */ EI_DECODE_2 (decode_long, 5, long, 0x07ffffff); EI_DECODE_2 (decode_long, 5, long, -0x08000000); - EI_DECODE_2 (decode_long, 7, long, 0x08000000); - EI_DECODE_2 (decode_long, 7, long, -0x08000001); + EI_DECODE_2 (decode_long, 5, long, 0x08000000); + EI_DECODE_2 (decode_long, 5, long, -0x08000001); - EI_DECODE_2 (decode_long, 7, long, 0x7fffffff); - EI_DECODE_2 (decode_long, 7, long, -ll(0x80000000)); /* Strange :-( */ + /* New 32 bit limits for INTEGER_EXT */ + EI_DECODE_2 (decode_long, 5, long, 0x7fffffff); + EI_DECODE_2 (decode_long, 5, long, -ll(0x80000000)); /* Strange :-( */ + if (sizeof(long) > 4) { + EI_DECODE_2(decode_long, 7, long, 0x80000000); + EI_DECODE_2(decode_long, 7, long, -ll(0x80000001)); + } + else { + EI_DECODE_2_FAIL(decode_long, 7, long, 0x80000000); + EI_DECODE_2_FAIL(decode_long, 7, long, -ll(0x80000001)); + } EI_DECODE_2_FAIL(decode_long, 7, long, 0x80000000); EI_DECODE_2_FAIL(decode_long, 7, long, 0xffffffff); @@ -280,11 +290,13 @@ TESTCASE(test_ei_decode_ulong) EI_DECODE_2 (decode_ulong, 5, unsigned long, 0x07ffffff); EI_DECODE_2_FAIL(decode_ulong, 5, unsigned long, -0x08000000); - EI_DECODE_2 (decode_ulong, 7, unsigned long, 0x08000000); - EI_DECODE_2_FAIL(decode_ulong, 7, unsigned long, -0x08000001); + EI_DECODE_2 (decode_ulong, 5, unsigned long, 0x08000000); + EI_DECODE_2_FAIL(decode_ulong, 5, unsigned long, -0x08000001); - EI_DECODE_2 (decode_ulong, 7, unsigned long, 0x7fffffff); - EI_DECODE_2_FAIL(decode_ulong, 7, unsigned long, -ll(0x80000000)); + EI_DECODE_2 (decode_ulong, 5, unsigned long, 0x7fffffff); + EI_DECODE_2_FAIL(decode_ulong, 5, unsigned long, -ll(0x80000000)); + EI_DECODE_2 (decode_ulong, 7, unsigned long, 0x80000000); + EI_DECODE_2_FAIL(decode_ulong, 7, unsigned long, -ll(0x80000001)); if (sizeof(long) > 4) { EI_DECODE_2 (decode_ulong, 11, unsigned long, ll(0x8000000000000000)); @@ -319,13 +331,14 @@ TESTCASE(test_ei_decode_longlong) EI_DECODE_2 (decode_longlong, 5, EI_LONGLONG, 0x07ffffff); EI_DECODE_2 (decode_longlong, 5, EI_LONGLONG, -0x08000000); - EI_DECODE_2 (decode_longlong, 7, EI_LONGLONG, 0x08000000); - EI_DECODE_2 (decode_longlong, 7, EI_LONGLONG, -0x08000001); - - EI_DECODE_2 (decode_longlong, 7, EI_LONGLONG, 0x7fffffff); - EI_DECODE_2 (decode_longlong, 7, EI_LONGLONG, -ll(0x80000000)); + EI_DECODE_2 (decode_longlong, 5, EI_LONGLONG, 0x08000000); + EI_DECODE_2 (decode_longlong, 5, EI_LONGLONG, -0x08000001); + EI_DECODE_2 (decode_longlong, 5, EI_LONGLONG, 0x7fffffff); + EI_DECODE_2 (decode_longlong, 5, EI_LONGLONG, -ll(0x80000000)); EI_DECODE_2 (decode_longlong, 7, EI_LONGLONG, 0x80000000); + EI_DECODE_2 (decode_longlong, 7, EI_LONGLONG, -ll(0x80000001)); + EI_DECODE_2 (decode_longlong, 7, EI_LONGLONG, 0xffffffff); EI_DECODE_2 (decode_longlong, 9, EI_LONGLONG, ll(0x7fffffffffff)); @@ -352,13 +365,14 @@ TESTCASE(test_ei_decode_ulonglong) EI_DECODE_2 (decode_ulonglong, 5, EI_ULONGLONG, 0x07ffffff); EI_DECODE_2_FAIL(decode_ulonglong, 5, EI_ULONGLONG, -0x08000000); - EI_DECODE_2 (decode_ulonglong, 7, EI_ULONGLONG, 0x08000000); - EI_DECODE_2_FAIL(decode_ulonglong, 7, EI_ULONGLONG, -0x08000001); - - EI_DECODE_2 (decode_ulonglong, 7, EI_ULONGLONG, 0x7fffffff); - EI_DECODE_2_FAIL(decode_ulonglong, 7, EI_ULONGLONG, -ll(0x80000000)); + EI_DECODE_2 (decode_ulonglong, 5, EI_ULONGLONG, 0x08000000); + EI_DECODE_2_FAIL(decode_ulonglong, 5, EI_ULONGLONG, -0x08000001); + EI_DECODE_2 (decode_ulonglong, 5, EI_ULONGLONG, 0x7fffffff); + EI_DECODE_2_FAIL(decode_ulonglong, 5, EI_ULONGLONG, -ll(0x80000000)); EI_DECODE_2 (decode_ulonglong, 7, EI_ULONGLONG, 0x80000000); + EI_DECODE_2_FAIL(decode_ulonglong, 7, EI_ULONGLONG, -0x80000001); + EI_DECODE_2 (decode_ulonglong, 7, EI_ULONGLONG, 0xffffffff); EI_DECODE_2 (decode_ulonglong, 9, EI_ULONGLONG, ll(0x7fffffffffff)); diff --git a/lib/erl_interface/test/erl_eterm_SUITE_data/eterm_test.c b/lib/erl_interface/test/erl_eterm_SUITE_data/eterm_test.c index 6b2ec8f766..f273efd532 100644 --- a/lib/erl_interface/test/erl_eterm_SUITE_data/eterm_test.c +++ b/lib/erl_interface/test/erl_eterm_SUITE_data/eterm_test.c @@ -63,32 +63,108 @@ TESTCASE(build_terms) report(1); } +static int abs_and_sign(ETERM* v, unsigned long long* av, int* sign) +{ + long long sv; + switch (ERL_TYPE(v)) { + case ERL_INTEGER: sv = ERL_INT_VALUE(v); break; + case ERL_U_INTEGER: *av = ERL_INT_UVALUE(v); *sign = 0; return 1; + case ERL_LONGLONG: sv = ERL_LL_VALUE(v); break; + case ERL_U_LONGLONG: *av = ERL_LL_UVALUE(v); *sign = 0; return 1; + default: return 0; + } + if (sv < 0) { + *av = -sv; + *sign = 1; + } + else { + *av = sv; + *sign = 0; + } + return 1; +} + +/* Shouldn't erl_match() cope with this? +*/ +static int eq_ints(ETERM* a, ETERM* b) +{ + unsigned long long a_abs, b_abs; + int a_sign, b_sign; + return abs_and_sign(a, &a_abs, &a_sign) && abs_and_sign(b, &b_abs, &b_sign) + && (a_abs == b_abs) && (a_sign == b_sign); +} + +static void encode_decode(ETERM* original, const char* text) +{ + static unsigned char encoded[16*1024]; + ETERM* new_terms; + int bytes = erl_encode(original, encoded); + + if (bytes == 0) { + fail("failed to encode terms"); + } + else if (bytes > sizeof(encoded)) { + fail("encoded terms buffer overflow"); + } + else if ((new_terms = erl_decode(encoded)) == NULL) { + fail("failed to decode terms"); + } + else if (!erl_match(original, new_terms) && !eq_ints(original, new_terms)) { + erl_print_term(stderr, original); + fprintf(stderr, "(%i) != (%i)", ERL_TYPE(original), ERL_TYPE(new_terms)); + erl_print_term(stderr, new_terms); + fprintf(stderr, " [%s]\r\n", text); + fail("decoded terms didn't match original"); + } + erl_free_term(original); + erl_free_term(new_terms); +} /* * Converts an Erlang term to the external term format and back again. */ TESTCASE(round_trip_conversion) { - ETERM* original; - ETERM* new_terms; - char encoded[16*1024]; - int n; + int n, i; erl_init(NULL, 0); - original = all_types(); - if (erl_encode(original, encoded) == 0) + encode_decode(all_types(), "ALL"); + { - fail("failed to encode terms"); - } else if ((new_terms = erl_decode(encoded)) == NULL) + int v; + for (v = 8; v; v <<= 1) { + for (i=-4; i<4; i++) { + encode_decode(erl_mk_int(v+i), "INT"); + encode_decode(erl_mk_int(-(v+i)), "NEG INT"); + } + } + } { - fail("failed to decode terms"); - } else if (!erl_match(original, new_terms)) + unsigned int v; + for (v = 8; v; v <<= 1) { + for (i=-4; i<4; i++) { + encode_decode(erl_mk_uint(v+i), "UINT"); + } + } + } { - fail("decoded terms didn't match original"); + long long v; + for (v = 8; v; v <<= 1) { + for (i=-4; i<4; i++) { + encode_decode(erl_mk_longlong(v+i), "LONGLONG"); + encode_decode(erl_mk_longlong(-(v+i)), "NEG LONGLONG"); + } + } + } + { + unsigned long long v; + for (v = 8; v; v <<= 1) { + for (i=-4; i<4; i++) { + encode_decode(erl_mk_ulonglong(v+i), "ULONGLONG"); + } + } } - erl_free_term(original); - erl_free_term(new_terms); report(1); } diff --git a/lib/erl_interface/vsn.mk b/lib/erl_interface/vsn.mk index 589b9e2f9c..672b1be55f 100644 --- a/lib/erl_interface/vsn.mk +++ b/lib/erl_interface/vsn.mk @@ -1 +1 @@ -EI_VSN = 3.6.5 +EI_VSN = 3.7 diff --git a/lib/eunit/doc/overview.edoc b/lib/eunit/doc/overview.edoc index 2583f0be25..be05a13fba 100644 --- a/lib/eunit/doc/overview.edoc +++ b/lib/eunit/doc/overview.edoc @@ -913,7 +913,7 @@ To make the descriptions simpler, we first list some definitions: <td>`CleanupX'</td><td>`(X::any(), R::any()) -> any()'</td> </tr> <tr> -<td>`Instantiator'</td><td>`((R::any()) -> Tests) | {with, [AbstractTestFun::((any()) -> any())]}'</td> +<td>`Instantiator'</td><td>`((R::any()) -> Tests | {with, [AbstractTestFun::((any()) -> any())]}'</td> </tr> <tr> <td>`Where'</td><td>`local | spawn | {spawn, Node::atom()}'</td> diff --git a/lib/gs/doc/src/notes.xml b/lib/gs/doc/src/notes.xml index 8e4032ccc1..352c335e23 100644 --- a/lib/gs/doc/src/notes.xml +++ b/lib/gs/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2009</year> + <year>2004</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>GS Release Notes</title> @@ -47,7 +47,22 @@ </section> </section> - <section><title>GS 1.5.11</title> + <section><title>GS 1.5.12</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Warnings due to new autoimported BIFs removed</p> + <p> + Own Id: OTP-8674 Aux Id: OTP-8579 </p> + </item> + </list> + </section> + +</section> + +<section><title>GS 1.5.11</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/gs/vsn.mk b/lib/gs/vsn.mk index 701d0e178d..35976d556a 100644 --- a/lib/gs/vsn.mk +++ b/lib/gs/vsn.mk @@ -1,2 +1,2 @@ -GS_VSN = 1.5.11 +GS_VSN = 1.5.12 diff --git a/lib/hipe/cerl/erl_bif_types.erl b/lib/hipe/cerl/erl_bif_types.erl index 838f9429f0..1f0247a040 100644 --- a/lib/hipe/cerl/erl_bif_types.erl +++ b/lib/hipe/cerl/erl_bif_types.erl @@ -2337,8 +2337,7 @@ type(lists, flatten, 1, Xs) -> t_list(); false -> X2 = type(lists, flatten, 1, [t_inf(X1, t_list())]), - t_sup(t_list(t_subtract(X1, t_list())), - X2) + t_sup(t_list(t_subtract(X1, t_list())), X2) end end end); @@ -2349,10 +2348,20 @@ type(lists, flatmap, 2, Xs) -> true -> t_nil(); false -> case check_fun_application(F, [t_list_elements(List)]) of - ok -> - case t_is_cons(List) of - true -> t_nonempty_list(t_list_elements(t_fun_range(F))); - false -> t_list(t_list_elements(t_fun_range(F))) + ok -> + R = t_fun_range(F), + case t_is_nil(R) of + true -> t_nil(); + false -> + Elems = t_list_elements(R), + case t_is_cons(List) of + true -> + case t_is_subtype(t_nil(), R) of + true -> t_list(Elems); + false -> t_nonempty_list(Elems) + end; + false -> t_list(Elems) + end end; error -> case t_is_cons(List) of @@ -4026,7 +4035,8 @@ arg_types(erlang, trace_info, 2) -> t_atom('flags'), t_atom('tracer'), %% while the following are items about a func t_atom('traced'), t_atom('match_spec'), t_atom('meta'), - t_atom('meta_match_spec'), t_atom('call_count'), t_atom('all')])]; + t_atom('meta_match_spec'), t_atom('call_count'), + t_atom('call_time'), t_atom('all')])]; arg_types(erlang, trace_pattern, 2) -> [t_sup(t_tuple([t_atom(), t_atom(), t_sup(t_arity(), t_atom('_'))]), t_atom('on_load')), @@ -4035,7 +4045,7 @@ arg_types(erlang, trace_pattern, 3) -> arg_types(erlang, trace_pattern, 2) ++ [t_list(t_sup([t_atom('global'), t_atom('local'), t_atom('meta'), t_tuple([t_atom('meta'), t_pid()]), - t_atom('call_count')]))]; + t_atom('call_count'), t_atom('call_time')]))]; arg_types(erlang, trunc, 1) -> [t_number()]; arg_types(erlang, tuple_size, 1) -> diff --git a/lib/hipe/doc/src/notes.xml b/lib/hipe/doc/src/notes.xml index 2c78558190..7b34a4f427 100644 --- a/lib/hipe/doc/src/notes.xml +++ b/lib/hipe/doc/src/notes.xml @@ -30,6 +30,68 @@ </header> <p>This document describes the changes made to HiPE.</p> +<section><title>Hipe 3.7.6</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p><c>receive</c> statements that can only read out a + newly created reference are now specially optimized so + that it will execute in constant time regardless of the + number of messages in the receive queue for the process. + That optimization will benefit calls to + <c>gen_server:call()</c>. (See <c>gen:do_call/4</c> for + an example of a receive statement that will be + optimized.)</p> + <p> + Own Id: OTP-8623</p> + </item> + <item> + <p> + Various changes to dialyzer-related files for R14.</p> + <p> + - Dialyzer properly supports the new attribute + -export_type and checks that remote types only refer to + exported types. A warning is produced if some + files/applications refer to types defined in modules + which are neither in the PLT nor in the analyzed + applications.</p> + <p> + - Support for detecting data races involving whereis/1 + and unregister/1.</p> + <p> + - More precise identification of the reason(s) why a + record construction violates the types declared for its + fields.</p> + <p> + - Fixed bug in the handling of the 'or' guard.</p> + <p> + - Better handling of the erlang:element/2 BIF.</p> + <p> + - Complete handling of Erlang BIFs.</p> + <p> + Own Id: OTP-8699</p> + </item> + <item> + <p><c>eprof</c> has been reimplemented with support in + the Erlang virtual machine and is now both faster (i.e. + slows down the code being measured less) and scales much + better. In measurements we saw speed-ups compared to the + old eprof ranging from 6 times (for sequential code that + only uses one scheduler/core) up to 84 times (for + parallel code that uses 8 cores).</p> + <p>Note: The API for the <c>eprof</c> has been cleaned up + and extended. See the documentation.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8706</p> + </item> + </list> + </section> + +</section> + <section><title>Hipe 3.7.5</title> <section><title>Improvements and New Features</title> diff --git a/lib/hipe/rtl/hipe_rtl_primops.erl b/lib/hipe/rtl/hipe_rtl_primops.erl index 560e0259f8..0361053676 100644 --- a/lib/hipe/rtl/hipe_rtl_primops.erl +++ b/lib/hipe/rtl/hipe_rtl_primops.erl @@ -701,9 +701,9 @@ gen_cons(Dst, [Arg1, Arg2]) -> %% Increase refcount %% fe->refc++; %% -%% Link to the process off_heap.funs list -%% funp->next = p->off_heap.funs; -%% p->off_heap.funs = funp; +%% Link to the process off_heap list +%% funp->next = p->off_heap.first; +%% p->off_heap.first = funp; %% %% Tag the thing %% return make_fun(funp); @@ -729,9 +729,9 @@ gen_mkfun([Dst], {_Mod, _FunId, _Arity} = MFidA, MagicNr, Index, FreeVars) -> %% fe->refc++; SkeletonCode = gen_fun_thing_skeleton(HP, MFidA, NumFree, MagicNr, Index), - %% Link to the process off_heap.funs list - %% funp->next = p->off_heap.funs; - %% p->off_heap.funs = funp; + %% Link to the process off_heap list + %% funp->next = p->off_heap.first; + %% p->off_heap.first = funp; LinkCode = gen_link_closure(HP), %% Tag the thing and increase the heap_pointer. @@ -797,14 +797,14 @@ gen_link_closure(FUNP) -> end. gen_link_closure_private(FUNP) -> - %% Link to the process off_heap.funs list - %% funp->next = p->off_heap.funs; - %% p->off_heap.funs = funp; + %% Link fun to the process off_heap list + %% funp->next = p->off_heap.first; + %% p->off_heap.first = funp; FunsVar = hipe_rtl:mk_new_reg(), - [load_p_field(FunsVar,?P_OFF_HEAP_FUNS), + [load_p_field(FunsVar,?P_OFF_HEAP_FIRST), hipe_rtl:mk_store(FUNP, hipe_rtl:mk_imm(?EFT_NEXT), FunsVar), - store_p_field(FUNP,?P_OFF_HEAP_FUNS)]. + store_p_field(FUNP,?P_OFF_HEAP_FIRST)]. gen_link_closure_non_private(_FUNP) -> []. diff --git a/lib/hipe/rtl/hipe_tagscheme.erl b/lib/hipe/rtl/hipe_tagscheme.erl index dc44b803a1..c0b6dfad8a 100644 --- a/lib/hipe/rtl/hipe_tagscheme.erl +++ b/lib/hipe/rtl/hipe_tagscheme.erl @@ -849,9 +849,9 @@ create_refc_binary(Base, Size, Flags, Dst) -> heap_arch_spec(HP) -> Tmp1 = hipe_rtl:mk_new_reg(), % MSO state - [hipe_rtl_arch:pcb_load(Tmp1, ?P_OFF_HEAP_MSO), + [hipe_rtl_arch:pcb_load(Tmp1, ?P_OFF_HEAP_FIRST), set_field_from_pointer({proc_bin, next}, HP, Tmp1), - hipe_rtl_arch:pcb_store(?P_OFF_HEAP_MSO, HP)]. + hipe_rtl_arch:pcb_store(?P_OFF_HEAP_FIRST, HP)]. test_heap_binary(Binary, TrueLblName, FalseLblName) -> Tmp1 = hipe_rtl:mk_new_reg_gcsafe(), diff --git a/lib/hipe/vsn.mk b/lib/hipe/vsn.mk index 129718a305..31c860ddec 100644 --- a/lib/hipe/vsn.mk +++ b/lib/hipe/vsn.mk @@ -1 +1 @@ -HIPE_VSN = 3.7.5 +HIPE_VSN = 3.7.6 diff --git a/lib/inviso/src/inviso.erl b/lib/inviso/src/inviso.erl index de42926ffe..0eda06a5c2 100644 --- a/lib/inviso/src/inviso.erl +++ b/lib/inviso/src/inviso.erl @@ -129,7 +129,7 @@ start(Options) -> add_node(Reference) ->
gen_server:call(?CONTROLLER,{add_nodes,[node()],[],Reference,any_ref},?CALL_TIMEOUT).
-add_node(Reference,Options) when list(Options) ->
+add_node(Reference,Options) when is_list(Options) -> gen_server:call(?CONTROLLER,{add_nodes,[node()],Options,Reference,any_ref},?CALL_TIMEOUT).
%% -----------------------------------------------------------------------------
@@ -142,7 +142,7 @@ add_node(Reference,Options) when list(Options) -> add_node_if_ref(Reference) ->
gen_server:call(?CONTROLLER,{add_nodes,[node()],[],Reference,if_ref},?CALL_TIMEOUT).
-add_node_if_ref(Reference,Options) when list(Options) ->
+add_node_if_ref(Reference,Options) when is_list(Options) -> gen_server:call(?CONTROLLER,{add_nodes,[node()],Options,Reference,if_ref},?CALL_TIMEOUT).
%% -----------------------------------------------------------------------------
@@ -156,10 +156,10 @@ add_node_if_ref(Reference,Options) when list(Options) -> %% system. By speicifying node() as the node where the runtime component shall
%% be started. The return value will then follow the rules of non distributed
%% returnvalues and not have a node indicator.
-add_nodes(Nodes,Reference) when list(Nodes) ->
+add_nodes(Nodes,Reference) when is_list(Nodes) -> gen_server:call(?CONTROLLER,{add_nodes,Nodes,[],Reference,any_ref},?CALL_TIMEOUT).
-add_nodes(Nodes,Reference,Options) when list(Nodes),list(Options) ->
+add_nodes(Nodes,Reference,Options) when is_list(Nodes),is_list(Options) -> gen_server:call(?CONTROLLER,{add_nodes,Nodes,Options,Reference,any_ref},?CALL_TIMEOUT).
%% -----------------------------------------------------------------------------
@@ -168,10 +168,10 @@ add_nodes(Nodes,Reference,Options) when list(Nodes),list(Options) -> %%
%% As add_nodes/2,/3 but will only connect to an already existing runtime component
%% if its reference is the same as the one given as argument.
-add_nodes_if_ref(Nodes,Reference) when list(Nodes) ->
+add_nodes_if_ref(Nodes,Reference) when is_list(Nodes) -> gen_server:call(?CONTROLLER,{add_nodes,Nodes,[],Reference,if_ref},?CALL_TIMEOUT).
-add_nodes_if_ref(Nodes,Reference,Options) when list(Nodes),list(Options) ->
+add_nodes_if_ref(Nodes,Reference,Options) when is_list(Nodes),is_list(Options) -> gen_server:call(?CONTROLLER,{add_nodes,Nodes,Options,Reference,if_ref},?CALL_TIMEOUT).
%% -----------------------------------------------------------------------------
@@ -182,9 +182,9 @@ add_nodes_if_ref(Nodes,Reference,Options) when list(Nodes),list(Options) -> %%
%% Change options on all or specified Nodes. This may result in for instance
%% reinitialization of overloadcheck.
-change_options(Options) when list(Options) ->
+change_options(Options) when is_list(Options) -> gen_server:call(?CONTROLLER,{change_options,all,Options},?CALL_TIMEOUT).
-change_options(Nodes,Options) when list(Nodes),list(Options) ->
+change_options(Nodes,Options) when is_list(Nodes),is_list(Options) -> gen_server:call(?CONTROLLER,{change_options,Nodes,Options},?CALL_TIMEOUT).
%% -----------------------------------------------------------------------------
@@ -234,7 +234,7 @@ change_options(Nodes,Options) when list(Nodes),list(Options) -> init_tracing(TracerDataList) ->
gen_server:call(?CONTROLLER,{init_tracing,TracerDataList},?CALL_TIMEOUT).
-init_tracing(Nodes,TracerData) when list(Nodes) ->
+init_tracing(Nodes,TracerData) when is_list(Nodes) -> gen_server:call(?CONTROLLER,{init_tracing,Nodes,TracerData},?CALL_TIMEOUT).
%% -----------------------------------------------------------------------------
@@ -270,10 +270,10 @@ stop_tracing(Nodes) when is_list(Nodes) -> clear() ->
gen_server:call(?CONTROLLER,{clear,all,[]},?CALL_TIMEOUT).
-clear(Nodes) when list(Nodes) ->
+clear(Nodes) when is_list(Nodes) -> gen_server:call(?CONTROLLER,{clear,Nodes,[]},?CALL_TIMEOUT).
-clear(Nodes,Options) when list(Nodes),list(Options) ->
+clear(Nodes,Options) when is_list(Nodes),is_list(Options) -> gen_server:call(?CONTROLLER,{clear,Nodes,Options},?CALL_TIMEOUT).
%% -----------------------------------------------------------------------------
@@ -355,9 +355,9 @@ stop_all() -> tp(Nodes,Module,Function,Arity,MatchSpec,Opts) ->
trace_pattern(Nodes,[{Module,Function,Arity,MatchSpec,Opts}],[global]).
-tp(Nodes,Module,Function,Arity,MatchSpec) when list(Nodes) ->
+tp(Nodes,Module,Function,Arity,MatchSpec) when is_list(Nodes) -> trace_pattern(Nodes,[{Module,Function,Arity,MatchSpec,[]}],[global]);
-tp(Module,Function,Arity,MatchSpec,Opts) when atom(Module) ->
+tp(Module,Function,Arity,MatchSpec,Opts) when is_atom(Module) -> trace_pattern(all,[{Module,Function,Arity,MatchSpec,Opts}],[global]).
tp(Module,Function,Arity,MatchSpec) ->
@@ -384,9 +384,9 @@ tp(PatternList) -> tpl(Nodes,Module,Function,Arity,MatchSpec,Opts) ->
trace_pattern(Nodes,[{Module,Function,Arity,MatchSpec,Opts}],[local]).
-tpl(Nodes,Module,Function,Arity,MatchSpec) when list(Nodes) ->
+tpl(Nodes,Module,Function,Arity,MatchSpec) when is_list(Nodes) -> trace_pattern(Nodes,[{Module,Function,Arity,MatchSpec,[]}],[local]);
-tpl(Module,Function,Arity,MatchSpec,Opts) when atom(Module) ->
+tpl(Module,Function,Arity,MatchSpec,Opts) when is_atom(Module) -> trace_pattern(all,[{Module,Function,Arity,MatchSpec,Opts}],[local]).
tpl(Module,Function,Arity,MatchSpec) ->
@@ -417,12 +417,12 @@ ctp(Nodes,Module,Function,Arity) -> ctp(Module,Function,Arity) ->
trace_pattern(all,[{Module,Function,Arity,false,[only_loaded]}],[global]).
-ctp(Nodes,PatternList) when list(PatternList) ->
+ctp(Nodes,PatternList) when is_list(PatternList) -> trace_pattern(Nodes,
lists:map(fun({M,F,A})->{M,F,A,false,[only_loaded]} end,PatternList),
[global]).
-ctp(PatternList) when list(PatternList) ->
+ctp(PatternList) when is_list(PatternList) -> trace_pattern(all,
lists:map(fun({M,F,A})->{M,F,A,false,[only_loaded]} end,PatternList),
[global]).
@@ -445,12 +445,12 @@ ctpl(Nodes,Module,Function,Arity) -> ctpl(Module,Function,Arity) ->
trace_pattern(all,[{Module,Function,Arity,false,[only_loaded]}],[local]).
-ctpl(Nodes,PatternList) when list(PatternList) ->
+ctpl(Nodes,PatternList) when is_list(PatternList) -> trace_pattern(Nodes,
lists:map(fun({M,F,A})->{M,F,A,false,[only_loaded]} end,PatternList),
[local]).
-ctpl(PatternList) when list(PatternList) ->
+ctpl(PatternList) when is_list(PatternList) -> trace_pattern(all,
lists:map(fun({M,F,A})->{M,F,A,false,[only_loaded]} end,PatternList),
[local]).
@@ -485,19 +485,19 @@ trace_pattern(Nodes,Patterns,FlagList) -> %% specifying a certain pid at all nodes. Or an empty TraceConfList for all
%% nodes.
%% When calling several nodes, the nodes are called in parallel.
-tf(Nodes,PidSpec,FlagList) when list(Nodes),list(FlagList) ->
+tf(Nodes,PidSpec,FlagList) when is_list(Nodes),is_list(FlagList) -> trace_flags(Nodes,[{PidSpec, FlagList}],true).
-tf(Nodes,TraceConfList) when list(Nodes),list(TraceConfList) ->
+tf(Nodes,TraceConfList) when is_list(Nodes),is_list(TraceConfList) -> trace_flags(Nodes,TraceConfList,true);
-tf(PidSpec,FlagList) when list(FlagList) ->
+tf(PidSpec,FlagList) when is_list(FlagList) -> trace_flags(all,[{PidSpec,FlagList}],true).
-tf(ArgList) when list(ArgList) -> % This one has triple functionality!
+tf(ArgList) when is_list(ArgList) -> % This one has triple functionality! case ArgList of
- [{_Process,Flags}|_] when list(Flags),atom(hd(Flags))-> % A call to all nodes.
+ [{_Process,Flags}|_] when is_list(Flags),is_atom(hd(Flags))-> % A call to all nodes. trace_flags(all,ArgList,true);
- [{_Node,TraceConfList}|_] when list(TraceConfList),tuple(hd(TraceConfList)) ->
+ [{_Node,TraceConfList}|_] when is_list(TraceConfList),is_tuple(hd(TraceConfList)) -> trace_flags(ArgList,true);
[{_Node,_TraceConfList,_How}|_] ->
trace_flags(ArgList);
@@ -517,15 +517,15 @@ tf(ArgList) when list(ArgList) -> % This one has triple functionality %% The functions without a Nodes argument means all nodes, in a non-distributed
%% environment it means the local node.
%% When calling several nodes, the nodes are called in parallel.
-ctf(Nodes,PidSpec,FlagList) when list(Nodes),list(FlagList) ->
+ctf(Nodes,PidSpec,FlagList) when is_list(Nodes),is_list(FlagList) -> trace_flags(Nodes,[{PidSpec,FlagList}],false).
-ctf(Nodes,TraceConfList) when list(Nodes),list(TraceConfList) ->
+ctf(Nodes,TraceConfList) when is_list(Nodes),is_list(TraceConfList) -> trace_flags(Nodes,TraceConfList,false);
-ctf(PidSpec,FlagList) when list(FlagList) ->
+ctf(PidSpec,FlagList) when is_list(FlagList) -> trace_flags(all,[{PidSpec,FlagList}],false).
-ctf(TraceConfList) when list(TraceConfList) ->
+ctf(TraceConfList) when is_list(TraceConfList) -> trace_flags(all,TraceConfList,false).
%% -----------------------------------------------------------------------------
@@ -668,10 +668,10 @@ init_tpm(Nodes,Mod,Func,Arity,InitFunc,CallFunc,ReturnFunc,RemoveFunc) -> tpm(Mod,Func,Arity,MS) ->
tpm(all,Mod,Func,Arity,MS).
-tpm(Nodes,Mod,Func,Arity,MS) when integer(Arity) ->
+tpm(Nodes,Mod,Func,Arity,MS) when is_integer(Arity) -> gen_server:call(?CONTROLLER,
{meta_pattern,Nodes,{tpm,[Mod,Func,Arity,MS]}});
-tpm(Mod,Func,Arity,MS,CallFunc) when integer(Arity) ->
+tpm(Mod,Func,Arity,MS,CallFunc) when is_integer(Arity) -> tpm(all,Mod,Func,Arity,MS,CallFunc).
tpm(Nodes,Mod,Func,Arity,MS,CallFunc) ->
@@ -695,11 +695,11 @@ tpm(Nodes,Mod,Func,Arity,MS,InitFunc,CallFunc,ReturnFunc,RemoveFunc) -> tpm_tracer(Mod,Func,Arity,MS) ->
tpm_tracer(all,Mod,Func,Arity,MS).
-tpm_tracer(Nodes,Mod,Func,Arity,MS) when integer(Arity) ->
+tpm_tracer(Nodes,Mod,Func,Arity,MS) when is_integer(Arity) -> gen_server:call(?CONTROLLER,
{meta_pattern,Nodes,{tpm_tracer,[Mod,Func,Arity,MS]}},
?CALL_TIMEOUT);
-tpm_tracer(Mod,Func,Arity,MS,CallFunc) when integer(Arity) ->
+tpm_tracer(Mod,Func,Arity,MS,CallFunc) when is_integer(Arity) -> tpm_tracer(all,Mod,Func,Arity,MS,CallFunc).
tpm_tracer(Nodes,Mod,Func,Arity,MS,CallFunc) ->
@@ -878,7 +878,7 @@ cancel_suspension() -> %% Status=running|{suspended,SReason}
%%
%% Get Status form all or specified runtime components.
-get_status(Nodes) when list(Nodes) ->
+get_status(Nodes) when is_list(Nodes) -> gen_server:call(?CONTROLLER,{get_status,Nodes},?CALL_TIMEOUT).
get_status() ->
@@ -918,7 +918,7 @@ get_tracerdata(Nodes) when is_list(Nodes) -> list_logs() ->
gen_server:call(?CONTROLLER,list_logs,?CALL_TIMEOUT).
-list_logs(TracerDataOrNodesList) when list(TracerDataOrNodesList) ->
+list_logs(TracerDataOrNodesList) when is_list(TracerDataOrNodesList) -> gen_server:call(?CONTROLLER,{list_logs,TracerDataOrNodesList},?CALL_TIMEOUT).
%% ------------------------------------------------------------------------------
@@ -960,17 +960,17 @@ list_logs(TracerDataOrNodesList) when list(TracerDataOrNodesList) -> %% Note that the client process using this function will wait until all files
%% are moved. The job can be cancelled, causing any already copied files to be
%% removed, by simply terminating the waiting client process.
-fetch_log(DestDir,Prefix) when list(DestDir),list(Prefix) ->
+fetch_log(DestDir,Prefix) when is_list(DestDir),is_list(Prefix) -> gen_server:call(?CONTROLLER,{fetch_log,node(),all,DestDir,Prefix},infinity).
-fetch_log(ToNode,DestDir,Prefix) when atom(ToNode),list(DestDir),list(Prefix) ->
+fetch_log(ToNode,DestDir,Prefix) when is_atom(ToNode),is_list(DestDir),is_list(Prefix) -> gen_server:call(?CONTROLLER,{fetch_log,ToNode,all,DestDir,Prefix},infinity);
-fetch_log(LogSpecList,DestDir,Prefix) when list(LogSpecList),list(DestDir),list(Prefix) ->
+fetch_log(LogSpecList,DestDir,Prefix) when is_list(LogSpecList),is_list(DestDir),is_list(Prefix) -> gen_server:call(?CONTROLLER,{fetch_log,node(),LogSpecList,DestDir,Prefix},infinity).
fetch_log(ToNode,LogSpecList,DestDir,Prefix)
- when atom(ToNode),list(LogSpecList),list(DestDir),list(Prefix) ->
+ when is_atom(ToNode),is_list(LogSpecList),is_list(DestDir),is_list(Prefix) -> gen_server:call(?CONTROLLER,{fetch_log,ToNode,LogSpecList,DestDir,Prefix},infinity).
%% ------------------------------------------------------------------------------
diff --git a/lib/inviso/src/inviso_c.erl b/lib/inviso/src/inviso_c.erl index ecbd5b0778..b1597a7f35 100644 --- a/lib/inviso/src/inviso_c.erl +++ b/lib/inviso/src/inviso_c.erl @@ -100,7 +100,7 @@ init({_Parent,Options}) -> end. %% ------------------------------------------------------------------------------ -handle_call({subscribe,Pid},_From,LD) when pid(Pid) -> +handle_call({subscribe,Pid},_From,LD) when is_pid(Pid) -> MRef=erlang:monitor(process,Pid), {reply,ok,LD#state{subscribers=[{Pid,MRef}|LD#state.subscribers]}}; handle_call({subscribe,Faulty},_From,LD) -> @@ -131,7 +131,7 @@ handle_call({change_options,Nodes,Opts},_From,LD) -> end; handle_call({init_tracing,TracerDataList},_From,LD) -> {reply,adapt_reply(LD,do_init_tracing(TracerDataList,LD)),LD}; -handle_call({init_tracing,Nodes,TracerData},_From,LD) when list(Nodes) -> +handle_call({init_tracing,Nodes,TracerData},_From,LD) when is_list(Nodes) -> TracerDataList= lists:map(fun(N)->{N,TracerData} end,started_trace_nodes(Nodes,LD)), {reply,adapt_reply(LD,do_init_tracing(TracerDataList,LD)),LD}; @@ -173,7 +173,7 @@ handle_call({fetch_log,ToNode,Spec,Dest,Prefix},From,LD) -> end; handle_call({delete_log,NodesOrNodeSpecs},_From,LD) -> {reply,adapt_reply(LD,do_delete_log(NodesOrNodeSpecs,LD)),LD}; -handle_call({delete_log,Nodes,Specs},_From,LD) when list(Nodes) -> +handle_call({delete_log,Nodes,Specs},_From,LD) when is_list(Nodes) -> Reply=do_delete_log(lists:map(fun(N)->{N,Specs} end,Nodes),LD), {reply,adapt_reply(LD,Reply),LD}; handle_call({delete_log,FaultyNodes,_Specs},_From,LD) -> @@ -283,7 +283,7 @@ do_change_option(Nodes,Options,LD) -> do_change_option_2([Node|Tail],Options,LD,Replies) -> case get_node_rec(Node,LD) of - Rec when record(Rec,node) -> + Rec when is_record(Rec,node) -> Answer=?RUNTIME:change_options(Rec#node.pid,Options), do_change_option_2(Tail,Options,LD,[{Node,Answer}|Replies]); Error -> @@ -333,7 +333,7 @@ do_init_tracing_2(What,_LD,_) -> %% Returns {ok,Reply} or {error,Reason}. distribute_tp(all,Patterns,FlagList,LD) -> distribute_tp(started_trace_nodes(all,LD),Patterns,FlagList,LD); -distribute_tp(Nodes,Patterns,FlagList,LD) when list(Nodes) -> +distribute_tp(Nodes,Patterns,FlagList,LD) when is_list(Nodes) -> RTpids=lists:map(fun(N)->case get_node_rec(N,LD) of #node{pid=Pid} -> {Pid,N}; @@ -354,7 +354,7 @@ distribute_tp(Faulty,_,_,_) -> %% Returns {ok,Reply} or {error,Reason}. distribute_tf(all,Args,How,LD) -> distribute_tf(started_trace_nodes(all,LD),Args,How,LD); -distribute_tf(Nodes,Args,How,LD) when list(Nodes) -> +distribute_tf(Nodes,Args,How,LD) when is_list(Nodes) -> RTpids=lists:map(fun(Node)-> case get_node_rec(Node,LD) of #node{pid=Pid} -> @@ -369,7 +369,7 @@ distribute_tf(Faulty,_,_,_) -> {error,{badarg,Faulty}}. %% As above but specific args for each node. -distribute_tf(NodeArgs,How,LD) when list(NodeArgs) -> +distribute_tf(NodeArgs,How,LD) when is_list(NodeArgs) -> RTpidArgs=lists:map(fun({Node,Args})-> case get_node_rec(Node,LD) of #node{pid=Pid} -> @@ -384,7 +384,7 @@ distribute_tf(Faulty,_,_) -> {error,{badarg,Faulty}}. %% As above but both specific args for each node and How (set or remove flag). -distribute_tf(NodeArgHows,LD) when list(NodeArgHows) -> +distribute_tf(NodeArgHows,LD) when is_list(NodeArgHows) -> RTpidArgHows= lists:map(fun({Node,Args,How}) -> case get_node_rec(Node,LD) of @@ -405,7 +405,7 @@ distribute_tf(Faulty,_) -> %% Returns {ok,Reply} or {error,Reason}. distribute_metapattern(all,Args,LD) -> distribute_metapattern(started_trace_nodes(all,LD),Args,LD); -distribute_metapattern(Nodes,Args,LD) when list(Nodes) -> +distribute_metapattern(Nodes,Args,LD) when is_list(Nodes) -> RTpids=lists:map(fun(N)->case get_node_rec(N,LD) of #node{pid=Pid} -> {Pid,N}; @@ -480,7 +480,7 @@ do_cancel_suspension_2(Faulty,_,_) -> %% Return {ok,Reply} or {error,Reason}. do_stop_tracing(all,LD) -> do_stop_tracing(started_trace_nodes(all,LD),LD); -do_stop_tracing(Nodes,LD) when list(Nodes) -> +do_stop_tracing(Nodes,LD) when is_list(Nodes) -> RTpids=lists:map(fun(N)->case get_node_rec(N,LD) of #node{pid=Pid} -> {Pid,N}; @@ -580,7 +580,7 @@ do_list_logs_2(Other,_LD,_Replies) -> %% proper strings. do_fetch_log(ToNode,all,Dest,Prefix,From,LD) -> do_fetch_log(ToNode,started_trace_nodes(all,LD),Dest,Prefix,From,LD); -do_fetch_log(ToNode,Specs,Dest,Prefix,From,LD) when list(Dest),list(Prefix) -> +do_fetch_log(ToNode,Specs,Dest,Prefix,From,LD) when is_list(Dest),is_list(Prefix) -> CollectPid=spawn_link(ToNode,?MODULE,log_rec_init,[self(),Dest,Prefix,From]), do_fetch_log_2(Specs,LD,CollectPid,[],[]); do_fetch_log(_ToNode,_Specs,Dest,Prefix,From,_LD) -> @@ -639,7 +639,7 @@ do_delete_log(NodeSpecs,LD) -> LD,[]); false -> if - list(NodeSpecs),list(hd(NodeSpecs)) -> % A list of files. + is_list(NodeSpecs),is_list(hd(NodeSpecs)) -> % A list of files. do_delete_log_2(lists:map(fun(N)->{N,NodeSpecs} end, started_trace_nodes(all,LD)), LD,[]); @@ -785,9 +785,9 @@ check_options(Options, Context) -> check_options_2([],_Context,Result) -> {ok,Result}; -check_options_2([{subscribe,Pid}|OptionsTail],start,Result) when pid(Pid) -> +check_options_2([{subscribe,Pid}|OptionsTail],start,Result) when is_pid(Pid) -> check_options_2(OptionsTail,start,[{subscribe,Pid}|Result]); -check_options_2([{unsubscribe,Pid}|OptionsTail],start,Result) when pid(Pid) -> +check_options_2([{unsubscribe,Pid}|OptionsTail],start,Result) when is_pid(Pid) -> check_options_2(OptionsTail,start,[{unsubscribe,Pid}|Result]); check_options_2([{dependency,How}|OptionsTail],Context,Result) -> check_options_2(OptionsTail,Context,[{dependency,How}|Result]); @@ -819,12 +819,12 @@ initiate_state(Options) -> LD1#state{distributed=true} end. -initiate_state_2([{subscribe,Proc}|Tail],LD) when pid(Proc);atom(Proc)-> +initiate_state_2([{subscribe,Proc}|Tail],LD) when is_pid(Proc);is_atom(Proc)-> MRef=erlang:monitor(process,Proc), initiate_state_2(Tail,LD#state{subscribers=[{Proc,MRef}|LD#state.subscribers]}); -initiate_state_2([Opt|Tail],LD) when tuple(Opt),size(Opt)>=1 -> +initiate_state_2([Opt|Tail],LD) when is_tuple(Opt),size(Opt)>=1 -> initiate_state_2(Tail,initiate_state_3(element(1,Opt),Opt,LD)); -initiate_state_2([Opt|Tail],LD) when atom(Opt) -> +initiate_state_2([Opt|Tail],LD) when is_atom(Opt) -> initiate_state_2(Tail,initiate_state_3(Opt,Opt,LD)); initiate_state_2([_|Tail],LD) -> initiate_state_2(Tail,LD); @@ -853,9 +853,9 @@ initiate_state_is_rt_option(_) -> false. %% or more values associated with the Parameter, or just an atom. merge_options([], Options) -> Options; -merge_options([T|DefaultTail],Options) when tuple(T),size(T)>=1 -> +merge_options([T|DefaultTail],Options) when is_tuple(T),size(T)>=1 -> merge_options(DefaultTail,merge_options_2(element(1,T),T,Options)); -merge_options([Param|DefaultTail],Options) when atom(Param) -> +merge_options([Param|DefaultTail],Options) when is_atom(Param) -> merge_options(DefaultTail,merge_options_2(Param,Param,Options)); merge_options([_|DefaultTail],Options) -> % Strange, bad default option! merge_options(DefaultTail,Options). @@ -868,7 +868,7 @@ merge_options_2(Param,Opt,Options) -> [Opt|Options] end. -merge_options_find(Param,[T|_]) when tuple(T),element(1,T)==Param -> +merge_options_find(Param,[T|_]) when is_tuple(T),element(1,T)==Param -> true; merge_options_find(Param,[Param|_]) -> true; @@ -883,7 +883,7 @@ merge_options_find(_,[]) -> %% It also checks the formatting of the tracerdata since runtime components %% does not accept too badly formatted tracerdata. %% Returns {ok,NewTraceData} or {error,Reason}. -check_modify_tracerdata(TracerData,LoopData) when list(TracerData) -> +check_modify_tracerdata(TracerData,LoopData) when is_list(TracerData) -> case lists:keysearch(trace,1,TracerData) of {value,{_,TraceTD}} -> % Examine the trace part. case check_modify_tracerdata(TraceTD,LoopData) of @@ -908,7 +908,7 @@ check_modify_tracerdata({relayer,Collector},LoopData) when is_atom(Collector) -> end; check_modify_tracerdata({Type,Data},_LoopData) when Type==ip;Type==file -> {ok,{Type,Data}}; -check_modify_tracerdata({Handler,Data},_LoopData) when function(Handler) -> +check_modify_tracerdata({Handler,Data},_LoopData) when is_function(Handler) -> {ok,{Handler,Data}}; check_modify_tracerdata(Data,_LoopData) -> {error,{bad_tracerdata,Data}}. @@ -999,7 +999,7 @@ set_node_rec_2(Rec,[NodeRec|Tail]) -> %% Help function finding a node record for Node in a list of #node or in loopdata. %% Returns the #node in question or {error,not_an_added_node}. -get_node_rec(Node,NodeList) when list(NodeList) -> +get_node_rec(Node,NodeList) when is_list(NodeList) -> get_node_rec_2(Node,NodeList); get_node_rec(Node,#state{nodes=NodeList}) -> get_node_rec_2(Node,NodeList). @@ -1016,7 +1016,7 @@ get_node_rec_2(Node,[_NodeRec|Tail]) -> %% structure. Returns a new list of #node or a new loopdata structure. delete_node_rec(Node,LD=#state{nodes=NodeList}) -> LD#state{nodes=delete_node_rec_2(Node,NodeList)}; -delete_node_rec(Node,NodeList) when list(NodeList) -> +delete_node_rec(Node,NodeList) when is_list(NodeList) -> delete_node_rec_2(Node,NodeList). delete_node_rec_2(_,[]) -> @@ -1059,7 +1059,7 @@ log_rec_init(Parent,Dest,Prefix,From={ClientPid,_}) -> Fetchers), CMRef=erlang:monitor(process,ClientPid), % Monitor the client. case log_rec_loop(Dest,Prefix,RTs,InitialReplies,CMRef) of - Reply when list(Reply) -> % It is an ok value. + Reply when is_list(Reply) -> % It is an ok value. gen_server:reply(From,{ok,Reply}); {error,Reason} -> gen_server:reply(From,{error,Reason}); diff --git a/lib/inviso/src/inviso_lfm.erl b/lib/inviso/src/inviso_lfm.erl index 362176c776..085048518c 100644 --- a/lib/inviso/src/inviso_lfm.erl +++ b/lib/inviso/src/inviso_lfm.erl @@ -87,13 +87,13 @@ %% close the outfile.
%%
%% Using merge/2 assumes you want to use default handlers writing to a file.
-merge(Files,OutputFile) when list(OutputFile) ->
+merge(Files,OutputFile) when is_list(OutputFile) -> merge(Files,fun outfile_opener/1,fun outfile_writer/4,fun outfile_closer/1,OutputFile,off).
-merge(Files,WorkHandlerFun,HandlerData) when function(WorkHandlerFun) ->
+merge(Files,WorkHandlerFun,HandlerData) when is_function(WorkHandlerFun) -> merge(Files,void,WorkHandlerFun,void,HandlerData,off);
-merge(Files,OutputFile,Dbg) when list(OutputFile) ->
+merge(Files,OutputFile,Dbg) when is_list(OutputFile) -> merge(Files,fun outfile_opener/1,fun outfile_writer/4,fun outfile_closer/1,OutputFile,Dbg).
-merge(Files,WorkHandlerFun,HandlerData,Dbg) when function(WorkHandlerFun) ->
+merge(Files,WorkHandlerFun,HandlerData,Dbg) when is_function(WorkHandlerFun) -> merge(Files,void,WorkHandlerFun,void,HandlerData,Dbg).
merge(Files,BeginHandlerFun,WorkHandlerFun,EndHandlerFun,HandlerData) ->
merge(Files,BeginHandlerFun,WorkHandlerFun,EndHandlerFun,HandlerData,off).
@@ -123,7 +123,7 @@ init_receiver(From,Files,BeginHandlerFun,WorkHandlerFun,EndHandlerFun,HandlerDat {ok,Readers} ->
process_flag(trap_exit,true),
if
- function(BeginHandlerFun) ->
+ is_function(BeginHandlerFun) -> case catch BeginHandlerFun(HandlerData) of
{ok,NewHandlerData} ->
init_receiver_2(From,WorkHandlerFun,EndHandlerFun,
@@ -145,7 +145,7 @@ init_receiver_2(From,WorkHandlerFun,EndHandlerFun,HandlerData,Dbg,Readers) -> {Reply,NewHandlerData}=
loop(From,WorkHandlerFun,HandlerData,NewReaders,EntryStruct,Dbg,0),
if
- function(EndHandlerFun) ->
+ is_function(EndHandlerFun) -> case EndHandlerFun(NewHandlerData) of
ok ->
From ! {reply,self(),Reply};
@@ -207,7 +207,7 @@ find_oldest_entry(EntryStruct) -> case list_all_entries(EntryStruct) of
[] -> % The we are done!
done;
- EntryList when list(EntryList) -> % Find smallest timestamp in here then.
+ EntryList when is_list(EntryList) -> % Find smallest timestamp in here then. {Pid,Node,PidMappings,_TS,Term}=
lists:foldl(fun({P,N,PMap,TS1,T},{_P,_N,_PMap,TS0,_T}) when TS1<TS0 ->
{P,N,PMap,TS1,T};
diff --git a/lib/inviso/src/inviso_lfm_tpfreader.erl b/lib/inviso/src/inviso_lfm_tpfreader.erl index d0db4b6d02..6de4d11fe0 100644 --- a/lib/inviso/src/inviso_lfm_tpfreader.erl +++ b/lib/inviso/src/inviso_lfm_tpfreader.erl @@ -60,9 +60,9 @@ %% File=string()
%% Spawn on this function to start a reader process for trace-port generated
%% logfiles, possibly with inviso-generated ti-files.
-init(RecPid,LogFiles=[Tuple|_]) when tuple(Tuple) -> % Only one LogFiles.
+init(RecPid,LogFiles=[Tuple|_]) when is_tuple(Tuple) -> % Only one LogFiles. init(RecPid,[LogFiles]);
-init(RecPid,FileStruct) when list(FileStruct) ->
+init(RecPid,FileStruct) when is_list(FileStruct) -> logfiles_loop(RecPid,FileStruct).
%% -----------------------------------------------------------------------------
@@ -135,7 +135,7 @@ read_traceport_file(FileName,FD) -> case file:read(FD,5) of % Trace-port file entries start with 5 bytes.
{ok,<<0,Size:32>>} -> % Each entry in a traceport file begins.
case file:read(FD,Size) of
- {ok,Bin} when binary(Bin),size(Bin)=:=Size ->
+ {ok,Bin} when is_binary(Bin),size(Bin)=:=Size -> try binary_to_term(Bin) of
Term -> % Bin was a properly formatted term!
{ok,Term}
@@ -244,7 +244,7 @@ find_timestamp_in_term(_) -> % Don't know if there is a timestamp %% (1) plain straight raw binary files.
handle_ti_file(FileStruct) ->
case lists:keysearch(ti_log,1,FileStruct) of
- {value,{_,[FileName]}} when list(FileName) -> % There is one ti-file in this set.
+ {value,{_,[FileName]}} when is_list(FileName) -> % There is one ti-file in this set. case file:open(FileName,[read,raw,binary]) of
{ok,FD} ->
TIdAlias=ets:new(list_to_atom("inviso_ti_atab_"++pid_to_list(self())),
@@ -308,9 +308,9 @@ handle_ti_file_2(FD,TIdAlias,TIdUnalias) -> handle_logfiles(FileStruct) ->
handle_logfiles_2(lists:keysearch(trace_log,1,FileStruct)).
-handle_logfiles_2({value,{_,[FileName]}}) when list(FileName)-> % One single plain file.
+handle_logfiles_2({value,{_,[FileName]}}) when is_list(FileName)-> % One single plain file. [FileName];
-handle_logfiles_2({value,{_,Files}}) when list(Files) -> % A wrap-set.
+handle_logfiles_2({value,{_,Files}}) when is_list(Files) -> % A wrap-set. handle_logfile_sort_wrapset(Files);
handle_logfiles_2(_) ->
[]. % Pretend there were no files otherwise.
diff --git a/lib/inviso/src/inviso_tool.erl b/lib/inviso/src/inviso_tool.erl index 7126ba4387..05158f58fe 100644 --- a/lib/inviso/src/inviso_tool.erl +++ b/lib/inviso/src/inviso_tool.erl @@ -267,9 +267,9 @@ stop(UntouchedNodes) -> %% tracing.
reconnect_nodes() ->
gen_server:call(?MODULE,{reconnect_nodes,local_runtime},?CALL_TIMEOUT).
-reconnect_nodes(Node) when atom(Node) ->
+reconnect_nodes(Node) when is_atom(Node) -> reconnect_nodes([Node]);
-reconnect_nodes(Nodes) when list(Nodes) ->
+reconnect_nodes(Nodes) when is_list(Nodes) -> gen_server:call(?MODULE,{reconnect_nodes,Nodes},?CALL_TIMEOUT).
%% -----------------------------------------------------------------------------
@@ -572,7 +572,7 @@ reactivator_reply(TPid,Counter) -> init(Config) ->
case fetch_configuration(Config) of % From conf-file and Config.
- {ok,LD} when record(LD,ld) ->
+ {ok,LD} when is_record(LD,ld) -> case start_inviso_at_c_node(LD) of
{ok,CPid} ->
LD2=start_runtime_components(LD),
@@ -650,7 +650,7 @@ start_runtime_components_2([],_,LD) -> start_runtime_components_mk_opts(Node,{M,F,Args}) ->
case catch apply(M,F,[Node|Args]) of
- {ok,Opts} when list(Opts) ->
+ {ok,Opts} when is_list(Opts) -> start_runtime_component_mk_opts_add_dependency(Opts);
_ ->
[?DEFAULT_DEPENDENCY]
@@ -698,7 +698,7 @@ handle_call({reconnect_nodes,Nodes},_From,LD) -> {reply,
build_reconnect_nodes_reply(Nodes,Nodes2,NodesErr,NewLD#ld.nodes),
NewLD};
- list(Nodes) ->
+ is_list(Nodes) -> {reply,
{ok,build_reconnect_nodes_reply(Nodes,Nodes2,NodesErr,NewLD#ld.nodes)},
NewLD}
@@ -711,7 +711,7 @@ handle_call({start_session,MoreTDGargs},_From,LD=#ld{session_state=SState}) -> case is_tracing(SState) of
false -> % No session running.
if
- list(MoreTDGargs) ->
+ is_list(MoreTDGargs) -> DateTime=calendar:universal_time(),
{M,F,Args}=LD#ld.tdg,
TDGargs=inviso_tool_lib:mk_tdg_args(DateTime,MoreTDGargs++Args),
@@ -757,15 +757,15 @@ handle_call({reinitiate_session,Nodes},_From,LD=#ld{session_state=SState}) -> end;
handle_call({restore_session,{FileName,MoreTDGargs}},_From,LD=#ld{chl=OldCHL})
- when list(MoreTDGargs) ->
+ when is_list(MoreTDGargs) -> case is_tracing(LD#ld.session_state) of
false ->
case catch make_absolute_path(FileName,LD#ld.dir) of
- AbsFileName when list(AbsFileName) ->
+ AbsFileName when is_list(AbsFileName) -> case file:read_file(AbsFileName) of
{ok,Bin} ->
if
- list(MoreTDGargs) ->
+ is_list(MoreTDGargs) -> case catch replace_history_chl(OldCHL,
binary_to_term(Bin)) of
{ok,CHL} -> % The file was well formatted.
@@ -803,7 +803,7 @@ handle_call({restore_session,MoreTDGargs},_From,LD=#ld{chl=CHL}) -> case history_exists_chl(CHL) of
true -> % There is a history to redo.
if
- list(MoreTDGargs) ->
+ is_list(MoreTDGargs) -> case h_restore_session(MoreTDGargs,LD) of
{ok,{SessionNr,ReturnVal,NewLD}} ->
{reply,
@@ -879,7 +879,7 @@ handle_call({sync_atc,{TC,Id,Vars,TimeOut}},_From,LD=#ld{session_state=SState}) case is_tracing(SState) of
true ->
if
- integer(TimeOut);TimeOut==infinity ->
+ is_integer(TimeOut);TimeOut==infinity -> case h_sync_atc(TC,Id,Vars,TimeOut,LD) of
{ok,NewLD,Result} ->
{reply,Result,NewLD};
@@ -897,7 +897,7 @@ handle_call({sync_rtc,{TC,Vars,TimeOut}},_From,LD=#ld{session_state=SState}) -> case is_tracing(SState) of
true ->
if
- integer(TimeOut);TimeOut==infinity ->
+ is_integer(TimeOut);TimeOut==infinity -> case h_sync_rtc(TC,Vars,TimeOut,LD) of
{ok,NewLD,Result} ->
{reply,Result,NewLD};
@@ -929,7 +929,7 @@ handle_call({sync_dtc,{TC,Id,TimeOut}},_From,LD=#ld{session_state=SState}) -> case is_tracing(SState) of % Check that we are tracing now.
true ->
if
- integer(TimeOut);TimeOut==infinity ->
+ is_integer(TimeOut);TimeOut==infinity -> case h_sync_dtc(TC,Id,TimeOut,LD) of
{ok,NewLD,Result} ->
{reply,Result,NewLD};
@@ -947,7 +947,7 @@ handle_call({inviso,{Cmd,Args}},_From,LD=#ld{session_state=SState}) -> case is_tracing(SState) of
true ->
if
- list(Args) ->
+ is_list(Args) -> case h_inviso(Cmd,Args,LD) of
{ok,{Reply,NewLD}} ->
{reply,Reply,NewLD};
@@ -1156,7 +1156,7 @@ h_reconnect_nodes(local_runtime,LD=#ld{nodes=NodesD}) -> % Non-distributed. _ -> % Allready connected!
{ok,{[],{error,already_connected},LD}}
end;
-h_reconnect_nodes(Nodes,LD=#ld{nodes=NodesD}) when list(Nodes) ->
+h_reconnect_nodes(Nodes,LD=#ld{nodes=NodesD}) when is_list(Nodes) -> {Nodes2,NodesErr}=
lists:foldl(fun(N,{Nodes2,NodesErr})->
case get_state_nodes(N,NodesD) of
@@ -1246,7 +1246,7 @@ h_start_session_ctp_all_2([],Errors,Nodes) -> %% Help function doing the actual init_tracing.
h_start_session_2(undefined,TracerData,_Errors) -> % Non distributed case.
case inviso:init_tracing(TracerData) of
- {ok,LogResult} when list(LogResult) ->
+ {ok,LogResult} when is_list(LogResult) -> {ok,{ok,LogResult}};
{error,already_initated} -> % Perhaps adopted!?
{ok,{error,already_initiated}}; % Not necessarily wrong.
@@ -1360,7 +1360,7 @@ h_reinitiate_session_2(local_runtime,NodesD,undefined) -> % Non distributed case _ ->
{ok,{[],{error,already_in_session}}}
end;
-h_reinitiate_session_2(Nodes,NodesD,CNode) when list(Nodes) ->
+h_reinitiate_session_2(Nodes,NodesD,CNode) when is_list(Nodes) -> {ok,lists:foldl(fun(N,{Nodes2,NodesErr})->
case get_state_nodes(N,NodesD) of
{inactive,running} -> % Only ok case.
@@ -1515,7 +1515,7 @@ h_atc(TC,Id,Vars,LD=#ld{c_node=CNode,tc_dict=TCdict,chl=CHL},Nodes) -> case check_bindings(Vars,TraceCase) of
{ok,Bindings} -> % Necessary vars exists in Vars.
if
- list(Nodes) -> % Nodes predefined.
+ is_list(Nodes) -> % Nodes predefined. h_atc_2(TC,Id,CNode,CHL,LD,TraceCase,Bindings,Nodes);
true -> % Use all tracing and running nodes.
Nodes1=get_nodenames_running_nodes(LD#ld.nodes),
@@ -1769,13 +1769,13 @@ h_reactivate(Node,CNode) -> h_save_history(HDir,Dir,FileName,SortedLog) ->
Dir0=
if
- list(HDir) -> % There is a history dir specified.
+ is_list(HDir) -> % There is a history dir specified. HDir; % Use it then.
true ->
Dir % Else use the tool dir.
end,
case catch make_absolute_path(FileName,Dir0) of
- AbsFileName when list(AbsFileName) ->
+ AbsFileName when is_list(AbsFileName) -> Log2=build_saved_history_data(SortedLog), % Remove stopped tracecases.
case file:write_file(AbsFileName,term_to_binary(Log2)) of
ok ->
@@ -1801,7 +1801,7 @@ h_get_autostart_data(local_runtime,_,Dependency,ASD,M,F,TDGargs,OptsG) -> Opts=[Dependency|lists:keydelete(dependency,1,Opts0)],
{ok,{ASD,{ok,{Opts,{tdg,{M,F,CompleteTDGargs}}}}}};
-h_get_autostart_data(Nodes,CNode,Dependency,ASD,M,F,TDGargs,OptsG) when list(Nodes) ->
+h_get_autostart_data(Nodes,CNode,Dependency,ASD,M,F,TDGargs,OptsG) when is_list(Nodes) -> {ok,{ASD,h_get_autostart_data_2(Nodes,CNode,Dependency,M,F,TDGargs,OptsG)}};
h_get_autostart_data(Nodes,_CNode,_Dependency,_ASD,_M,_F,_TDGargs,_OptsG) ->
{error,{badarg,Nodes}}.
@@ -2144,14 +2144,14 @@ expand_module_regexps(Args,_RegExpNode,_Nodes,false) -> {ok,Args};
expand_module_regexps([PatternList],RegExpNode,Nodes,{tp,1,1}) ->
case catch expand_module_regexps_tp(PatternList,RegExpNode,Nodes) of
- NewPatternList when list(NewPatternList) ->
+ NewPatternList when is_list(NewPatternList) -> {ok,[NewPatternList]};
{error,Reason} ->
{error,Reason}
end;
expand_module_regexps([PatternList],RegExpNode,Nodes,{ctp,1,1}) ->
case catch expand_module_regexps_ctp(PatternList,RegExpNode,Nodes) of
- NewPatternList when list(NewPatternList) ->
+ NewPatternList when is_list(NewPatternList) -> {ok,[NewPatternList]};
{error,Reason} ->
{error,Reason}
@@ -2164,9 +2164,9 @@ expand_module_regexps([M,F,Arity],RegExpNode,Nodes,{ctp,3,1}) -> expand_module_regexps([[{M,F,Arity}]],RegExpNode,Nodes,{ctp,1,1}).
-expand_module_regexps_tp([E={M,_,_,_,_}|Rest],RegExpNode,Nodes) when atom(M) ->
+expand_module_regexps_tp([E={M,_,_,_,_}|Rest],RegExpNode,Nodes) when is_atom(M) -> [E|expand_module_regexps_tp(Rest,RegExpNode,Nodes)];
-expand_module_regexps_tp([{M,F,Arity,MS,Opts}|Rest],RegExpNode,Nodes) when list(M);tuple(M) ->
+expand_module_regexps_tp([{M,F,Arity,MS,Opts}|Rest],RegExpNode,Nodes) when is_list(M);is_tuple(M) -> case inviso_tool_lib:expand_module_names([RegExpNode],
M,
[{expand_only_at,RegExpNode}]) of
@@ -2193,9 +2193,9 @@ expand_module_regexps_tp_2([M|MRest],F,Arity,MS,Opts,Rest,RegExpNode,Nodes) -> expand_module_regexps_tp_2([],_,_,_,_,Rest,RegExpNode,Nodes) ->
expand_module_regexps_tp(Rest,RegExpNode,Nodes).
-expand_module_regexps_ctp([E={M,_,_}|Rest],RegExpNode,Nodes) when atom(M) ->
+expand_module_regexps_ctp([E={M,_,_}|Rest],RegExpNode,Nodes) when is_atom(M) -> [E|expand_module_regexps_ctp(Rest,RegExpNode,Nodes)];
-expand_module_regexps_ctp([{M,F,Arity}|Rest],RegExpNode,Nodes) when list(M);tuple(M) ->
+expand_module_regexps_ctp([{M,F,Arity}|Rest],RegExpNode,Nodes) when is_list(M);is_tuple(M) -> case inviso_tool_lib:expand_module_names([RegExpNode],
M,
[{expand_only_at,RegExpNode}]) of
@@ -2450,7 +2450,7 @@ fetch_configuration(Config) -> %% Returns {ok,FileName} or 'false'. The latter if no name could be determined.
fetch_config_filename(Config) ->
case catch lists:keysearch(config_file,1,Config) of
- {value,{_,FName}} when list(FName) ->
+ {value,{_,FName}} when is_list(FName) -> {ok,FName};
_ -> % No filename in the start argument.
fetch_config_filename_2()
@@ -2458,7 +2458,7 @@ fetch_config_filename(Config) -> fetch_config_filename_2() ->
case application:get_env(inviso_tool_config_file) of
- {ok,FName} when list(FName) ->
+ {ok,FName} when is_list(FName) -> {ok,FName};
_ -> % Application parameter not specified.
false % Means no config file will be used.
@@ -2499,14 +2499,14 @@ read_config_list_2(LD,Terms,Tag) -> %% Function updating a named field in a record. Returns a new record. Note that
%% this function must be maintained due the fact that field names are removed
%% at compile time.
-update_ld_record(LD,nodes,Value) when record(LD,ld) ->
+update_ld_record(LD,nodes,Value) when is_record(LD,ld) -> case mk_nodes(Value) of
{ok,NodesD} ->
LD#ld{nodes=NodesD};
error ->
LD
end;
-update_ld_record(LD,Tag,Value) when record(LD,ld) ->
+update_ld_record(LD,Tag,Value) when is_record(LD,ld) -> Index=
case Tag of
c_node -> % atom()
@@ -2546,7 +2546,7 @@ update_ld_record(LD,Tag,Value) when record(LD,ld) -> %% ActivationFileName=DeactivationFileName=string()
read_trace_case_definitions(LD) ->
case LD#ld.tc_def_file of
- TCfileName when list(TCfileName) ->
+ TCfileName when is_list(TCfileName) -> case catch file:consult(TCfileName) of
{ok,Terms} ->
Dir=LD#ld.dir, % The working directory of the tool.
@@ -2636,8 +2636,8 @@ get_status(CNode,Nodes) -> %% We can end up here if a session is stopped with this node suspended.
%% Returns a nodes database structure filled with the nodes Nodes.
-mk_nodes(Nodes) when list(Nodes) ->
- {ok,lists:map(fun(N) when atom(N)->{N,down} end,Nodes)};
+mk_nodes(Nodes) when is_list(Nodes) -> + {ok,lists:map(fun(N) when is_atom(N)->{N,down} end,Nodes)}; mk_nodes(local_runtime) -> % The non-distributed case.
down;
mk_nodes(_Nodes) ->
@@ -2783,7 +2783,7 @@ set_suspended_nodes(_,{up,{State,_}}) -> %% This function is called when reactivation is completed. Hence it moves the
%% node to no longer suspended. Note this can mean that the node is either
%% tracing or inactive. Reactivation is not allowed for a node have trace_failure.
-set_running_nodes(Node,NodesD) when list(NodesD) ->
+set_running_nodes(Node,NodesD) when is_list(NodesD) -> case lists:keysearch(Node,1,NodesD) of
{value,{_,AvailableStatus}} ->
lists:keyreplace(Node,1,NodesD,{Node,set_running_nodes_2(AvailableStatus)});
@@ -2902,7 +2902,7 @@ get_available_nodes([]) -> %% suspended or not.
%% Returns {State,Status} | reactivating | down
%% where
-get_state_nodes(Node,NodesD) when list(NodesD) ->
+get_state_nodes(Node,NodesD) when is_list(NodesD) -> case lists:keysearch(Node,1,NodesD) of
{value,{_,AvailableStatus}} ->
get_state_nodes_2(AvailableStatus);
diff --git a/lib/inviso/src/inviso_tool_lib.erl b/lib/inviso/src/inviso_tool_lib.erl index 20a8b509ae..7953acedd6 100644 --- a/lib/inviso/src/inviso_tool_lib.erl +++ b/lib/inviso/src/inviso_tool_lib.erl @@ -83,7 +83,7 @@ inviso_cmd(NodeName,Func,Args) -> %% In the non-distributed case the singlenode_expansion will be returned.
expand_module_names(_Nodes,Mod={_,'_'},_) ->
{error,{faulty_regexp_combination,Mod}};
-expand_module_names(Nodes,{DirStr,ModStr},Opts) when list(DirStr), list(ModStr) ->
+expand_module_names(Nodes,{DirStr,ModStr},Opts) when is_list(DirStr), is_list(ModStr) -> case expand_module_names_special_regexp(DirStr) of
{ok,NewDirStr} ->
case expand_module_names_special_regexp(ModStr) of
@@ -97,11 +97,11 @@ expand_module_names(Nodes,{DirStr,ModStr},Opts) when list(DirStr), list(ModStr) end;
expand_module_names(_,'_',_Opts) -> % If we want to trace all modules
wildcard; % we shall not expand it.
-expand_module_names(_Nodes,Mod,_Opts) when atom(Mod) ->
+expand_module_names(_Nodes,Mod,_Opts) when is_atom(Mod) -> module; % If it is an atom, no expansion.
expand_module_names(Nodes,"*",Opts) -> % Treat this as a reg.exp.
expand_module_names(Nodes,".*",Opts);
-expand_module_names(Nodes,ModStr,Opts) when list(ModStr) ->
+expand_module_names(Nodes,ModStr,Opts) when is_list(ModStr) -> case expand_module_names_special_regexp(ModStr) of
{ok,NewModStr} ->
expand_module_names_2(Nodes,NewModStr,Opts);
@@ -115,7 +115,7 @@ expand_module_names_2(Nodes,ModStr,Opts) -> case get_expand_regexp_at_opts(Opts) of
{ok,Node} -> % Expansion only at this node.
case inviso_rt_lib:expand_regexp([Node],ModStr,Opts) of
- [{Node,Modules}] when list(Modules) ->
+ [{Node,Modules}] when is_list(Modules) -> {singlenode_expansion,Modules};
[{Node,_}] -> % Most likely badrpc.
{error,{faulty_node,Node}}
@@ -130,7 +130,7 @@ expand_module_names_2(Nodes,DirStr,ModStr,Opts) -> case get_expand_regexp_at_opts(Opts) of
{ok,Node} -> % Expansion only at this node.
case inviso_rt_lib:expand_regexp([Node],DirStr,ModStr,Opts) of
- [{Node,Modules}] when list(Modules) ->
+ [{Node,Modules}] when is_list(Modules) -> {singlenode_expansion,Modules};
[{Node,_}] -> % Most likely badrpc.
{error,{faulty_node,Node}}
@@ -186,12 +186,12 @@ make_patterns(Catches,Opts,Dbg,NodeModsOrMods,F,A,MS) -> make_patterns_2(Catches,OwnArg,Dbg,NodeModsOrMods,F,A,MS)
end.
-make_patterns_2(Catches,OwnArg,Dbg,[{Node,Mods}|Rest],F,A,MS) when list(Mods) ->
+make_patterns_2(Catches,OwnArg,Dbg,[{Node,Mods}|Rest],F,A,MS) when is_list(Mods) -> TPs=make_patterns_3(Catches,OwnArg,Dbg,Mods,F,A,MS,[]),
[{Node,join_patterns(TPs)}|make_patterns_2(Catches,OwnArg,Dbg,Rest,F,A,MS)];
make_patterns_2(Catches,OwnArg,Dbg,[{_Node,_}|Rest],F,A,MS) -> % badrpc!?
make_patterns_2(Catches,OwnArg,Dbg,Rest,F,A,MS);
-make_patterns_2(Catches,OwnArg,Dbg,Modules,F,A,MS) when list(Modules) ->
+make_patterns_2(Catches,OwnArg,Dbg,Modules,F,A,MS) when is_list(Modules) -> TPs=make_patterns_3(Catches,OwnArg,Dbg,Modules,F,A,MS,[]),
join_patterns(TPs);
make_patterns_2(_,_,_,[],_,_,_) ->
@@ -330,7 +330,7 @@ get_datetime_from_tdg_args([DateTime|_]) -> %% Returns a list.
get_ownarg_opts(Opts) ->
case lists:keysearch(arg,1,Opts) of
- {value,{_,OwnArg}} when list(OwnArg) ->
+ {value,{_,OwnArg}} when is_list(OwnArg) -> OwnArg;
{value,{_,OwnArg}} ->
[OwnArg];
@@ -350,7 +350,7 @@ get_disable_safety_opts(Opts) -> get_expand_regexp_at_opts(Opts) ->
case lists:keysearch(expand_only_at,1,Opts) of
- {value,{_,Node}} when atom(Node) ->
+ {value,{_,Node}} when is_atom(Node) -> {ok,Node};
_ ->
false
diff --git a/lib/jinterface/java_src/Makefile b/lib/jinterface/java_src/Makefile index 37a57352ad..22c55328b8 100644 --- a/lib/jinterface/java_src/Makefile +++ b/lib/jinterface/java_src/Makefile @@ -35,7 +35,21 @@ VSN=$(JINTERFACE_VSN) SPECIAL_TARGETS = +TARGET_FILES= $(POM_TARGET) +SPECIAL_TARGETS = + +POM_FILE= pom.xml + +POM_TARGET= ../$(POM_FILE) +POM_SRC= $(POM_FILE).src + +# ---------------------------------------------------- +# Special Build Targets +# ---------------------------------------------------- + +$(POM_TARGET): $(POM_SRC) ../vsn.mk + sed -e 's;%VSN%;$(VSN);' $< > $@ # ---------------------------------------------------- # Default Subdir Targets @@ -43,7 +57,7 @@ SPECIAL_TARGETS = .PHONY: debug opt instr release docs release_docs tests release_tests clean depend -debug opt instr release docs release_docs tests release_tests clean depend: +debug opt instr release docs release_docs tests release_tests clean depend: $(TARGET_FILES) set -e; set -x; \ case "$(MAKE)" in *clearmake*) tflag="-T";; *) tflag="";; esac; \ if test -f com/ericsson/otp/erlang/ignore_config_record.inf; then xflag=$$tflag; fi; \ diff --git a/lib/jinterface/java_src/pom.xml.src b/lib/jinterface/java_src/pom.xml.src new file mode 100644 index 0000000000..cef49b735a --- /dev/null +++ b/lib/jinterface/java_src/pom.xml.src @@ -0,0 +1,106 @@ +<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> + <modelVersion>4.0.0</modelVersion> + <groupId>org.erlang.otp</groupId> + <artifactId>jinterface</artifactId> + <packaging>jar</packaging> + <version>%VSN%</version> + <name>jinterface</name> + <description> + Jinterface Java package contains java classes, which help you integrate programs written in Java with Erlang. + Erlang is a programming language designed at the Ericsson Computer Science Laboratory. + </description> + <url>http://erlang.org/</url> + <licenses> + <license> + <name>ERLANG PUBLIC LICENSE 1.1</name> + <url>http://www.erlang.org/EPLICENSE</url> + <distribution>repo</distribution> + </license> + </licenses> + <scm> + <connection>git://github.com/erlang/otp.git</connection> + <developerConnection>git://github.com/erlang/otp.git</developerConnection> + <url>http://github.com/erlang/otp</url> + </scm> + <developers> + <developer> + <email>[email protected]</email> + </developer> + </developers> + <organization> + <name>Open Source Erlang</name> + <url>http://www.erlang.org/</url> + </organization> + <build> + <plugins> + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-compiler-plugin</artifactId> + <configuration> + <source>1.5</source> + <target>1.5</target> + </configuration> + </plugin> + <plugin> + <groupId>org.codehaus.mojo</groupId> + <artifactId>build-helper-maven-plugin</artifactId> + <executions> + <execution> + <phase>generate-sources</phase> + <goals> + <goal>add-source</goal> + </goals> + <configuration> + <sources> + <source>java_src</source> + </sources> + </configuration> + </execution> + </executions> + </plugin> + </plugins> + </build> + <distributionManagement> + <repository> + <id>ossrh</id> + <url>http://oss.sonatype.org/service/local/staging/deploy/maven2</url> + </repository> + <snapshotRepository> + <id>ossrh</id> + <url>http://oss.sonatype.org/content/repositories/snapshots</url> + </snapshotRepository> + </distributionManagement> + <profiles> + <profile> + <id>release-sign-artifacts</id> + <activation> + <property> + <name>performRelease</name> + <value>true</value> + </property> + </activation> + <build> + <plugins> + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-gpg-plugin</artifactId> + <version>1.0-alpha-4</version> + <executions> + <execution> + <id>sign-artifacts</id> + <phase>verify</phase> + <goals> + <goal>sign</goal> + </goals> + </execution> + </executions> + </plugin> + </plugins> + </build> + </profile> + </profiles> + <properties> + <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> + </properties> +</project> diff --git a/lib/kernel/doc/src/code.xml b/lib/kernel/doc/src/code.xml index 19e1d3221c..b8db509fa8 100644 --- a/lib/kernel/doc/src/code.xml +++ b/lib/kernel/doc/src/code.xml @@ -54,7 +54,7 @@ for and tries to load the module.</p> </item> </list> - <p>To prevent accidentaly reloading modules affecting the Erlang + <p>To prevent accidentally reloading modules affecting the Erlang runtime system itself, the <c>kernel</c>, <c>stdlib</c> and <c>compiler</c> directories are considered <em>sticky</em>. This means that the system issues a warning and rejects the request if diff --git a/lib/kernel/doc/src/file.xml b/lib/kernel/doc/src/file.xml index a9ceac0bcf..2044b074ee 100644 --- a/lib/kernel/doc/src/file.xml +++ b/lib/kernel/doc/src/file.xml @@ -1229,7 +1229,7 @@ f.txt: {person, "kalle", 25}. </item> <tag><c>{no_translation, unicode, latin1}</c></tag> <item> - <p>The file is was opened with another <c>encoding</c> than <c>latin1</c> and the data on the file can not be translated to the byte-oriented data that this function returns.</p> + <p>The file was opened with another <c>encoding</c> than <c>latin1</c> and the data in the file can not be translated to the byte-oriented data that this function returns.</p> </item> </taglist> </desc> diff --git a/lib/kernel/doc/src/gen_sctp.xml b/lib/kernel/doc/src/gen_sctp.xml index 3a8011e28b..fb09092f1c 100644 --- a/lib/kernel/doc/src/gen_sctp.xml +++ b/lib/kernel/doc/src/gen_sctp.xml @@ -1173,7 +1173,7 @@ client_loop(S, Peer1, Port1, AssocId1, Peer2, Port2, AssocId2) -> <title>SEE ALSO</title> <p><seealso marker="inet">inet(3)</seealso>, <seealso marker="gen_tcp">gen_tcp(3)</seealso>, - <seealso marker="gen_udp">gen_upd(3)</seealso>, + <seealso marker="gen_udp">gen_udp(3)</seealso>, <url href="http://www.rfc-archive.org/getrfc.php?rfc=2960">RFC2960</url> (Stream Control Transmission Protocol), <url href="http://tools.ietf.org/html/draft-ietf-tsvwg-sctpsocket-13">Sockets API Extensions for SCTP.</url></p> <marker id="authors"></marker> diff --git a/lib/kernel/doc/src/notes.xml b/lib/kernel/doc/src/notes.xml index 9c3f6524ae..b503716037 100644 --- a/lib/kernel/doc/src/notes.xml +++ b/lib/kernel/doc/src/notes.xml @@ -30,6 +30,121 @@ </header> <p>This document describes the changes made to the Kernel application.</p> +<section><title>Kernel 2.14</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + os:find_executable can now be fed with the complete name + of the executable on Windows and still find it. I.e + os:find_executable("werl.exe") will work as + os:find_executable("werl").</p> + <p> + Own Id: OTP-3626</p> + </item> + <item> + <p> + The shell's line editing has been improved to more + resemble the behaviour of readline and other shells. + (Thanks to Dave Peticolas)</p> + <p> + Own Id: OTP-8635</p> + </item> + <item> + <p>Under certain circumstances the net kernel could hang. + (Thanks to Scott Lystig Fritchie.)</p> + <p> + Own Id: OTP-8643 Aux Id: seq11584 </p> + </item> + <item> + <p> + The kernel DNS resolver was leaking one or two ports if + the DNS reply could not be parsed or if the resolver(s) + caused noconnection type errors. Bug now fixed. A DNS + specification borderline truncated reply triggering the + port leakage bug has also been fixed.</p> + <p> + Own Id: OTP-8652</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>As of this version, the global name server no longer + supports nodes running Erlang/OTP R11B.</p> + <p> + Own Id: OTP-8527</p> + </item> + <item> + <p> + The file module's functions write,read and read_line now + handles named io_servers like 'standard_io' and + 'standard_error' correctly.</p> + <p> + Own Id: OTP-8611</p> + </item> + <item> + <p> + The functions file:advise/4 and file:datasync/1 have been + added. (Thanks to Filipe David Manana.)</p> + <p> + Own Id: OTP-8637</p> + </item> + <item> + <p>When exchanging groups between nodes <c>pg2</c> did + not remove duplicated members. This bug was introduced in + R13B03 (kernel-2.13.4).</p> + <p> + Own Id: OTP-8653</p> + </item> + <item> + <p> + There is a new option 'exclusive' to file:open/2 that + uses the OS O_EXCL flag where supported to open the file + in exclusive mode.</p> + <p> + Own Id: OTP-8670</p> + </item> + </list> + </section> + +</section> + +<section><title>Kernel 2.13.5.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + A bug introduced in Kernel 2.13.5.2 has been fixed.</p> + <p> + Own Id: OTP-8686 Aux Id: OTP-8643</p> + </item> + </list> + </section> + +</section> + +<section><title>Kernel 2.13.5.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Under certain circumstances the net kernel could hang. + (Thanks to Scott Lystig Fritchie.)</p> + <p> + Own Id: OTP-8643 Aux Id: seq11584</p> + </item> + </list> + </section> + +</section> + <section><title>Kernel 2.13.5.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/kernel/internal_doc/distribution_handshake.txt b/lib/kernel/internal_doc/distribution_handshake.txt index f64ebe0302..6a3ee22ed3 100644 --- a/lib/kernel/internal_doc/distribution_handshake.txt +++ b/lib/kernel/internal_doc/distribution_handshake.txt @@ -11,7 +11,7 @@ The TCP/IP distribution uses a handshake which expects a connection based protocol, i.e. the protocol does not include any authentication after the handshake procedure. -This is not entirelly safe, as it is vulnerable against takeover +This is not entirely safe, as it is vulnerable against takeover attacks, but it is a tradeoff between fair safety and performance. The cookies are never sent in cleartext and the handshake procedure @@ -23,7 +23,7 @@ random numbers. DEFINITIONS ----------- -A challenge is a 32 bit integer number in big endian. Below the function +A challenge is a 32 bit integer number in big endian order. Below the function gen_challenge() returns a random 32 bit integer used as a challenge. A digest is a (16 bytes) MD5 hash of [the Challenge (as text) concatenated @@ -46,19 +46,19 @@ The cookies are text strings that can be viewed as passwords. Every message in the handshake starts with a 16 bit big endian integer which contains the length of the message (not counting the two initial bytes). In erlang this corresponds to the gen_tcp option {packet, 2}. Note that after -the handshake, the distribution switches to 4 byte backet headers. +the handshake, the distribution switches to 4 byte packet headers. THE HANDSHAKE IN DETAIL ----------------------- -Imagine two nodes, node A, which initiates the handshake and node B, whitch +Imagine two nodes, node A, which initiates the handshake and node B, which accepts the connection. 1) connect/accept: A connects to B via TCP/IP and B accepts the connection. 2) send_name/receive_name: A sends an initial identification to B. B receives the message. The message looks -like this (every "square" beeing one byte and the packet header removed): +like this (every "square" being one byte and the packet header removed): +---+--------+--------+-----+-----+-----+-----+-----+-----+-...-+-----+ |'n'|Version0|Version1|Flag0|Flag1|Flag2|Flag3|Name0|Name1| ... |NameN| @@ -67,7 +67,7 @@ like this (every "square" beeing one byte and the packet header removed): The 'n' is just a message tag, Version0 & Version1 is the distribution version selected by node A, based on information from EPMD. (16 bit big endian) -Flag0 ... Flag3 is capability flags, the capabilities defined in dist.hrl. +Flag0 ... Flag3 are capability flags, the capabilities defined in dist.hrl. (32 bit big endian) Name0 ... NameN is the full nodename of A, as a string of bytes (the packet length denotes how long it is). @@ -91,9 +91,9 @@ alive: A connection to the node is already active, which either means This is the format of the status message: -+---+-------+-------+ ... +-------+ ++---+-------+-------+-...-+-------+ |'s'|Status0|Status1| ... |StatusN| -+---+-------+-------+ ... +-------+ ++---+-------+-------+-...-+-------+ 's' is the message tag Status0 ... StatusN is the status as a string (not terminated) @@ -111,35 +111,35 @@ initially sent from A to B, with the addition of a 32 bit challenge: +---+--------+--------+-----+-----+-----+-----+-----+-----+-----+-----+--- |'n'|Version0|Version1|Flag0|Flag1|Flag2|Flag3|Chal0|Chal1|Chal2|Chal3| -+---+--------+--------+-----+-----+-----+-----+-----+-----+---- +-----+--- ++---+--------+--------+-----+-----+-----+-----+-----+-----+-----+-----+--- ------+-----+-...-+-----+ Name0|Name1| ... |NameN| ------+-----+-... +-----+ -Where Chal0 ... Chal3 is the challenge as a 32 bit biog endian integer +Where Chal0 ... Chal3 is the challenge as a 32 bit big endian integer and the other fields are B's version, flags and full nodename. 5) send_challenge_reply/recv_challenge_reply: Now A has generated -a digest and it's own challenge. Those are sent together in a package +a digest and its own challenge. Those are sent together in a package to B: -+---+-----+-----+-----+-----+-----+-----+-----+-----+ -|'r'|Chal0|Chal1|Chal2|Chal3|Dige0|Dige1|Dige2|Dige3| -+---+-----+-----+-----+-----+-----+-----+---- +-----+ ++---+-----+-----+-----+-----+-----+-----+-----+-----+-...-+------+ +|'r'|Chal0|Chal1|Chal2|Chal3|Dige0|Dige1|Dige2|Dige3| ... |Dige15| ++---+-----+-----+-----+-----+-----+-----+-----+-----+-...-+------+ Where 'r' is the tag, Chal0 ... Chal3 is A's challenge for B to handle and -Dige0 ... Dige3 is the digest that A constructed from the challenge B sent +Dige0 ... Dige15 is the digest that A constructed from the challenge B sent in the previous step. 6) recv_challenge_ack/send_challenge_ack: B checks that the digest received from A is correct and generates a digest from the challenge received from A. The digest is then sent to A. The message looks like this: -+---+-----+-----+-----+-----+ -|'a'|Dige0|Dige1|Dige2|Dige3| -+---+-----+-----+---- +-----+ ++---+-----+-----+-----+-----+-...-+------+ +|'a'|Dige0|Dige1|Dige2|Dige3| ... |Dige15| ++---+-----+-----+-----+-----+-...-+------+ -Where 'a' is the tag and Dige0 ... Dige3 is the digest calculated by B +Where 'a' is the tag and Dige0 ... Dige15 is the digest calculated by B for A's challenge. 7) A checks the digest from B and the connection is up. @@ -206,7 +206,7 @@ Currently the following capability flags are defined: %% The node implements distributed process monitoring. -define(DFLAG_DIST_MONITOR,8). -%% The node uses separate tag for fun's (labmdas) in the distribution protocol. +%% The node uses separate tag for fun's (lambdas) in the distribution protocol. -define(DFLAG_FUN_TAGS,16). An R6 erlang node implements all of the above, while a C or Java node only diff --git a/lib/kernel/src/auth.erl b/lib/kernel/src/auth.erl index 7fe30ae828..5c7fe2421d 100644 --- a/lib/kernel/src/auth.erl +++ b/lib/kernel/src/auth.erl @@ -50,6 +50,8 @@ %% Exported functions %%---------------------------------------------------------------------- +-spec start_link() -> {'ok',pid()} | {'error', term()} | 'ignore'. + start_link() -> gen_server:start_link({local, auth}, auth, [], []). @@ -134,7 +136,9 @@ init([]) -> %% The net kernel will let all message to the auth server %% through as is --type calls() :: 'echo' | 'sync_cookie' | {'set_cookie', node(), term()}. +-type calls() :: 'echo' | 'sync_cookie' + | {'get_cookie', node()} + | {'set_cookie', node(), term()}. -spec handle_call(calls(), {pid(), term()}, state()) -> {'reply', 'hello' | 'true' | 'nocookie' | cookie(), state()}. diff --git a/lib/kernel/src/code.erl b/lib/kernel/src/code.erl index 42d4818f08..ec256d5806 100644 --- a/lib/kernel/src/code.erl +++ b/lib/kernel/src/code.erl @@ -304,6 +304,8 @@ do_start(Flags) -> true -> ok end, + % Quietly load the native code for all modules loaded so far. + catch load_native_code_for_all_loaded(), Ok2; Other -> Other @@ -496,3 +498,19 @@ has_ext(Ext, Extlen,File) -> to_path(X) -> filename:join(packages:split(X)). + +-spec load_native_code_for_all_loaded() -> ok. +load_native_code_for_all_loaded() -> + Architecture = erlang:system_info(hipe_architecture), + ChunkName = hipe_unified_loader:chunk_name(Architecture), + lists:foreach(fun({Module, BeamFilename}) -> + case code:is_module_native(Module) of + false -> + case beam_lib:chunks(BeamFilename, [ChunkName]) of + {ok,{_,[{_,Bin}]}} when is_binary(Bin) -> + load_native_partial(Module, Bin); + {error, beam_lib, _} -> ok + end; + true -> ok + end + end, all_loaded()). diff --git a/lib/kernel/test/code_SUITE.erl b/lib/kernel/test/code_SUITE.erl index 08b6477c9d..c9437df258 100644 --- a/lib/kernel/test/code_SUITE.erl +++ b/lib/kernel/test/code_SUITE.erl @@ -31,7 +31,7 @@ where_is_file_cached/1, where_is_file_no_cache/1, purge_stacktrace/1, mult_lib_roots/1, bad_erl_libs/1, code_archive/1, code_archive2/1, on_load/1, - on_load_embedded/1, on_load_errors/1]). + on_load_embedded/1, on_load_errors/1, native_early_modules/1]). -export([init_per_testcase/2, fin_per_testcase/2, init_per_suite/1, end_per_suite/1, @@ -53,7 +53,7 @@ all(suite) -> where_is_file_no_cache, where_is_file_cached, purge_stacktrace, mult_lib_roots, bad_erl_libs, code_archive, code_archive2, on_load, on_load_embedded, - on_load_errors]. + on_load_errors, native_early_modules]. init_per_suite(Config) -> %% The compiler will no longer create a Beam file if @@ -1333,6 +1333,34 @@ do_on_load_error(ReturnValue) -> ?line {undef,[{on_load_error,main,[]}|_]} = Exit end. +native_early_modules(suite) -> []; +native_early_modules(doc) -> ["Test that the native code of early loaded modules is loaded"]; +native_early_modules(Config) when is_list(Config) -> + case erlang:system_info(hipe_architecture) of + undefined -> + {skip,"Native code support is not enabled"}; + Architecture -> + native_early_modules_1(Architecture) + end. + +native_early_modules_1(Architecture) -> + ?line {lists, ListsBinary, _ListsFilename} = code:get_object_code(lists), + ?line ChunkName = hipe_unified_loader:chunk_name(Architecture), + ?line NativeChunk = beam_lib:chunks(ListsBinary, [ChunkName]), + ?line IsHipeCompiled = case NativeChunk of + {ok,{_,[{_,Bin}]}} when is_binary(Bin) -> true; + {error, beam_lib, _} -> false + end, + case IsHipeCompiled of + false -> + {skip,"OTP apparently not configured with --enable-native-libs"}; + true -> + ?line true = lists:all(fun code:is_module_native/1, + [ets,file,filename,gb_sets,gb_trees, + hipe_unified_loader,lists,os,packages]), + ok + end. + %%----------------------------------------------------------------- %% error_logger handler. %% (Copied from stdlib/test/proc_lib_SUITE.erl.) diff --git a/lib/kernel/test/gen_udp_SUITE.erl b/lib/kernel/test/gen_udp_SUITE.erl index bd5685952e..fa1991872b 100644 --- a/lib/kernel/test/gen_udp_SUITE.erl +++ b/lib/kernel/test/gen_udp_SUITE.erl @@ -34,12 +34,12 @@ -export([send_to_closed/1, buffer_size/1, binary_passive_recv/1, bad_address/1, - read_packets/1, open_fd/1]). + read_packets/1, open_fd/1, connect/1]). all(suite) -> [send_to_closed, buffer_size, binary_passive_recv, bad_address, read_packets, - open_fd]. + open_fd, connect]. init_per_testcase(_Case, Config) -> ?line Dog=test_server:timetrap(?default_timeout), @@ -408,3 +408,20 @@ start_node(Name) -> stop_node(Node) -> ?t:stop_node(Node). + + +connect(suite) -> + []; +connect(doc) -> + ["Test that connect/3 has effect"]; +connect(Config) when is_list(Config) -> + Addr = {127,0,0,1}, + {ok,S1} = gen_udp:open(0), + {ok,P1} = inet:port(S1), + {ok,S2} = gen_udp:open(0), + ok = inet:setopts(S2, [{active,false}]), + ok = gen_udp:close(S1), + ok = gen_udp:connect(S2, Addr, P1), + ok = gen_udp:send(S2, <<16#deadbeef:32>>), + {error,econnrefused} = gen_udp:recv(S2, 0, 5), + ok. diff --git a/lib/kernel/vsn.mk b/lib/kernel/vsn.mk index 64a358f114..9a191f9aeb 100644 --- a/lib/kernel/vsn.mk +++ b/lib/kernel/vsn.mk @@ -1,4 +1,4 @@ -# +# # %CopyrightBegin% # # Copyright Ericsson AB 1997-2010. All Rights Reserved. @@ -15,6 +15,6 @@ # under the License. # # %CopyrightEnd% -# +# -KERNEL_VSN = 2.14 +KERNEL_VSN = 2.14.1 diff --git a/lib/megaco/doc/src/megaco.xml b/lib/megaco/doc/src/megaco.xml index ae9e250965..b9bf414299 100644 --- a/lib/megaco/doc/src/megaco.xml +++ b/lib/megaco/doc/src/megaco.xml @@ -662,7 +662,7 @@ megaco_incr_timer() = #megaco_incr_timer{} <taglist> <tag><c><![CDATA[none]]></c></tag> <item> - <p>Do not segment outgoing reply messages. This is usefull when + <p>Do not segment outgoing reply messages. This is useful when either it is known that messages are never to large or that the transport protocol can handle such things on its own (e.g. TCP or SCTP).</p> @@ -1182,7 +1182,7 @@ megaco_incr_timer() = #megaco_incr_timer{} <taglist> <tag><c><![CDATA[none]]></c></tag> <item> - <p>Do not segment outgoing reply messages. This is usefull when + <p>Do not segment outgoing reply messages. This is useful when either it is known that messages are never to large or that the transport protocol can handle such things on its own (e.g. TCP or SCTP).</p> diff --git a/lib/megaco/doc/src/megaco_flex_scanner.xml b/lib/megaco/doc/src/megaco_flex_scanner.xml index eb206e5d13..18c40bb71a 100644 --- a/lib/megaco/doc/src/megaco_flex_scanner.xml +++ b/lib/megaco/doc/src/megaco_flex_scanner.xml @@ -128,7 +128,7 @@ megaco_version() = integer() >= 1 <v>Boolean = boolean()</v> </type> <desc> - <p>Checks if a port is a flex scanner port or not (usefull when + <p>Checks if a port is a flex scanner port or not (useful when if a port exits). </p> <marker id="scan"></marker> diff --git a/lib/megaco/doc/src/megaco_user.xml b/lib/megaco/doc/src/megaco_user.xml index 7332fa684d..7987ed3392 100644 --- a/lib/megaco/doc/src/megaco_user.xml +++ b/lib/megaco/doc/src/megaco_user.xml @@ -504,7 +504,7 @@ protocol_version() = integer() ]]></code> not, is also included in the <c><![CDATA[UserReply]]></c>. </p> <p>The <c><![CDATA[ReplyData]]></c> defaults to <c><![CDATA[megaco:lookup(ConnHandle, reply_data)]]></c>, - but may be explicitely overridden by a + but may be explicitly overridden by a <c><![CDATA[megaco:cast/3]]></c> option in order to forward info about the calling context of the originating process.</p> <p>At <c><![CDATA[success()]]></c>, the <c><![CDATA[UserReply]]></c> either contains:</p> diff --git a/lib/observer/doc/src/notes.xml b/lib/observer/doc/src/notes.xml index 48c7c21363..3d7c4fa269 100644 --- a/lib/observer/doc/src/notes.xml +++ b/lib/observer/doc/src/notes.xml @@ -31,6 +31,21 @@ <p>This document describes the changes made to the Observer application.</p> +<section><title>Observer 0.9.8.3</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The test suite has been updated for R14A.</p> + <p> + Own Id: OTP-8708</p> + </item> + </list> + </section> + +</section> + <section><title>Observer 0.9.8.2</title> <section><title>Improvements and New Features</title> diff --git a/lib/observer/vsn.mk b/lib/observer/vsn.mk index ff06fb992d..499cce6b97 100644 --- a/lib/observer/vsn.mk +++ b/lib/observer/vsn.mk @@ -1 +1 @@ -OBSERVER_VSN = 0.9.8.2 +OBSERVER_VSN = 0.9.8.3 diff --git a/lib/odbc/doc/src/databases.xml b/lib/odbc/doc/src/databases.xml index 9776736909..a6ba0e5245 100644 --- a/lib/odbc/doc/src/databases.xml +++ b/lib/odbc/doc/src/databases.xml @@ -258,7 +258,7 @@ when p >= 16 </cell> that contains more than one SQL query. For example, the following SQLServer-specific statement creates a procedure that returns a result set containing information about employees - that work at the department and and a result set listing the + that work at the department and a result set listing the customers of that department. </p> <code type="none"> CREATE PROCEDURE DepartmentInfo (@DepartmentID INT) AS diff --git a/lib/parsetools/doc/src/notes.xml b/lib/parsetools/doc/src/notes.xml index 8a6f2c2714..63c37bc27d 100644 --- a/lib/parsetools/doc/src/notes.xml +++ b/lib/parsetools/doc/src/notes.xml @@ -30,6 +30,28 @@ </header> <p>This document describes the changes made to the Parsetools application.</p> +<section><title>Parsetools 2.0.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>Yecc failed to report reduce/reduce conflicts where + one of the reductions involved the root symbol. This bug + has been fixed. (Thanks to Manolis Papadakis.)</p> + <p> + Own Id: OTP-8483</p> + </item> + <item> + <p>A bug introduced in Parsetools 1.4.4 (R12B-2) has been + fixed. (Thanks to Manolis Papadakis.)</p> + <p> + Own Id: OTP-8486</p> + </item> + </list> + </section> + +</section> + <section><title>Parsetools 2.0.2</title> <section><title>Improvements and New Features</title> diff --git a/lib/parsetools/vsn.mk b/lib/parsetools/vsn.mk index b1354e89d8..f3e2dc0fb4 100644 --- a/lib/parsetools/vsn.mk +++ b/lib/parsetools/vsn.mk @@ -1 +1 @@ -PARSETOOLS_VSN = 2.0.2 +PARSETOOLS_VSN = 2.0.3 diff --git a/lib/public_key/src/public_key.app.src b/lib/public_key/src/public_key.app.src index edede7c874..d5e1705827 100644 --- a/lib/public_key/src/public_key.app.src +++ b/lib/public_key/src/public_key.app.src @@ -13,4 +13,5 @@ {registered, []}, {env, []} ] -}.
\ No newline at end of file +}. + diff --git a/lib/reltool/bin/reltool.escript b/lib/reltool/bin/reltool.escript index 0dcd5ad1e9..0dcd5ad1e9 100644..100755 --- a/lib/reltool/bin/reltool.escript +++ b/lib/reltool/bin/reltool.escript diff --git a/lib/reltool/doc/src/notes.xml b/lib/reltool/doc/src/notes.xml index 524d728901..95e379db53 100644 --- a/lib/reltool/doc/src/notes.xml +++ b/lib/reltool/doc/src/notes.xml @@ -37,6 +37,73 @@ thus constitutes one section in this document. The title of each section is the version number of Reltool.</p> + <section><title>Reltool 0.5.4</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Added function <c>zip:foldl/3</c> to iterate over zip + archives.</p> + <p> + Added functions to create and extract escripts. See + <c>escript:create/2</c> and <c>escript:extract/2</c>.</p> + <p> + The undocumented function <c>escript:foldl/3</c> has been + removed. The same functionality can be achieved with the + more flexible functions <c>escript:extract/2</c> and + <c>zip:foldl/3</c>.</p> + <p> + Record fields has been annotated with type info. Source + files as been adapted to fit within 80 chars and trailing + whitespace has been removed.</p> + <p> + Own Id: OTP-8521</p> + </item> + <item> + <p>A new escript, called <c>reltool</c>, has been + introduced in order to simplify the usage of the reltool + application from makefiles.</p> + <p>The handling of applications included in releases has + been improved. Applications that are required to be + started before other applications in a release are now + automatically included in the release. The <c>kernel</c> + and <c>stdlib</c> applications are always included as + they are mandatory.</p> + <p>Applications that are (explicitly or implicitly) + included in a release are now automatically included as + if they were explicitly included with the incl_cond + flag.</p> + <p>A new <c>embedded_app_type</c> option has been + introduced. It is intended to be used for embedded + systems where all included applications must be loaded + from the boot script, as these systems does not utilize + dynamic code loading. If <c>embedded_app_type </c> is set + to something else than <c>undefined</c>, all included + applications will be included in both the release as well + as in the boot script. If the <c>profile</c> is + <c>embedded</c> the <c>embedded_app_type</c> option + defaults to <c>load</c>.</p> + <p>A new function called <c>reltool:get_status/1</c> has + been introduced. It returns status about the + configuration in the server.</p> + <p>The API functions that may take <c>PidOrOptions</c> as + input and actually gets <c>Options</c> does now print out + warnings.</p> + <p>The internal error handling has been improved. For + example <c>{error,Reason}</c> is always returned in case + of errors even when the server dies.</p> + <p><c>app</c> and <c>appup</c> files has been added as + well as a corresponding test suite.</p> + <p>Various cleanups has been made in the code and in the + documentation.</p> + <p> + Own Id: OTP-8590</p> + </item> + </list> + </section> + + </section> <section><title>Reltool 0.5.3</title> diff --git a/lib/reltool/doc/src/reltool.xml b/lib/reltool/doc/src/reltool.xml index 0c2b7d2a2b..598594145a 100644 --- a/lib/reltool/doc/src/reltool.xml +++ b/lib/reltool/doc/src/reltool.xml @@ -48,8 +48,8 @@ <c>root_dir</c> is the root directory of the analysed system and it defaults to the system executing <c>reltool</c>. Applications may also be located outside <c>root_dir</c>. <c>lib_dirs</c> - defines additional library directories where applications - additional may reside and it defaults to the the directories + defines library directories where additional applications + may reside and it defaults to the directories listed by the operating system environment variable <c>ERL_LIBS</c>. See the module <c>code</c> for more info. Finally single modules and entire applications may be read from @@ -58,10 +58,10 @@ <p>Some configuration parameters control the behavior of Reltool on system (<c>sys</c>) level. Others provide control on application (<c>app</c>) level and yet others are on module - (<c>mod</c>) level. Module level parameters overrides application - level parameters and application level parameters overrides system + (<c>mod</c>) level. Module level parameters override application + level parameters and application level parameters override system level parameters. Escript <c>escript</c> level parameters - overrides system level parameters.</p> + override system level parameters.</p> <p>The following top level <c>options</c> are supported:</p> @@ -74,7 +74,7 @@ a name of a <c>file</c> containing a sys tuple.</p> </item> - <tag><c>trap_exit></c></tag> + <tag><c>trap_exit</c></tag> <item> <p>This option controls the error handling behavior of <c>reltool</c>. By default the window processes traps @@ -123,10 +123,10 @@ defaults to <c>all</c> which means that if an application is included (either explicitly or implicitly) all modules in that application will be included. This implies that both modules - that exists on the <c>ebin</c> directory of the application, + that exist in the <c>ebin</c> directory of the application, as well as modules that are named in the <c>app</c> file will be included. If the parameter is set to <c>ebin</c>, both - modules on the <c>ebin</c> directory and derived modules are + modules in the <c>ebin</c> directory and derived modules are included. If the parameter is set to <c>app</c>, both modules in the <c>app</c> file and derived modules are included. <c>derived</c> means that only modules that are used by other @@ -138,13 +138,13 @@ <item> <p>This parameter controls the application and escript inclusion policy. It defaults to <c>derived</c> which means - that the applications that not have any explicit + that the applications that do not have any explicit <c>incl_cond</c> setting, will only be included if any other (explicitly or implicitly included) application uses it. The value <c>include</c> implies that all applications and - escripts that that not have any explicit <c>incl_cond</c> + escripts that do not have any explicit <c>incl_cond</c> setting will be included. <c>exclude</c> implies that all - applications and escripts) that that not have any explicit + applications and escripts) that do not have any explicit <c>incl_cond</c> setting will be excluded.</p> </item> @@ -158,7 +158,7 @@ <tag><c>rel</c></tag> <item> <p>Release specific configuration. Each release maps to a - <c>rel</c>, <c>script</c> and <c>boot </c> file. See the + <c>rel</c>, <c>script</c> and <c>boot</c> file. See the module <c>systools</c> for more info about the details. Each release has a name, a version and a set of applications with a few release specific parameters such as type and included @@ -168,7 +168,7 @@ <tag><c>relocatable</c></tag> <item> <p>This parameter controls whether the <c>erl</c> executable - in the target system automatically should determine where it + in the target system should automatically determine where it is installed or if it should use a hardcoded path to the installation. In the latter case the target system must be installed with <c>reltool:install/2</c> before it can be @@ -182,7 +182,7 @@ <p>The creation of the specification for a target system is performed in two steps. In the first step a complete specification is generated. It will likely contain much more - files than you are interested in your customized target + files than you are interested in in your customized target system. In the second step the specification will be filtered according to your filters. There you have the ability to specify filters per application as well as system wide @@ -224,12 +224,12 @@ <tag><c>incl_sys_filters</c></tag> <item> <p>This parameter normally contains a list of regular - expressions that controls which files in the system that + expressions that controls which files in the system should be included. Each file in the target system must match at least one of the listed regular expressions in order to be included. Further the files may not match any filter in <c>excl_sys_filters</c> in order to be included. Which - application files that should be included are controlled with + application files should be included is controlled with the parameters <c>incl_app_filters</c> and <c>excl_app_filters</c>. This parameter defaults to <c>[".*"]</c>.</p> @@ -238,8 +238,8 @@ <tag><c>excl_sys_filters</c></tag> <item> <p>This parameter normally contains a list of regular - expressions that controls which files in the system that not - should be included in the target system. In order to be + expressions that controls which files in the system should + not be included in the target system. In order to be included, a file must match some filter in <c>incl_sys_filters</c> but not any filter in <c>excl_sys_filters</c>. This parameter defaults to @@ -260,7 +260,7 @@ <item> <p>This parameter normally contains a list of regular expressions that controls which application specific files - that not should be included in the target system. In order to + should not be included in the target system. In order to be included, a file must match some filter in <c>incl_app_filters</c> but not any filter in <c>excl_app_filters</c>. This parameter defaults to @@ -271,8 +271,8 @@ <item> <p>This parameter normally contains a list of regular expressions that controls which top level directories in an - application that should be included in an archive file (as - opposed of beeing included as a regular directory outside the + application should be included in an archive file (as + opposed to being included as a regular directory outside the archive). Each top directory in the application must match at least one of the listed regular expressions in order to be included. Further the files may not match any filter in @@ -284,7 +284,7 @@ <item> <p>This parameter normally contains a list of regular expressions that controls which top level directories in an - application that not should be included in an archive file. In + application should not be included in an archive file. In order to be included in the application archive, a top directory must match some filter in <c>incl_archive_filters</c> but not any filter in <c>excl_archive_filters</c>. This @@ -295,15 +295,15 @@ <item> <p>This parameter contains a list of options that are given to <c>zip:create/3</c> when application specific files are - packaged into an archive. All options are not supported. The - most useful options in this context, are the ones that - controls which types of files that should be compressed. This + packaged into an archive. Only a subset of the options are + supported. The most useful options in this context are the ones + that control which types of files should be compressed. This parameter defaults to <c>[]</c>.</p> </item> </taglist> - <p>On application (<c>escript</c>) level,the following options are + <p>On application (<c>escript</c>) level, the following options are supported:</p> <taglist> @@ -314,7 +314,7 @@ </item> </taglist> - <p>On application (<c>app</c>) level,the following options are + <p>On application (<c>app</c>) level, the following options are supported:</p> <taglist> @@ -322,8 +322,8 @@ <item> <p>The version of the application. In an installed system there may exist several versions of an application. The <c>vsn</c> parameter - controls which version of the application that will be choosen. If it - is omitted, the latest version will be choosen.</p> + controls which version of the application will be chosen. If it + is omitted, the latest version will be chosen.</p> </item> <tag><c>mod</c></tag> <item> @@ -380,7 +380,7 @@ </item> </taglist> - <p>On module (<c>mod</c>) level,the following options are + <p>On module (<c>mod</c>) level, the following options are supported:</p> <taglist> @@ -391,9 +391,9 @@ will be used to control whether the module is included or not. The value of <c>incl_cond</c> overrides the module inclusion policy. <c>include</c> implies that the module is included, while - <c>exclude</c> implies that the module not is included. - <c>derived</c> implies that the is included if any included uses the - module.</p> + <c>exclude</c> implies that the module is not included. + <c>derived</c> implies that the module is included if it is used + by any other included module.</p> </item> <tag><c>debug_info</c></tag> <item> @@ -546,12 +546,12 @@ target_spec() = [target_spec()] files are by default copied to the target system. The <c>releases</c> directory contains generated <c>rel</c>, <c>script</c>, and <c>boot</c> files. The <c>lib</c> directory - contains the applications. Which applications that are included + contains the applications. Which applications are included and if they should be customized (archived, stripped from debug info etc.) is specified with various configuration parameters. The files in the <c>bin</c> directory are copied from the <c>erts-vsn/bin</c> directory, but only those files - that was originally included in <c>bin</c> directory of the + that were originally included in the <c>bin</c> directory of the source system.</p> <p>If the configuration parameter <c>relocatable</c> was set to @@ -584,10 +584,10 @@ target_spec() = [target_spec()] <v>Reason = reason()</v> </type> <desc><p>Get reltool configuration. Normally, only the explicit - configuration parameters with values that differs from their + configuration parameters with values that differ from their defaults are interesting. But the builtin default values can be returned by setting <c>InclDefaults</c> to <c>true</c>. The - derived configuration can be return by setting + derived configuration can be returned by setting <c>InclDerived</c> to <c>true</c>.</p></desc> </func> @@ -705,7 +705,7 @@ target_spec() = [target_spec()] <v>Reason = reason()</v> </type> <desc><p>Start a server process with options. The server process - identity can be given as argument to several other functions in the + identity can be given as an argument to several other functions in the API.</p></desc> </func> diff --git a/lib/reltool/doc/src/reltool_usage.xml b/lib/reltool/doc/src/reltool_usage.xml index 885828d1f0..0a053a014e 100644 --- a/lib/reltool/doc/src/reltool_usage.xml +++ b/lib/reltool/doc/src/reltool_usage.xml @@ -44,7 +44,7 @@ <section> <title>System window</title> <p>The system window is started with the function - <c>reltool:start/1</c>. At startup the tool will process the all + <c>reltool:start/1</c>. At startup the tool will process all <c>beam</c> files and <c>app</c> files in order to find out dependencies between applications and their modules. Once all this information has been derived, it will be possible to explore the tool.</p> @@ -67,29 +67,29 @@ <section> <title>Libraries</title> <p>On the library page it is possible to control which sources - that the tool will use. The page is organized as a tree which + the tool will use. The page is organized as a tree which can be expanded and collapsed by clicking on the little symbol in the beginning of the expandable/collapsible lines.</p> <p>The <c>Root directory</c> can be edited by selecting the line where the path of the root directory is displayed and - clicking with the right mouse button. Choose edit in the menu - that pops up. </p> + clicking the right mouse button. Choose edit in the menu + that pops up.</p> <p>Library directories can be added, edited or deleted. This is done by selecting the line where the path to a library - directory is displayed and clicking with the right mouse + directory is displayed and clicking the right mouse button. Choose add, edit or delete in the menu that pops up. New library directories can also be added by selecting the - line <c>Library directories</c> and clicking with the right + line <c>Library directories</c> and clicking the right mouse button. Choose add in the menu that pops up.</p> <p>Escript files can be added, edited or deleted. This is done by selecting the line where the path to an escript file is - displayed and clicking with the right mouse button. Choose + displayed and clicking the right mouse button. Choose add, edit or delete in the menu that pops up. New escripts can also be added by selecting the line <c>Escript files</c> and - clicking with the right mouse button. Choose add in the menu + clicking the right mouse button. Choose add in the menu that pops up.</p> <p>When libraries and escripts are expanded, the names of @@ -102,7 +102,7 @@ <p>On the system settings page it is possible to control some global settings that are used as defaults for all applications. Set the <c>Application inclusion policy</c> to - <c>include</c> to include all applications that not are + <c>include</c> to include all applications that are not explicitly excluded. See <c>incl_cond</c> (application inclusion) and <c>mod_cond</c> (module inclusion) in the reference manual for the module <c>reltool</c> for more @@ -131,14 +131,14 @@ <p>The symbols in front of the application names are intended to describe the status of the application. There are error - symbols and warning symbols that means that there are - something that needs attention. The tick symbol means that the + and warning symbols to signalize that there is + something which needs attention. The tick symbol means that the application is included or derived and no problem has been detected. The cross symbol means that the application is excluded or available and no problem has been detected. Applications with error symbols are listed first in - each category, then comes the warnings and the normal ones - (ticks and crosses) are found at the end.</p> + each category and are followed by the warnings and the + normal ones (ticks and crosses) at the end.</p> <p>Double click on an application to launch its application window.</p> @@ -175,8 +175,8 @@ </item> <item> <p><c>Save configuration</c> - Saves the current - configuration to file. Normally, only the explictit - configuration parameters with values that differs from their + configuration to file. Normally, only the explicit + configuration parameters with values that differ from their defaults are saved. But the configuration with or without default values and with or without derived values may also be saved.</p> @@ -204,21 +204,21 @@ <p>It is possible to perform some limited manipulations of the graph. Nodes can be moved, selected, locked or deleted. Move a single node or the entire graph by moving the mouse while the - left mouse button is pressed. A node is can be locked into a fix + left mouse button is pressed. A node can be locked into a fix position by holding down the shift button when the left mouse button is released. Select several nodes by moving the mouse - while the control key and the left mouse button i + while the control key and the left mouse button are pressed. Selected nodes can be locked, unlocked or deleted by - klicking on a suitable button.</p> + clicking on a suitable button.</p> <p>The algorithm that is used to draw a graph with as few crossed links as possible is called force graph. A force graph - consists of nodes and directed link between nodes. Each node is + consists of nodes and directed links between nodes. Each node is associated with a repulsive force that pushes nodes away from each other. This force can be adjusted with the left slider or with the mouse wheel. Each link is associated with an attractive - force that pulls the nodes nearer each other. This force can be - adjusted with the right slider. If this force becomes to strong, + force that pulls the nodes nearer to each other. This force can be + adjusted with the right slider. If this force becomes too strong, the graph will be unstable. The third parameter that can be adjusted is the length of the links. It is adjusted with the middle slider.</p> @@ -256,7 +256,7 @@ <p>Select version of the application in the <c>Source selection policy</c> part of the page. By default the latest version of the application is selected, but it is possible to override this by - explicitly select another version.</p> + explicitly selecting another version.</p> <p>By default the <c>Application inclusion policy</c> on system level is used for all applications. Set the value to @@ -272,10 +272,10 @@ want actually used modules to be included. Set it to <c>app</c> if you, besides derived modules, also want the modules listed in the app file to be included. Set it to <c>ebin</c> if you, besides - derived modules, also want the modules that exists as beam files - on the ebin directory to be included. Set it to <c>all</c> if you + derived modules, also want the modules that exist as beam files + in the ebin directory to be included. Set it to <c>all</c> if you want all modules to be included, that is the union of modules - found on the ebin directory and listed in the app file.</p> + found in the ebin directory and listed in the app file.</p> <p>The application settings page is rather incomplete.</p> </section> @@ -299,16 +299,16 @@ undone.</p> <p>The symbols in front of the module names are intended to - describe the status of the module. There are error symbols - and warning symbols that means that there are something that needs + describe the status of the module. There are error and + and warning symbols to signalize that there is something that needs attention. The tick symbol means that the module is included or derived and no problem has been detected. The cross symbol means that the module is excluded or available and no problem has been detected. Modules with error symbols are listed - first in each category, then comes the warnings and the normal - ones (ticks and crosses) are found at the end.</p> + first in each category and are followed by warnings and the + normal ones (ticks and crosses) at the end.</p> - <p>Double click on an module to launch its module window.</p> + <p>Double click on a module to launch its module window.</p> </section> @@ -323,7 +323,7 @@ <c>app</c> file. If the application includes other applications, these are listed under <c>Included</c>. These applications are listed in the <c>included_applications</c> part of the <c>app</c> - file. If the application uses modules other applications, these + file. If the application uses other applications, these are listed under <c>Uses</c>.</p> <p>Double click on an application name to launch an application window.</p> @@ -336,7 +336,7 @@ <p>There are two categories of modules on the <c>Module dependencies</c> page. If the module is used by other modules, these are listed under <c>Modules used by others</c>. If the - module uses modules other modules, these are listed under <c>Used + module uses other modules, these are listed under <c>Used modules</c>.</p> <p>Double click on an module name to launch a module window.</p> @@ -365,8 +365,8 @@ <p>There are two categories of modules on the <c>Dependencies</c> page. If the module is used by other modules, these are listed - under <c>Modules used by others</c>. If the module uses modules - other modules, these are listed under <c>Used modules</c>.</p> + under <c>Modules used by others</c>. If the module uses other + modules, these are listed under <c>Used modules</c>.</p> <p>Double click on an module name to launch a module window.</p> @@ -378,7 +378,7 @@ <p>On the <c>Code</c> page the Erlang source code is displayed. It is possible to search forwards and backwards for text in the module. Enter a regular expression in the <c>Find</c> field and - press enter. It is also possible to goto a certain line on the + press enter. It is also possible to go to a certain line in the module. The <c>Back</c> button can be used to go back to the previous position.</p> diff --git a/lib/reltool/src/reltool_mod_win.erl b/lib/reltool/src/reltool_mod_win.erl index f68c61fd6f..281d2c8ad4 100644 --- a/lib/reltool/src/reltool_mod_win.erl +++ b/lib/reltool/src/reltool_mod_win.erl @@ -334,9 +334,9 @@ find_regular_bin(App, Mod) -> ActiveDir = App#app.active_dir, SrcDir = filename:join([ActiveDir, "src"]), ModStr = atom_to_list(Mod#mod.name), - Base = ModStr ++ ".erl", - Find = fun(F, _Acc) -> file:read_file(F) end, - case filelib:fold_files(SrcDir, Base, true, Find, {error, enoent}) of + Base = "^" ++ ModStr ++ "\\.erl$", + Find = fun(F, _Acc) -> throw(file:read_file(F)) end, + case catch filelib:fold_files(SrcDir, Base, true, Find, {error, enoent}) of {ok, Bin} -> Bin; {error, enoent} -> diff --git a/lib/reltool/vsn.mk b/lib/reltool/vsn.mk index b0561a6110..f23a7e84a2 100644 --- a/lib/reltool/vsn.mk +++ b/lib/reltool/vsn.mk @@ -16,9 +16,10 @@ # # %CopyrightEnd% -RELTOOL_VSN = 0.5.3 +RELTOOL_VSN = 0.5.4 -TICKETS = OTP-8057 +TICKETS = OTP-8521 OTP-8590 +TICKETS_0_5_3 = OTP-8057 TICKETS_0_5_2 = OTP-8254 TICKETS_0_5_1 = OTP-8199 TICKETS_0_5 = OTP-7949 diff --git a/lib/runtime_tools/doc/src/notes.xml b/lib/runtime_tools/doc/src/notes.xml index 9eb13727c7..e1f954dda7 100644 --- a/lib/runtime_tools/doc/src/notes.xml +++ b/lib/runtime_tools/doc/src/notes.xml @@ -31,6 +31,21 @@ <p>This document describes the changes made to the Runtime_Tools application.</p> +<section><title>Runtime_Tools 1.8.4</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Miscellaneous updates.</p> + <p> + Own Id: OTP-8705</p> + </item> + </list> + </section> + +</section> + <section><title>Runtime_Tools 1.8.3</title> <section><title>Improvements and New Features</title> diff --git a/lib/runtime_tools/vsn.mk b/lib/runtime_tools/vsn.mk index 4bbdef19de..9e87d5b144 100644 --- a/lib/runtime_tools/vsn.mk +++ b/lib/runtime_tools/vsn.mk @@ -1 +1 @@ -RUNTIME_TOOLS_VSN = 1.8.3 +RUNTIME_TOOLS_VSN = 1.8.4 diff --git a/lib/snmp/doc/src/snmpa.xml b/lib/snmp/doc/src/snmpa.xml index 1be6abe6dd..f546724a78 100644 --- a/lib/snmp/doc/src/snmpa.xml +++ b/lib/snmp/doc/src/snmpa.xml @@ -1233,7 +1233,7 @@ snmp_agent:register_subagent(SA1,[1,2,3], SA2). </type> <desc> <p>Restart the worker process of a multi-threaded agent.</p> - <p>This is a utility function, that can be usefull when + <p>This is a utility function, that can be useful when e.g. debugging instrumentation functions.</p> <marker id="restart_set_worker"></marker> @@ -1249,7 +1249,7 @@ snmp_agent:register_subagent(SA1,[1,2,3], SA2). </type> <desc> <p>Restart the set worker process of a multi-threaded agent.</p> - <p>This is a utility function, that can be usefull when + <p>This is a utility function, that can be useful when e.g. debugging instrumentation functions.</p> <marker id="verbosity"></marker> diff --git a/lib/ssl/Makefile b/lib/ssl/Makefile index a3dec8da38..b8b51270c9 100644 --- a/lib/ssl/Makefile +++ b/lib/ssl/Makefile @@ -1,19 +1,19 @@ # # %CopyrightBegin% -# -# Copyright Ericsson AB 1999-2009. All Rights Reserved. -# +# +# Copyright Ericsson AB 1999-2010. All Rights Reserved. +# # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in # compliance with the License. You should have received a copy of the # Erlang Public License along with this software. If not, it can be # retrieved online at http://www.erlang.org/. -# +# # Software distributed under the License is distributed on an "AS IS" # basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See # the License for the specific language governing rights and limitations # under the License. -# +# # %CopyrightEnd% # @@ -36,9 +36,9 @@ SKIP_BUILDING_BINARIES := false endif ifeq ($(SKIP_BUILDING_BINARIES), true) -SUB_DIRECTORIES = pkix src c_src doc/src +SUB_DIRECTORIES = src c_src doc/src else -SUB_DIRECTORIES = pkix src c_src doc/src examples/certs examples/src +SUB_DIRECTORIES = src c_src doc/src examples/certs examples/src endif include vsn.mk diff --git a/lib/ssl/doc/src/Makefile b/lib/ssl/doc/src/Makefile index fa263d28ab..3119d37af0 100644 --- a/lib/ssl/doc/src/Makefile +++ b/lib/ssl/doc/src/Makefile @@ -37,7 +37,7 @@ RELSYSDIR = $(RELEASE_PATH)/lib/$(APPLICATION)-$(VSN) # Target Specs # ---------------------------------------------------- XML_APPLICATION_FILES = refman.xml -XML_REF3_FILES = ssl.xml new_ssl.xml +XML_REF3_FILES = ssl.xml old_ssl.xml ssl_session_cache_api.xml XML_REF6_FILES = ssl_app.xml XML_PART_FILES = release_notes.xml usersguide.xml @@ -45,9 +45,7 @@ XML_CHAPTER_FILES = \ ssl_protocol.xml \ using_ssl.xml \ pkix_certs.xml \ - create_certs.xml \ ssl_distribution.xml \ - licenses.xml \ notes.xml BOOK_FILES = book.xml diff --git a/lib/ssl/doc/src/book.xml b/lib/ssl/doc/src/book.xml index 9122addb74..85d6b56b26 100644 --- a/lib/ssl/doc/src/book.xml +++ b/lib/ssl/doc/src/book.xml @@ -28,9 +28,6 @@ <rev>A</rev> <file>book.sgml</file> </header> - <insidecover> - <include file="insidecover"></include> - </insidecover> <pagetext>SSL Application</pagetext> <preamble> <contents level="2"></contents> diff --git a/lib/ssl/doc/src/create_certs.xml b/lib/ssl/doc/src/create_certs.xml deleted file mode 100644 index 79cc8a0537..0000000000 --- a/lib/ssl/doc/src/create_certs.xml +++ /dev/null @@ -1,148 +0,0 @@ -<?xml version="1.0" encoding="latin1" ?> -<!DOCTYPE chapter SYSTEM "chapter.dtd"> - -<chapter> - <header> - <copyright> - <year>2003</year><year>2009</year> - <holder>Ericsson AB. All Rights Reserved.</holder> - </copyright> - <legalnotice> - The contents of this file are subject to the Erlang Public License, - Version 1.1, (the "License"); you may not use this file except in - compliance with the License. You should have received a copy of the - Erlang Public License along with this software. If not, it can be - retrieved online at http://www.erlang.org/. - - Software distributed under the License is distributed on an "AS IS" - basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See - the License for the specific language governing rights and limitations - under the License. - - </legalnotice> - - <title>Creating Certificates</title> - <prepared>UAB/F/P Peter Högfeldt</prepared> - <docno></docno> - <date>2003-06-16</date> - <rev>A</rev> - <file>create_certs.xml</file> - </header> - <p>Here we consider the creation of example certificates. - </p> - - <section> - <title>The openssl Command</title> - <p>The <c>openssl</c> command is a utility that comes with the - OpenSSL distribution. It provides a variety of subcommands. Each - subcommand is invoked as</p> - <code type="none"><![CDATA[ - openssl subcmd <options and arguments> ]]></code> - <p>where <c>subcmd</c> denotes the subcommand in question. - </p> - <p>We shall use the following subcommands to create certificates for - the purpose of testing Erlang/OTP SSL: - </p> - <list type="bulleted"> - <item><em>req</em> to create certificate requests and a - self-signed certificates, - </item> - <item><em>ca</em> to create certificates from certificate requests.</item> - </list> - <p>We create the following certificates: - </p> - <list type="bulleted"> - <item>the <em>erlangCA</em> root certificate (a self-signed - certificate), </item> - <item>the <em>otpCA</em> certificate signed by the <em>erlangCA</em>, </item> - <item>a client certificate signed by the <em>otpCA</em>, and</item> - <item>a server certificate signed by the <em>otpCA</em>.</item> - </list> - - <section> - <title>The openssl configuration file</title> - <p>An <c>openssl</c> configuration file consist of a number of - sections, where each section starts with one line containing - <c>[ section_name ]</c>, where <c>section_name</c> is the name - of the section. The first section of the file is either - unnamed, or is named <c>[ default ]</c>. For further details - see the OpenSSL config(5) manual page. - </p> - <p>The required sections for the subcommands we are going to - use are as follows: - </p> - <table> - <row> - <cell align="left" valign="middle">subcommand</cell> - <cell align="left" valign="middle">required/default section</cell> - <cell align="left" valign="middle">override command line option</cell> - <cell align="left" valign="middle">configuration file option</cell> - </row> - <row> - <cell align="left" valign="middle">req</cell> - <cell align="left" valign="middle">[req]</cell> - <cell align="left" valign="middle">-</cell> - <cell align="left" valign="middle"><c>-config FILE</c></cell> - </row> - <row> - <cell align="left" valign="middle">ca</cell> - <cell align="left" valign="middle">[ca]</cell> - <cell align="left" valign="middle"><c>-name section</c></cell> - <cell align="left" valign="middle"><c>-config FILE</c></cell> - </row> - <tcaption>openssl subcommands to use</tcaption> - </table> - </section> - - <section> - <title>Creating the Erlang root CA</title> - <p>The Erlang root CA is created with the command</p> - <code type="none"> - openssl req -new -x509 -config /some/path/req.cnf \\ - -keyout /some/path/key.pem -out /some/path/cert.pem </code> - <p>where the option <c>-new</c> indicates that we want to create - a new certificate request and the option <c>-x509</c> implies - that a self-signed certificate is created. - </p> - </section> - - <section> - <title>Creating the OTP CA</title> - <p>The OTP CA is created by first creating a certificate request - with the command</p> - <code type="none"> - openssl req -new -config /some/path/req.cnf \\ - -keyout /some/path/key.pem -out /some/path/req.pem </code> - <p>and the ask the Erlang CA to sign it:</p> - <code type="none"> - openssl ca -batch -notext -config /some/path/req.cnf \\ - -extensions ca_cert -in /some/path/req.pem -out /some/path/cert.pem </code> - <p>where the option <c>-extensions</c> refers to a section in the - configuration file saying that it should create a CA certificate, - and not a plain user certificate. - </p> - <p>The <c>client</c> and <c>server</c> certificates are created - similarly, except that the option <c>-extensions</c> then has the - value <c>user_cert</c>. - </p> - </section> - </section> - - <section> - <title>An Example</title> - <p>The following module <c>create_certs</c> is used by the Erlang/OTP - SSL application for generating certificates to be used in tests. The - source code is also found in <c>ssl-X.Y.Z/examples/certs/src</c>. - </p> - <p>The purpose of the <c>create_certs:all/1</c> function is to make - it possible to provide from the <c>erl</c> command line, the - full path name of the <c>openssl</c> command. - </p> - <p>Note that the module creates temporary OpenSSL configuration files - for the <c>req</c> and <c>ca</c> subcommands. - </p> - <codeinclude file="../../examples/certs/src/make_certs.erl" tag="" type="erl"></codeinclude> - </section> -</chapter> - - diff --git a/lib/ssl/doc/src/insidecover.xml b/lib/ssl/doc/src/insidecover.xml deleted file mode 100644 index 4f3f5e5951..0000000000 --- a/lib/ssl/doc/src/insidecover.xml +++ /dev/null @@ -1,14 +0,0 @@ -<?xml version="1.0" encoding="latin1" ?> -<!DOCTYPE bookinsidecover SYSTEM "bookinsidecover.dtd"> - -<bookinsidecover> -The Erlang/OTP SSL application includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (http://www.openssl.org/). Copyright (c) 1998-2002 The OpenSSL Project. All rights reserved. <br></br> -This product includes cryptographic software written by Eric Young ([email protected]). This product includes software written by Tim Hudson ([email protected]). Copyright (C) 1995-1998 Eric Young ([email protected]). All rights reserved. <br></br> -For further OpenSSL and SSLeay license information se the chapter <bold>Licenses</bold> -. <vfill></vfill> - <br></br> - <tt>http://www.erlang.org</tt> - <br></br> -</bookinsidecover> - - diff --git a/lib/ssl/doc/src/licenses.xml b/lib/ssl/doc/src/licenses.xml deleted file mode 100644 index 0969f9ad6e..0000000000 --- a/lib/ssl/doc/src/licenses.xml +++ /dev/null @@ -1,156 +0,0 @@ -<?xml version="1.0" encoding="latin1" ?> -<!DOCTYPE chapter SYSTEM "chapter.dtd"> - -<chapter> - <header> - <copyright> - <year>2003</year><year>2009</year> - <holder>Ericsson AB. All Rights Reserved.</holder> - </copyright> - <legalnotice> - The contents of this file are subject to the Erlang Public License, - Version 1.1, (the "License"); you may not use this file except in - compliance with the License. You should have received a copy of the - Erlang Public License along with this software. If not, it can be - retrieved online at http://www.erlang.org/. - - Software distributed under the License is distributed on an "AS IS" - basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See - the License for the specific language governing rights and limitations - under the License. - - </legalnotice> - - <title>Licenses</title> - <prepared>Peter Högfeldt</prepared> - <docno></docno> - <date>2003-05-26</date> - <rev>A</rev> - <file>licenses.xml</file> - </header> - <p> <marker id="licenses"></marker> -This chapter contains in extenso versions - of the OpenSSL and SSLeay licenses. - </p> - - <section> - <title>OpenSSL License</title> - <code type="none"> -/* ==================================================================== - * Copyright (c) 1998-2002 The OpenSSL Project. All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in - * the documentation and/or other materials provided with the - * distribution. - * - * 3. All advertising materials mentioning features or use of this - * software must display the following acknowledgment: - * "This product includes software developed by the OpenSSL Project - * for use in the OpenSSL Toolkit. (http://www.openssl.org/)" - * - * 4. The names "OpenSSL Toolkit" and "OpenSSL Project" must not be used to - * endorse or promote products derived from this software without - * prior written permission. For written permission, please contact - * [email protected]. - * - * 5. Products derived from this software may not be called "OpenSSL" - * nor may "OpenSSL" appear in their names without prior written - * permission of the OpenSSL Project. - * - * 6. Redistributions of any form whatsoever must retain the following - * acknowledgment: - * "This product includes software developed by the OpenSSL Project - * for use in the OpenSSL Toolkit (http://www.openssl.org/)" - * - * THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT ``AS IS'' AND ANY - * EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE OpenSSL PROJECT OR - * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT - * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, - * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) - * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED - * OF THE POSSIBILITY OF SUCH DAMAGE. - * ==================================================================== - * - * This product includes cryptographic software written by Eric Young - * ([email protected]). This product includes software written by Tim - * Hudson ([email protected]). - * - */ </code> - </section> - - <section> - <title>SSLeay License</title> - <code type="none"> -/* Copyright (C) 1995-1998 Eric Young ([email protected]) - * All rights reserved. - * - * This package is an SSL implementation written - * by Eric Young ([email protected]). - * The implementation was written so as to conform with Netscapes SSL. - * - * This library is free for commercial and non-commercial use as long as - * the following conditions are aheared to. The following conditions - * apply to all code found in this distribution, be it the RC4, RSA, - * lhash, DES, etc., code; not just the SSL code. The SSL documentation - * included with this distribution is covered by the same copyright terms - * except that the holder is Tim Hudson ([email protected]). - * - * Copyright remains Eric Young's, and as such any Copyright notices in - * the code are not to be removed. - * If this package is used in a product, Eric Young should be given attribution - * as the author of the parts of the library used. - * This can be in the form of a textual message at program startup or - * in documentation (online or textual) provided with the package. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * 1. Redistributions of source code must retain the copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * 3. All advertising materials mentioning features or use of this software - * must display the following acknowledgement: - * "This product includes cryptographic software written by - * Eric Young ([email protected])" - * The word 'cryptographic' can be left out if the rouines from the library - * being used are not cryptographic related :-). - * 4. If you include any Windows specific code (or a derivative thereof) from - * the apps directory (application code) you must include an acknowledgement: - * "This product includes software written by Tim Hudson ([email protected])" - * - * THIS SOFTWARE IS PROVIDED BY ERIC YOUNG ``AS IS'' AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS - * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) - * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT - * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY - * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF - * SUCH DAMAGE. - * - * The licence and distribution terms for any publically available version or - * derivative of this code cannot be changed. i.e. this code cannot simply be - * copied and put under another distribution licence - * [including the GNU Public Licence.] - */ </code> - </section> -</chapter> - - diff --git a/lib/ssl/doc/src/make.dep b/lib/ssl/doc/src/make.dep deleted file mode 100644 index 2ff81bee1f..0000000000 --- a/lib/ssl/doc/src/make.dep +++ /dev/null @@ -1,30 +0,0 @@ -# ---------------------------------------------------- -# >>>> Do not edit this file <<<< -# This file was automaticly generated by -# /home/otp/bin/docdepend -# ---------------------------------------------------- - - -# ---------------------------------------------------- -# TeX files that the DVI file depend on -# ---------------------------------------------------- - -book.dvi: book.tex create_certs.tex licenses.tex new_ssl.tex \ - pkix_certs.tex refman.tex ssl.tex ssl_app.tex \ - ssl_distribution.tex ssl_protocol.tex usersguide.tex \ - using_ssl.tex - -# ---------------------------------------------------- -# Source inlined when transforming from source to LaTeX -# ---------------------------------------------------- - -book.tex: refman.xml - -create_certs.tex: ../../examples/certs/src/make_certs.erl - -using_ssl.tex: ../../examples/src/client_server.erl - -pkix_certs.tex: ../../../../system/doc/definitions/cite.defs - -ssl_protocol.tex: ../../../../system/doc/definitions/cite.defs - diff --git a/lib/ssl/doc/src/new_ssl.xml b/lib/ssl/doc/src/new_ssl.xml deleted file mode 100644 index 69298759bd..0000000000 --- a/lib/ssl/doc/src/new_ssl.xml +++ /dev/null @@ -1,694 +0,0 @@ -<?xml version="1.0" encoding="latin1" ?> -<!DOCTYPE erlref SYSTEM "erlref.dtd"> - -<erlref> - <header> - <copyright> - <year>1999</year> - <year>2007</year> - <holder>Ericsson AB, All Rights Reserved</holder> - </copyright> - <legalnotice> - The contents of this file are subject to the Erlang Public License, - Version 1.1, (the "License"); you may not use this file except in - compliance with the License. You should have received a copy of the - Erlang Public License along with this software. If not, it can be - retrieved aniline's at http://www.erlang.org/. - - Software distributed under the License is distributed on an "AS IS" - basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See - the License for the specific language governing rights and limitations - under the License. - - The Initial Developer of the Original Code is Ericsson AB. - </legalnotice> - <title>ssl</title> - <prepared>Ingela Anderton Andin</prepared> - <responsible>Ingela Anderton Andin</responsible> - <docno></docno> - <approved></approved> - <checked></checked> - <date>2003-03-25</date> - <rev></rev> - <file>new_ssl.xml</file> - </header> - <module>new_ssl</module> - <modulesummary>Interface Functions for Secure Socket Layer</modulesummary> - <description> - <p>This module contains interface functions to the Secure Socket - Layer. - </p> - </description> - - <section> - <title>NEW SSL</title> - - <p>This manual page describes functions that are defined - in the ssl module and represents the new ssl implementation - that coexists with the old one, as the new implementation - is not yet complete enough to replace the old one.</p> - - <p>The new implementation can be - accessed by providing the option {ssl_imp, new} to the - ssl:connect and ssl:listen functions.</p> - - <p>The new implementation is Erlang based and all logic - is in Erlang and only payload encryption calculations are - done in C via the crypto application. The main reason for - making a new implementation is that the old solution was - very crippled as the control of the ssl-socket was deep - down in openssl making it hard if not impossible to - support all inet options, ipv6 and upgrade of a tcp - connection to a ssl connection. This version has a - few limitations that will be removed before the ssl-4.0 - release. Main differences and limitations are listed below.</p> - - <list type="bulleted"> - <item>New ssl requires the crypto - application.</item> - <item>The option reuseaddr is - supported and the default value is false as in gen_tcp. - Old ssl is patched to accept that the option is set to - true to provide a smoother migration between the - versions. In old ssl the option is hard coded to - true.</item> - <item>ssl:version/0 is replaced by - ssl:versions/0</item> - <item>ssl:ciphers/0 is replaced by - ssl:cipher_suites/0</item> - <item>ssl:pid/1 is a - meaningless function in new ssl and will be deprecated in - ssl-4.0 until it is removed it will return a valid but - meaningless pid.</item> - <item>New API functions are - ssl:shutdown/2, ssl:cipher_suites/[0,1] and - ssl:versions/0, ssl:renegotiate/1</item> - <item>CRL and policy certificate - extensions are not supported yet. </item> - <item>Supported SSL/TLS-versions are SSL-3.0 and TLS-1.0 </item> - <item>For security reasons sslv2 is not supported.</item> - <item>Ephemeral Diffie-Hellman cipher suites are supported - but not Diffie Hellman Certificates cipher suites.</item> - <item>Export cipher suites are not supported as the - U.S. lifted its export restrictions in early 2000.</item> - </list> - - </section> - - <section> - <title>COMMON DATA TYPES</title> - <p>The following data types are used in the functions below: - </p> - - <p><c>boolean() = true | false</c></p> - - <p><c>property() = atom()</c></p> - - <p><c>option() = socketoption() | ssloption() | transportoption()</c></p> - - <p><c>socketoption() = [{property(), term()}] - defaults to - [{mode,list},{packet, 0},{header, 0},{active, true}]. - </c></p> - - <p>For valid options - see <seealso marker="kernel:inet">inet(3) </seealso> and - <seealso marker="kernel:gen_tcp">gen_tcp(3) </seealso>. - </p> - - <p> <c>ssloption() = {verify, verify_type()} | - {fail_if_no_peer_cert, boolean()} - {depth, integer()} | - {certfile, path()} | {keyfile, path()} | {password, string()} | - {cacertfile, path()} | {dhfile, path()} | {ciphers, ciphers()} | - {ssl_imp, ssl_imp()} | {reuse_sessions, boolean()} | {reuse_session, fun()} - </c></p> - - <p><c>transportoption() = {CallbackModule, DataTag, ClosedTag} - - defaults to {gen_tcp, tcp, tcp_closed}. Ssl may be - run over any reliable transport protocol that has - an equivalent API to gen_tcp's.</c></p> - - <p><c> CallbackModule = - atom()</c> - </p> <p><c> DataTag = - atom() - tag used in socket data message.</c></p> - <p><c> ClosedTag = atom() - tag used in - socket close message.</c></p> - - <p><c>verify_type() = verify_none | verify_peer</c></p> - - <p><c>path() = string() - representing a file path.</c></p> - - <p><c>host() = hostname() | ipaddress()</c></p> - - <p><c>hostname() = string()</c></p> - - <p><c> - ip_address() = {N1,N2,N3,N4} % IPv4 - | {K1,K2,K3,K4,K5,K6,K7,K8} % IPv6 </c></p> - - <p><c>sslsocket() - opaque to the user. </c></p> - - <p><c>protocol() = sslv3 | tlsv1 </c></p> - - <p><c>ciphers() = [ciphersuite()] | string() (according to old API)</c></p> - - <p><c>ciphersuite() = - {key_exchange(), cipher(), hash()}</c></p> - - <p><c>key_exchange() = rsa | dhe_dss | dhe_rsa - </c></p> - - <p><c>cipher() = rc4_128 | des_cbc | '3des_ede_cbc' - | aes_128_cbc | aes_256_cbc </c></p> - - <p> <c>hash() = md5 | sha - </c></p> - - <p><c>ssl_imp() = new | old - default is old.</c></p> - - </section> - -<section> - <title>SSL OPTION DESCRIPTIONS</title> - - <taglist> - <tag>{verify, verify_type()}</tag> - <item> If <c>verify_none</c> is specified x509-certificate - path validation errors at the client side - will not automatically cause the connection to fail, as - it will if the verify type is <c>verify_peer</c>. See also - the option verify_fun. - Servers only do the path validation if <c>verify_peer</c> is set to - true, as it then will - send a certificate request to - the client (this message is not sent if the verify option is - <c>verify_none</c>) and you may then also want to specify - the option <c>fail_if_no_peer_cert</c>. - </item> - - <tag>{fail_if_no_peer_cert, boolean()}</tag> - <item>Used together with {verify, verify_peer} by a ssl server. - If set to true, - the server will fail if the client does not have a certificate - to send, e.i sends a empty certificate, if set to false it will - only fail if the client sends a invalid certificate (an empty - certificate is considered valid). - </item> - - <tag>{verify_fun, fun(ErrorList) -> boolean()}</tag> - <item>Used by the ssl client to determine if - x509-certificate path validations errors are acceptable or - if the connection should fail. Defaults to: - -<code> -fun(ErrorList) -> - case lists:foldl(fun({bad_cert,unknown_ca}, Acc) -> - Acc; - (Other, Acc) -> - [Other | Acc] - end, [], ErrorList) of - [] -> - true; - [_|_] -> - false - end -end -</code> - I.e. by default if the only error found was that the CA-certificate - holder was unknown this will be accepted. - - Possible errors in the error list are: - {bad_cert, cert_expired}, {bad_cert, invalid_issuer}, - {bad_cert, invalid_signature}, {bad_cert, name_not_permitted}, - {bad_cert, unknown_ca}, - {bad_cert, cert_expired}, {bad_cert, invalid_issuer}, - {bad_cert, invalid_signature}, {bad_cert, name_not_permitted}, - {bad_cert, cert_revoked} (not implemented yet), - {bad_cert, unknown_critical_extension} or {bad_cert, term()} (Will - be relevant later when an option is added for the user to be able to verify application specific extensions.) - </item> - - <tag>{depth, integer()}</tag> - <item>Specifies the maximum - verification depth, i.e. how far in a chain of certificates the - verification process can proceed before the verification is - considered to fail. Peer certificate = 0, CA certificate = 1, - higher level CA certificate = 2, etc. The value 2 thus means - that a chain can at most contain peer cert, CA cert, next CA - cert, and an additional CA cert. The default value is 1. - </item> - - <tag>{certfile, path()}</tag> - <item>Path to a file containing the - user's certificate. Optional for clients but note - that some servers requires that the client can certify - itself. </item> - <tag>{keyfile, path()}</tag> - <item>Path to file containing user's - private PEM encoded key. As PEM-files may contain several - entries this option defaults to the same file as given by - certfile option.</item> - <tag>{password, string()}</tag> - <item>String containing the user's password. - Only used if the private keyfile is password protected. - </item> - <tag>{cacertfile, path()}</tag> - <item>Path to file containing PEM encoded - CA certificates (trusted certificates used for verifying a peer - certificate). May be omitted if you do not want to verify - the peer.</item> - - <tag>{dhfile, path()}</tag> - <item>Path to file containing PEM encoded Diffie Hellman parameters, - for the server to use if a cipher suite using Diffie Hellman key exchange - is negotiated. If not specified hardcode parameters will be used. - </item> - - <tag>{ciphers, ciphers()}</tag> - <item>The function <c>ciphers_suites/0</c> can - be used to find all available ciphers. - </item> - - <tag>{ssl_imp, ssl_imp()}</tag> - <item>Specify which ssl implementation you want to use. - </item> - - <tag>{reuse_sessions, boolean()}</tag> - <item>Specifies if ssl sessions should be reused - when possible. - </item> - - <tag>{reuse_session, fun(SuggestedSessionId, - PeerCert, Compression, CipherSuite) -> boolean()}</tag> - <item>Enables the ssl server to have a local policy - for deciding if a session should be reused or not, - only meaning full if <c>reuse_sessions</c> is set to true. - SuggestedSessionId is a binary(), PeerCert is a DER encoded - certificate, Compression is an enumeration integer - and CipherSuite of type ciphersuite(). - </item> - </taglist> - </section> - - <section> - <title>General</title> - - <p>When a ssl socket is in active mode (the default), data from the - socket is delivered to the owner of the socket in the form of - messages: - </p> - <list type="bulleted"> - <item>{ssl, Socket, Data} - </item> - <item>{ssl_closed, Socket} - </item> - <item> - {ssl_error, Socket, Reason} - </item> - </list> - - <p>A <c>Timeout</c> argument specifies a timeout in milliseconds. The - default value for a <c>Timeout</c> argument is <c>infinity</c>. - </p> - </section> - - <funcs> - <func> - <name>cipher_suites() -></name> - <name>cipher_suites(Type) -> ciphers()</name> - <fsummary> Returns a list of supported cipher suites</fsummary> - <type> - <v>Type = erlang | openssl</v> - - </type> - <desc><p>Returns a list of supported cipher suites. - cipher_suites() is equivalent to cipher_suites(erlang). - Type openssl is provided for backwards compatibility with - old ssl that used openssl. - </p> - </desc> - </func> - - <func> - <name>connect(Socket, SslOptions) -> </name> - <name>connect(Socket, SslOptions, Timeout) -> {ok, SslSocket} - | {error, Reason}</name> - <fsummary> Upgrades a gen_tcp, or - equivalent, connected socket to a ssl socket. </fsummary> - <type> - <v>Socket = socket()</v> - <v>SslOptions = [ssloption()]</v> - <v>Timeout = integer() | infinity</v> - <v>SslSocket = sslsocket()</v> - <v>Reason = term()</v> - </type> - <desc> <p>Upgrades a gen_tcp, or equivalent, - connected socket to a ssl socket e.i performs the - client-side ssl handshake.</p> - </desc> - </func> - - <func> - <name>connect(Host, Port, Options) -></name> - <name>connect(Host, Port, Options, Timeout) -> - {ok, SslSocket} | {error, Reason}</name> - <fsummary>Opens an ssl connection to Host, Port. </fsummary> - <type> - <v>Host = host()</v> - <v>Port = integer()</v> - <v>Options = [option()]</v> - <v>Timeout = integer() | infinity</v> - <v>SslSocket = sslsocket()</v> - <v>Reason = term()</v> - </type> - <desc> <p>Opens an ssl connection to Host, Port.</p> </desc> - </func> - - <func> - <name>close(SslSocket) -> ok | {error, Reason}</name> - <fsummary>Close a ssl connection</fsummary> - <type> - <v>SslSocket = sslsocket()</v> - <v>Reason = term()</v> - </type> - <desc><p>Close a ssl connection.</p> - </desc> - </func> - - <func> - <name>controlling_process(SslSocket, NewOwner) -> - ok | {error, Reason}</name> - - <fsummary>Assigns a new controlling process to the - ssl-socket.</fsummary> - - <type> - <v>SslSocket = sslsocket()</v> - <v>NewOwner = pid()</v> - <v>Reason = term()</v> - </type> - <desc><p>Assigns a new controlling process to the ssl-socket. A - controlling process is the owner of a ssl-socket, and receives - all messages from the socket.</p> - </desc> - </func> - - <func> - <name>connection_info(SslSocket) -> - {ok, {ProtocolVersion, CipherSuite}} | {error, Reason} </name> - <fsummary>Returns the negotiated protocol version and cipher suite. - </fsummary> - <type> - <v>CipherSuite = ciphersuite()</v> - <v>ProtocolVersion = protocol()</v> - </type> - <desc><p>Returns the negotiated protocol version and cipher suite.</p> - </desc> - </func> - - <func> - <name>format_error(Reason) -> string()</name> - <fsummary>Return an error string.</fsummary> - <type> - <v>Reason = term()</v> - </type> - <desc> - <p>Presents the error returned by an ssl function as a printable string.</p> - </desc> - </func> - - <func> - <name>getopts(Socket) -> </name> - <name>getopts(Socket, OptionNames) -> - {ok, [socketoption()]} | {error, Reason}</name> - <fsummary>Get the value of the specified options.</fsummary> - <type> - <v>Socket = sslsocket()</v> - <v>OptionNames = [property()]</v> - </type> - <desc> - <p>Get the value of the specified socket options, if no - options are specified all options are returned. - </p> - </desc> - </func> - - <func> - <name>listen(Port, Options) -> - {ok, ListenSocket} | {error, Reason}</name> - <fsummary>Creates a ssl listen socket.</fsummary> - <type> - <v>Port = integer()</v> - <v>Options = options()</v> - <v>ListenSocket = sslsocket()</v> - </type> - <desc> - <p>Creates a ssl listen socket.</p> - </desc> - </func> - - <func> - <name>peercert(Socket) -> {ok, Cert} | {error, Reason}</name> - <fsummary>Return the peer certificate.</fsummary> - <type> - <v>Socket = sslsocket()</v> - <v>Cert = binary()</v> - <v>Subject = term()</v> - </type> - <desc> - <p>The peer certificate is returned as a DER encoded binary. - The certificate can be decoded with <c>public_key:pkix_decode_cert/2</c>. - </p> - </desc> - </func> - <func> - <name>peername(Socket) -> {ok, {Address, Port}} | - {error, Reason}</name> - <fsummary>Return peer address and port.</fsummary> - <type> - <v>Socket = sslsocket()</v> - <v>Address = ipaddress()</v> - <v>Port = integer()</v> - </type> - <desc> - <p>Returns the address and port number of the peer.</p> - </desc> - </func> - - <func> - <name>recv(Socket, Length) -> </name> - <name>recv(Socket, Length, Timeout) -> {ok, Data} | {error, - Reason}</name> - <fsummary>Receive data on a socket.</fsummary> - <type> - <v>Socket = sslsocket()</v> - <v>Length = integer()</v> - <v>Timeout = integer()</v> - <v>Data = [char()] | binary()</v> - </type> - <desc> - <p>This function receives a packet from a socket in passive - mode. A closed socket is indicated by a return value - <c>{error, closed}</c>.</p> - <p>The <c>Length</c> argument is only meaningful when - the socket is in <c>raw</c> mode and denotes the number of - bytes to read. If <c>Length</c> = 0, all available bytes are - returned. If <c>Length</c> > 0, exactly <c>Length</c> - bytes are returned, or an error; possibly discarding less - than <c>Length</c> bytes of data when the socket gets closed - from the other side.</p> - <p>The optional <c>Timeout</c> parameter specifies a timeout in - milliseconds. The default value is <c>infinity</c>.</p> - </desc> - </func> - - <func> - <name>renegotiate(Socket) -> ok | {error, Reason}</name> - <fsummary> Initiates a new handshake.</fsummary> - <type> - <v>Socket = sslsocket()</v> - </type> - <desc><p>Initiates a new handshake. A notable return value is - <c>{error, renegotiation_rejected}</c> indicating that the peer - refused to go through with the renegotiation but the connection - is still active using the previously negotiated session.</p> - </desc> - </func> - - <func> - <name>send(Socket, Data) -> ok | {error, Reason}</name> - <fsummary>Write data to a socket.</fsummary> - <type> - <v>Socket = sslsocket()</v> - <v>Data = iolist() | binary()</v> - </type> - <desc> - <p>Writes <c>Data</c> to <c>Socket</c>. </p> - <p>A notable return value is <c>{error, closed}</c> indicating that - the socket is closed.</p> - </desc> - </func> - <func> - <name>setopts(Socket, Options) -> ok | {error, Reason}</name> - <fsummary>Set socket options.</fsummary> - <type> - <v>Socket = sslsocket()</v> - <v>Options = [socketoption]()</v> - </type> - <desc> - <p>Sets options according to <c>Options</c> for the socket - <c>Socket</c>. </p> - </desc> - </func> - - <func> - <name>shutdown(Socket, How) -> ok | {error, Reason}</name> - <fsummary>Immediately close a socket</fsummary> - <type> - <v>Socket = sslsocket()</v> - <v>How = read | write | read_write</v> - <v>Reason = reason()</v> - </type> - <desc> - <p>Immediately close a socket in one or two directions.</p> - <p><c>How == write</c> means closing the socket for writing, - reading from it is still possible.</p> - <p>To be able to handle that the peer has done a shutdown on - the write side, the <c>{exit_on_close, false}</c> option - is useful.</p> - </desc> - </func> - - <func> - <name>ssl_accept(ListenSocket) -> </name> - <name>ssl_accept(ListenSocket, Timeout) -> ok | {error, Reason}</name> - <fsummary>Perform server-side SSL handshake</fsummary> - <type> - <v>ListenSocket = sslsocket()</v> - <v>Timeout = integer()</v> - <v>Reason = term()</v> - </type> - <desc> - <p>The <c>ssl_accept</c> function establish the SSL connection - on the server side. It should be called directly after - <c>transport_accept</c>, in the spawned server-loop.</p> - </desc> - </func> - - <func> - <name>ssl_accept(ListenSocket, SslOptions) -> </name> - <name>ssl_accept(ListenSocket, SslOptions, Timeout) -> {ok, Socket} | {error, Reason}</name> - <fsummary>Perform server-side SSL handshake</fsummary> - <type> - <v>ListenSocket = socket()</v> - <v>SslOptions = ssloptions()</v> - <v>Timeout = integer()</v> - <v>Reason = term()</v> - </type> - <desc> - <p> Upgrades a gen_tcp, or - equivalent, socket to a ssl socket e.i performs the - ssl server-side handshake.</p> - <p><note>Note that the listen socket should be in {active, false} mode - before telling the client that the server is ready to upgrade - and calling this function, otherwise the upgrade may - or may not succeed depending on timing.</note></p> - </desc> - </func> - - <func> - <name>sockname(Socket) -> {ok, {Address, Port}} | - {error, Reason}</name> - <fsummary>Return the local address and port.</fsummary> - <type> - <v>Socket = sslsocket()</v> - <v>Address = ipaddress()</v> - <v>Port = integer()</v> - </type> - <desc> - <p>Returns the local address and port number of the socket - <c>Socket</c>.</p> - </desc> - </func> - - <func> - <name>start() -> </name> - <name>start(Type) -> ok | {error, Reason}</name> - <fsummary>Starts the Ssl application. </fsummary> - <type> - <v>Type = permanent | transient | temporary</v> - </type> - <desc> - <p>Starts the Ssl application. Default type - is temporary. - <seealso marker="kernel:application">application(3)</seealso></p> - </desc> - </func> - <func> - <name>stop() -> ok </name> - <fsummary>Stops the Ssl application.</fsummary> - <desc> - <p>Stops the Ssl application. - <seealso marker="kernel:application">application(3)</seealso></p> - </desc> - </func> - - <func> - <name>transport_accept(Socket) -></name> - <name>transport_accept(Socket, Timeout) -> - {ok, NewSocket} | {error, Reason}</name> - <fsummary>Accept an incoming connection and - prepare for <c>ssl_accept</c></fsummary> - <type> - <v>Socket = NewSocket = sslsocket()</v> - <v>Timeout = integer()</v> - <v>Reason = reason()</v> - </type> - <desc> - <p>Accepts an incoming connection request on a listen socket. - <c>ListenSocket</c> must be a socket returned from - <c>listen/2</c>. The socket returned should be passed to - <c>ssl_accept</c> to complete ssl handshaking and - establishing the connection.</p> - <warning> - <p>The socket returned can only be used with <c>ssl_accept</c>, - no traffic can be sent or received before that call.</p> - </warning> - <p>The accepted socket inherits the options set for - <c>ListenSocket</c> in <c>listen/2</c>.</p> - <p>The default - value for <c>Timeout</c> is <c>infinity</c>. If - <c>Timeout</c> is specified, and no connection is accepted - within the given time, <c>{error, timeout}</c> is - returned.</p> - </desc> - </func> - - <func> - <name>versions() -> - [{SslAppVer, SupportedSslVer, AvailableSslVsn}]</name> - <fsummary>Returns version information relevant for the - ssl application.</fsummary> - <type> - <v>SslAppVer = string()</v> - <v>SupportedSslVer = [protocol()]</v> - <v>AvailableSslVsn = [protocol()]</v> - </type> - <desc> - <p> - Returns version information relevant for the - ssl application.</p> - </desc> - </func> - </funcs> - - <section> - <title>SEE ALSO</title> - <p><seealso marker="kernel:inet">inet(3) </seealso> and - <seealso marker="kernel:gen_tcp">gen_tcp(3) </seealso> - </p> - </section> - -</erlref> - diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index f213bd11ae..95e968aa22 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -31,6 +31,31 @@ <p>This document describes the changes made to the SSL application. </p> + <section><title>SSL 4.0</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + New ssl now support client/server-certificates signed by + dsa keys.</p> + <p> + Own Id: OTP-8587</p> + </item> + <item> + <p> + Ssl has now switched default implementation and removed + deprecated certificate handling. All certificate handling + is done by the public_key application.</p> + <p> + Own Id: OTP-8695</p> + </item> + </list> + </section> + + </section> + + <section><title>SSL 3.11.1</title> <section><title>Fixed Bugs and Malfunctions</title> @@ -794,7 +819,7 @@ <title>Fixed Bugs and Malfunctions</title> <list type="bulleted"> <item> - <p>When a file descriptor was marked for closing, and and + <p>When a file descriptor was marked for closing, and end-of-file condition had already been detected, the file descriptor was never closed.</p> <p>Own Id: OTP-5093 Aux Id: seq8806 </p> diff --git a/lib/ssl/doc/src/old_ssl.xml b/lib/ssl/doc/src/old_ssl.xml new file mode 100644 index 0000000000..0d2e1afdbd --- /dev/null +++ b/lib/ssl/doc/src/old_ssl.xml @@ -0,0 +1,709 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>1999</year><year>2010</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + + <title>ssl</title> + <prepared>Peter Högfeldt</prepared> + <responsible>Peter Högfeldt</responsible> + <docno></docno> + <approved>Peter Högfeldt</approved> + <checked></checked> + <date>2003-03-25</date> + <rev>D</rev> + <file>old_ssl.xml</file> + </header> + <module>old_ssl</module> + <modulesummary>Interface Functions for Secure Socket Layer</modulesummary> + <description> + <p>This module contains interface functions to the Secure Socket Layer.</p> + </description> + + <section> + <title>General</title> + + <p>This manual page describes functions that are defined + in the ssl module and represents the old ssl implementation + that coexists with the new one until it has been + totally phased out. </p> + + <p>The old implementation can be + accessed by providing the option {ssl_imp, old} to the + ssl:connect and ssl:listen functions.</p> + + <p>The reader is advised to also read the <c>ssl(6)</c> manual page + describing the SSL application. + </p> + <warning> + <p>It is strongly advised to seed the random generator after + the ssl application has been started (see <c>seed/1</c> + below), and before any connections are established. Although + the port program interfacing to the ssl libraries does a + "random" seeding of its own in order to make everything work + properly, that seeding is by no means random for the world + since it has a constant value which is known to everyone + reading the source code of the port program.</p> + </warning> + </section> + + <section> + <title>Common data types</title> + <p>The following datatypes are used in the functions below: + </p> + <list type="bulleted"> + <item> + <p><c>options() = [option()]</c></p> + </item> + <item> + <p><c>option() = socketoption() | ssloption()</c></p> + </item> + <item> + <p><c>socketoption() = {mode, list} | {mode, binary} | binary | {packet, packettype()} | {header, integer()} | {nodelay, boolean()} | {active, activetype()} | {backlog, integer()} | {ip, ipaddress()} | {port, integer()}</c></p> + </item> + <item> + <p><c>ssloption() = {verify, code()} | {depth, depth()} | {certfile, path()} | {keyfile, path()} | {password, string()} | {cacertfile, path()} | {ciphers, string()}</c></p> + </item> + <item> + <p><c>packettype()</c> (see inet(3))</p> + </item> + <item> + <p><c>activetype()</c> (see inet(3))</p> + </item> + <item> + <p><c>reason() = atom() | {atom(), string()}</c></p> + </item> + <item> + <p><c>bytes() = [byte()]</c></p> + </item> + <item> + <p><c>string() = [byte()]</c></p> + </item> + <item> + <p><c>byte() = 0 | 1 | 2 | ... | 255</c></p> + </item> + <item> + <p><c>code() = 0 | 1 | 2</c></p> + </item> + <item> + <p><c>depth() = byte()</c></p> + </item> + <item> + <p><c>address() = hostname() | ipstring() | ipaddress()</c></p> + </item> + <item> + <p><c>ipaddress() = ipstring() | iptuple()</c></p> + </item> + <item> + <p><c>hostname() = string()</c></p> + </item> + <item> + <p><c>ipstring() = string()</c></p> + </item> + <item> + <p><c>iptuple() = {byte(), byte(), byte(), byte()}</c></p> + </item> + <item> + <p><c>sslsocket()</c></p> + </item> + <item> + <p><c>protocol() = sslv2 | sslv3 | tlsv1</c></p> + </item> + <item> + <p><c></c></p> + </item> + </list> + <p>The socket option <c>{backlog, integer()}</c> is for + <c>listen/2</c> only, and the option <c>{port, integer()}</c> + is for <c>connect/3/4</c> only. + </p> + <p>The following socket options are set by default: <c>{mode, list}</c>, <c>{packet, 0}</c>, <c>{header, 0}</c>, <c>{nodelay, false}</c>, <c>{active, true}</c>, <c>{backlog, 5}</c>, + <c>{ip, {0,0,0,0}}</c>, and <c>{port, 0}</c>. + </p> + <p>Note that the options <c>{mode, binary}</c> and <c>binary</c> + are equivalent. Similarly <c>{mode, list}</c> and the absence of + option <c>binary</c> are equivalent. + </p> + <p>The ssl options are for setting specific SSL parameters as follows: + </p> + <list type="bulleted"> + <item> + <p><c>{verify, code()}</c> Specifies type of verification: + 0 = do not verify peer; 1 = verify peer, 2 = verify peer, + fail if no peer certificate. The default value is 0. + </p> + </item> + <item> + <p><c>{depth, depth()}</c> Specifies the maximum + verification depth, i.e. how far in a chain of certificates + the verification process can proceed before the verification + is considered to fail. + </p> + <p>Peer certificate = 0, CA certificate = 1, higher level CA + certificate = 2, etc. The value 2 thus means that a chain + can at most contain peer cert, CA cert, next CA cert, and an + additional CA cert. + </p> + <p>The default value is 1. + </p> + </item> + <item> + <p><c>{certfile, path()}</c> Path to a file containing the + user's certificate. + chain of PEM encoded certificates.</p> + </item> + <item> + <p><c>{keyfile, path()}</c> Path to file containing user's + private PEM encoded key.</p> + </item> + <item> + <p><c>{password, string()}</c> String containing the user's + password. Only used if the private keyfile is password protected.</p> + </item> + <item> + <p><c>{cacertfile, path()}</c> Path to file containing PEM encoded + CA certificates (trusted certificates used for verifying a peer + certificate).</p> + </item> + <item> + <p><c>{ciphers, string()}</c> String of ciphers as a colon + separated list of ciphers. The function <c>ciphers/0</c> can + be used to find all available ciphers.</p> + </item> + </list> + <p>The type <c>sslsocket()</c> is opaque to the user. + </p> + <p>The owner of a socket is the one that created it by a call to + <c>transport_accept/[1,2]</c>, <c>connect/[3,4]</c>, + or <c>listen/2</c>. + </p> + <p>When a socket is in active mode (the default), data from the + socket is delivered to the owner of the socket in the form of + messages: + </p> + <list type="bulleted"> + <item> + <p><c>{ssl, Socket, Data}</c></p> + </item> + <item> + <p><c>{ssl_closed, Socket}</c></p> + </item> + <item> + <p><c>{ssl_error, Socket, Reason}</c></p> + </item> + </list> + <p>A <c>Timeout</c> argument specifies a timeout in milliseconds. The + default value for a <c>Timeout</c> argument is <c>infinity</c>. + </p> + <p>Functions listed below may return the value <c>{error, closed}</c>, which only indicates that the SSL socket is + considered closed for the operation in question. It is for + instance possible to have <c>{error, closed}</c> returned from + an call to <c>send/2</c>, and a subsequent call to <c>recv/3</c> + returning <c>{ok, Data}</c>. + </p> + <p>Hence a return value of <c>{error, closed}</c> must not be + interpreted as if the socket was completely closed. On the + contrary, in order to free all resources occupied by an SSL + socket, <c>close/1</c> must be called, or else the process owning + the socket has to terminate. + </p> + <p>For each SSL socket there is an Erlang process representing the + socket. When a socket is opened, that process links to the + calling client process. Implementations that want to detect + abnormal exits from the socket process by receiving <c>{'EXIT', Pid, Reason}</c> messages, should use the function <c>pid/1</c> + to retrieve the process identifier from the socket, in order to + be able to match exit messages properly.</p> + </section> + <funcs> + <func> + <name>ciphers() -> {ok, string()} | {error, enotstarted}</name> + <fsummary>Get supported ciphers.</fsummary> + <desc> + <p>Returns a string consisting of colon separated cipher + designations that are supported by the current SSL library + implementation. + </p> + <p>The SSL application has to be started to return the string + of ciphers.</p> + </desc> + </func> + <func> + <name>close(Socket) -> ok | {error, Reason}</name> + <fsummary>Close a socket returned by <c>transport_accept/[1,2]</c>, <c>connect/3/4</c>, or <c>listen/2</c>.</fsummary> + <type> + <v>Socket = sslsocket()</v> + </type> + <desc> + <p>Closes a socket returned by <c>transport_accept/[1,2]</c>, + <c>connect/[3,4]</c>, or <c>listen/2</c></p> + </desc> + </func> + <func> + <name>connect(Address, Port, Options) -> {ok, Socket} | {error, Reason}</name> + <name>connect(Address, Port, Options, Timeout) -> {ok, Socket} | {error, Reason}</name> + <fsummary>Connect to <c>Port</c>at <c>Address</c>.</fsummary> + <type> + <v>Address = address()</v> + <v>Port = integer()</v> + <v>Options = [connect_option()]</v> + <v>connect_option() = {mode, list} | {mode, binary} | binary | {packet, packettype()} | {header, integer()} | {nodelay, boolean()} | {active, activetype()} | {ip, ipaddress()} | {port, integer()} | {verify, code()} | {depth, depth()} | {certfile, path()} | {keyfile, path()} | {password, string()} | {cacertfile, path()} | {ciphers, string()}</v> + <v>Timeout = integer()</v> + <v>Socket = sslsocket()</v> + </type> + <desc> + <p>Connects to <c>Port</c> at <c>Address</c>. If the optional + <c>Timeout</c> argument is specified, and a connection could not + be established within the given time, <c>{error, timeout}</c> is + returned. The default value for <c>Timeout</c> is <c>infinity</c>. + </p> + <p>The <c>ip</c> and <c>port</c> options are for binding to a + particular <em>local</em> address and port, respectively.</p> + </desc> + </func> + <func> + <name>connection_info(Socket) -> {ok, {Protocol, Cipher}} | {error, Reason}</name> + <fsummary>Get current protocol version and cipher.</fsummary> + <type> + <v>Socket = sslsocket()</v> + <v>Protocol = protocol()</v> + <v>Cipher = string()</v> + </type> + <desc> + <p>Gets the chosen protocol version and cipher for an established + connection (accepted och connected). </p> + </desc> + </func> + <func> + <name>controlling_process(Socket, NewOwner) -> ok | {error, Reason}</name> + <fsummary>Assign a new controlling process to the socket.</fsummary> + <type> + <v>Socket = sslsocket()</v> + <v>NewOwner = pid()</v> + </type> + <desc> + <p>Assigns a new controlling process to <c>Socket</c>. A controlling + process is the owner of a socket, and receives all messages from + the socket.</p> + </desc> + </func> + <func> + <name>format_error(ErrorCode) -> string()</name> + <fsummary>Return an error string.</fsummary> + <type> + <v>ErrorCode = term()</v> + </type> + <desc> + <p>Returns a diagnostic string describing an error.</p> + </desc> + </func> + <func> + <name>getopts(Socket, OptionsTags) -> {ok, Options} | {error, Reason}</name> + <fsummary>Get options set for socket</fsummary> + <type> + <v>Socket = sslsocket()</v> + <v>OptionTags = [optiontag()]()</v> + </type> + <desc> + <p>Returns the options the tags of which are <c>OptionTags</c> for + for the socket <c>Socket</c>. </p> + </desc> + </func> + <func> + <name>listen(Port, Options) -> {ok, ListenSocket} | {error, Reason}</name> + <fsummary>Set up a socket to listen on a port on the local host.</fsummary> + <type> + <v>Port = integer()</v> + <v>Options = [listen_option()]</v> + <v>listen_option() = {mode, list} | {mode, binary} | binary | {packet, packettype()} | {header, integer()} | {active, activetype()} | {backlog, integer()} | {ip, ipaddress()} | {verify, code()} | {depth, depth()} | {certfile, path()} | {keyfile, path()} | {password, string()} | {cacertfile, path()} | {ciphers, string()}</v> + <v>ListenSocket = sslsocket()</v> + </type> + <desc> + <p>Sets up a socket to listen on port <c>Port</c> at the local host. + If <c>Port</c> is zero, <c>listen/2</c> picks an available port + number (use <c>port/1</c> to retrieve it). + </p> + <p>The listen queue size defaults to 5. If a different value is + wanted, the option <c>{backlog, Size}</c> should be added to the + list of options. + </p> + <p>An empty <c>Options</c> list is considered an error, and + <c>{error, enooptions}</c> is returned. + </p> + <p>The returned <c>ListenSocket</c> can only be used in calls to + <c>transport_accept/[1,2]</c>.</p> + </desc> + </func> + <func> + <name>peercert(Socket) -> {ok, Cert} | {error, Reason}</name> + <fsummary>Return the peer certificate.</fsummary> + <type> + <v>Socket = sslsocket()</v> + <v>Cert = binary()()</v> + <v>Subject = term()()</v> + </type> + <desc> + <p>Returns the DER encoded peer certificate, the certificate can be decoded with + <c>public_key:pkix_decode_cert/2</c>. + </p> + </desc> + </func> + <func> + <name>peername(Socket) -> {ok, {Address, Port}} | {error, Reason}</name> + <fsummary>Return peer address and port.</fsummary> + <type> + <v>Socket = sslsocket()</v> + <v>Address = ipaddress()</v> + <v>Port = integer()</v> + </type> + <desc> + <p>Returns the address and port number of the peer.</p> + </desc> + </func> + <func> + <name>pid(Socket) -> pid()</name> + <fsummary>Return the pid of the socket process.</fsummary> + <type> + <v>Socket = sslsocket()</v> + </type> + <desc> + <p>Returns the pid of the socket process. The returned pid should + only be used for receiving exit messages.</p> + </desc> + </func> + <func> + <name>recv(Socket, Length) -> {ok, Data} | {error, Reason}</name> + <name>recv(Socket, Length, Timeout) -> {ok, Data} | {error, Reason}</name> + <fsummary>Receive data on socket.</fsummary> + <type> + <v>Socket = sslsocket()</v> + <v>Length = integer() >= 0</v> + <v>Timeout = integer()</v> + <v>Data = bytes() | binary()</v> + </type> + <desc> + <p>Receives data on socket <c>Socket</c> when the socket is in + passive mode, i.e. when the option <c>{active, false}</c> + has been specified. + </p> + <p>A notable return value is <c>{error, closed}</c> which + indicates that the socket is closed. + </p> + <p>A positive value of the <c>Length</c> argument is only + valid when the socket is in raw mode (option <c>{packet, 0}</c> is set, and the option <c>binary</c> is <em>not</em> + set); otherwise it should be set to 0, whence all available + bytes are returned. + </p> + <p>If the optional <c>Timeout</c> parameter is specified, and + no data was available within the given time, <c>{error, timeout}</c> is returned. The default value for + <c>Timeout</c> is <c>infinity</c>.</p> + </desc> + </func> + <func> + <name>seed(Data) -> ok | {error, Reason}</name> + <fsummary>Seed the ssl random generator.</fsummary> + <type> + <v>Data = iolist() | binary()</v> + </type> + <desc> + <p>Seeds the ssl random generator. + </p> + <p>It is strongly advised to seed the random generator after + the ssl application has been started, and before any + connections are established. Although the port program + interfacing to the OpenSSL libraries does a "random" seeding + of its own in order to make everything work properly, that + seeding is by no means random for the world since it has a + constant value which is known to everyone reading the source + code of the seeding. + </p> + <p>A notable return value is <c>{error, edata}}</c> indicating that + <c>Data</c> was not a binary nor an iolist.</p> + </desc> + </func> + <func> + <name>send(Socket, Data) -> ok | {error, Reason}</name> + <fsummary>Write data to a socket.</fsummary> + <type> + <v>Socket = sslsocket()</v> + <v>Data = iolist() | binary()</v> + </type> + <desc> + <p>Writes <c>Data</c> to <c>Socket</c>. </p> + <p>A notable return value is <c>{error, closed}</c> indicating that + the socket is closed.</p> + </desc> + </func> + <func> + <name>setopts(Socket, Options) -> ok | {error, Reason}</name> + <fsummary>Set socket options.</fsummary> + <type> + <v>Socket = sslsocket()</v> + <v>Options = [socketoption]()</v> + </type> + <desc> + <p>Sets options according to <c>Options</c> for the socket + <c>Socket</c>. </p> + </desc> + </func> + <func> + <name>ssl_accept(Socket) -> ok | {error, Reason}</name> + <name>ssl_accept(Socket, Timeout) -> ok | {error, Reason}</name> + <fsummary>Perform server-side SSL handshake and key exchange</fsummary> + <type> + <v>Socket = sslsocket()</v> + <v>Timeout = integer()</v> + <v>Reason = atom()</v> + </type> + <desc> + <p>The <c>ssl_accept</c> function establish the SSL connection + on the server side. It should be called directly after + <c>transport_accept</c>, in the spawned server-loop.</p> + <p>Note that the ssl connection is not complete until <c>ssl_accept</c> + has returned <c>true</c>, and if an error is returned, the socket + is unavailable and for instance <c>close/1</c> will crash.</p> + </desc> + </func> + <func> + <name>sockname(Socket) -> {ok, {Address, Port}} | {error, Reason}</name> + <fsummary>Return the local address and port.</fsummary> + <type> + <v>Socket = sslsocket()</v> + <v>Address = ipaddress()</v> + <v>Port = integer()</v> + </type> + <desc> + <p>Returns the local address and port number of the socket + <c>Socket</c>.</p> + </desc> + </func> + <func> + <name>transport_accept(Socket) -> {ok, NewSocket} | {error, Reason}</name> + <name>transport_accept(Socket, Timeout) -> {ok, NewSocket} | {error, Reason}</name> + <fsummary>Accept an incoming connection and prepare for <c>ssl_accept</c></fsummary> + <type> + <v>Socket = NewSocket = sslsocket()</v> + <v>Timeout = integer()</v> + <v>Reason = atom()</v> + </type> + <desc> + <p>Accepts an incoming connection request on a listen socket. + <c>ListenSocket</c> must be a socket returned from <c>listen/2</c>. + The socket returned should be passed to <c>ssl_accept</c> to + complete ssl handshaking and establishing the connection.</p> + <warning> + <p>The socket returned can only be used with <c>ssl_accept</c>, + no traffic can be sent or received before that call.</p> + </warning> + <p>The accepted socket inherits the options set for <c>ListenSocket</c> + in <c>listen/2</c>.</p> + <p>The default value for <c>Timeout</c> is <c>infinity</c>. If + <c>Timeout</c> is specified, and no connection is accepted within + the given time, <c>{error, timeout}</c> is returned.</p> + </desc> + </func> + <func> + <name>version() -> {ok, {SSLVsn, CompVsn, LibVsn}}</name> + <fsummary>Return the version of SSL.</fsummary> + <type> + <v>SSLVsn = CompVsn = LibVsn = string()()</v> + </type> + <desc> + <p>Returns the SSL application version (<c>SSLVsn</c>), the library + version used when compiling the SSL application port program + (<c>CompVsn</c>), and the actual library version used when + dynamically linking in runtime (<c>LibVsn</c>). + </p> + <p>If the SSL application has not been started, <c>CompVsn</c> and + <c>LibVsn</c> are empty strings. + </p> + </desc> + </func> + </funcs> + + <section> + <title>ERRORS</title> + <p>The possible error reasons and the corresponding diagnostic strings + returned by <c>format_error/1</c> are either the same as those defined + in the <c>inet(3)</c> reference manual, or as follows: + </p> + <taglist> + <tag><c>closed</c></tag> + <item> + <p>Connection closed for the operation in question. + </p> + </item> + <tag><c>ebadsocket</c></tag> + <item> + <p>Connection not found (internal error). + </p> + </item> + <tag><c>ebadstate</c></tag> + <item> + <p>Connection not in connect state (internal error). + </p> + </item> + <tag><c>ebrokertype</c></tag> + <item> + <p>Wrong broker type (internal error). + </p> + </item> + <tag><c>ecacertfile</c></tag> + <item> + <p>Own CA certificate file is invalid. + </p> + </item> + <tag><c>ecertfile</c></tag> + <item> + <p>Own certificate file is invalid. + </p> + </item> + <tag><c>echaintoolong</c></tag> + <item> + <p>The chain of certificates provided by peer is too long. + </p> + </item> + <tag><c>ecipher</c></tag> + <item> + <p>Own list of specified ciphers is invalid. + </p> + </item> + <tag><c>ekeyfile</c></tag> + <item> + <p>Own private key file is invalid. + </p> + </item> + <tag><c>ekeymismatch</c></tag> + <item> + <p>Own private key does not match own certificate. + </p> + </item> + <tag><c>enoissuercert</c></tag> + <item> + <p>Cannot find certificate of issuer of certificate provided + by peer. + </p> + </item> + <tag><c>enoservercert</c></tag> + <item> + <p>Attempt to do accept without having set own certificate. + </p> + </item> + <tag><c>enotlistener</c></tag> + <item> + <p>Attempt to accept on a non-listening socket. + </p> + </item> + <tag><c>enoproxysocket</c></tag> + <item> + <p>No proxy socket found (internal error). + </p> + </item> + <tag><c>enooptions</c></tag> + <item> + <p>The list of options is empty. + </p> + </item> + <tag><c>enotstarted</c></tag> + <item> + <p>The SSL application has not been started. + </p> + </item> + <tag><c>eoptions</c></tag> + <item> + <p>Invalid list of options. + </p> + </item> + <tag><c>epeercert</c></tag> + <item> + <p>Certificate provided by peer is in error. + </p> + </item> + <tag><c>epeercertexpired</c></tag> + <item> + <p>Certificate provided by peer has expired. + </p> + </item> + <tag><c>epeercertinvalid</c></tag> + <item> + <p>Certificate provided by peer is invalid. + </p> + </item> + <tag><c>eselfsignedcert</c></tag> + <item> + <p>Certificate provided by peer is self signed. + </p> + </item> + <tag><c>esslaccept</c></tag> + <item> + <p>Server SSL handshake procedure between client and server failed. + </p> + </item> + <tag><c>esslconnect</c></tag> + <item> + <p>Client SSL handshake procedure between client and server failed. + </p> + </item> + <tag><c>esslerrssl</c></tag> + <item> + <p>SSL protocol failure. Typically because of a fatal alert + from peer. + </p> + </item> + <tag><c>ewantconnect</c></tag> + <item> + <p>Protocol wants to connect, which is not supported in + this version of the SSL application. + </p> + </item> + <tag><c>ex509lookup</c></tag> + <item> + <p>Protocol wants X.509 lookup, which is not supported in + this version of the SSL application. + </p> + </item> + <tag><c>{badcall, Call}</c></tag> + <item> + <p>Call not recognized for current mode (active or passive) and + state of socket. + </p> + </item> + <tag><c>{badcast, Cast}</c></tag> + <item> + <p>Call not recognized for current mode (active or passive) and + state of socket. + </p> + </item> + <tag><c>{badinfo, Info}</c></tag> + <item> + <p>Call not recognized for current mode (active or passive) and + state of socket. + </p> + </item> + </taglist> + </section> + + <section> + <title>SEE ALSO</title> + <p>gen_tcp(3), inet(3) public_key(3) </p> + </section> + +</erlref> + + diff --git a/lib/ssl/doc/src/refman.xml b/lib/ssl/doc/src/refman.xml index 3ad5a01b46..68f84660f3 100644 --- a/lib/ssl/doc/src/refman.xml +++ b/lib/ssl/doc/src/refman.xml @@ -4,7 +4,7 @@ <application xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>1999</year><year>2009</year> + <year>1999</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>SSL Reference Manual</title> @@ -45,7 +45,8 @@ </description> <xi:include href="ssl_app.xml"/> <xi:include href="ssl.xml"/> - <xi:include href="new_ssl.xml"/> + <xi:include href="old_ssl.xml"/> + <xi:include href="ssl_session_cache_api.xml"/> </application> diff --git a/lib/ssl/doc/src/ssl.xml b/lib/ssl/doc/src/ssl.xml index 217eb791d0..def61bcf03 100644 --- a/lib/ssl/doc/src/ssl.xml +++ b/lib/ssl/doc/src/ssl.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1999</year><year>2009</year> + <year>1999</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,355 +13,429 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - - </legalnotice> + </legalnotice> <title>ssl</title> - <prepared>Peter Högfeldt</prepared> - <responsible>Peter Högfeldt</responsible> - <docno></docno> - <approved>Peter Högfeldt</approved> - <checked></checked> - <date>2003-03-25</date> - <rev>D</rev> - <file>ssl.sgml</file> + <file>ssl.xml</file> </header> <module>ssl</module> <modulesummary>Interface Functions for Secure Socket Layer</modulesummary> <description> - <p>This module contains interface functions to the Secure Socket Layer.</p> + <p>This module contains interface functions to the Secure Socket + Layer. + </p> </description> + + <section> + <title>SSL</title> + <list type="bulleted"> + <item>ssl requires the crypto an public_key applications.</item> + <item>Supported SSL/TLS-versions are SSL-3.0 and TLS-1.0 </item> + <item>For security reasons sslv2 is not supported.</item> + <item>Ephemeral Diffie-Hellman cipher suites are supported + but not Diffie Hellman Certificates cipher suites.</item> + <item>Export cipher suites are not supported as the + U.S. lifted its export restrictions in early 2000.</item> + <item>CRL and policy certificate + extensions are not supported yet. </item> + </list> + + </section> + <section> - <title>General</title> + <title>COMMON DATA TYPES</title> + <p>The following data types are used in the functions below: + </p> - <p>There is a new implementation of ssl available in - this module but until it is 100 % complete, so that it can replace - the old implementation in all aspects it will be - described here <seealso marker="new_ssl"> new ssl API </seealso></p> + <p><c>boolean() = true | false</c></p> - <p>The reader is advised to also read the <c>ssl(6)</c> manual page - describing the SSL application. - </p> - <warning> - <p>It is strongly advised to seed the random generator after - the ssl application has been started (see <c>seed/1</c> - below), and before any connections are established. Although - the port program interfacing to the ssl libraries does a - "random" seeding of its own in order to make everything work - properly, that seeding is by no means random for the world - since it has a constant value which is known to everyone - reading the source code of the port program.</p> - </warning> - </section> + <p><c>property() = atom()</c></p> + + <p><c>option() = socketoption() | ssloption() | transportoption()</c></p> - <section> - <title>Common data types</title> - <p>The following datatypes are used in the functions below: - </p> - <list type="bulleted"> - <item> - <p><c>options() = [option()]</c></p> - </item> - <item> - <p><c>option() = socketoption() | ssloption()</c></p> - </item> - <item> - <p><c>socketoption() = {mode, list} | {mode, binary} | binary | {packet, packettype()} | {header, integer()} | {nodelay, boolean()} | {active, activetype()} | {backlog, integer()} | {ip, ipaddress()} | {port, integer()}</c></p> - </item> - <item> - <p><c>ssloption() = {verify, code()} | {depth, depth()} | {certfile, path()} | {keyfile, path()} | {password, string()} | {cacertfile, path()} | {ciphers, string()}</c></p> - </item> - <item> - <p><c>packettype()</c> (see inet(3))</p> - </item> - <item> - <p><c>activetype()</c> (see inet(3))</p> - </item> - <item> - <p><c>reason() = atom() | {atom(), string()}</c></p> - </item> - <item> - <p><c>bytes() = [byte()]</c></p> - </item> - <item> - <p><c>string() = [byte()]</c></p> - </item> - <item> - <p><c>byte() = 0 | 1 | 2 | ... | 255</c></p> - </item> - <item> - <p><c>code() = 0 | 1 | 2</c></p> - </item> - <item> - <p><c>depth() = byte()</c></p> - </item> - <item> - <p><c>address() = hostname() | ipstring() | ipaddress()</c></p> - </item> - <item> - <p><c>ipaddress() = ipstring() | iptuple()</c></p> - </item> - <item> - <p><c>hostname() = string()</c></p> - </item> - <item> - <p><c>ipstring() = string()</c></p> - </item> - <item> - <p><c>iptuple() = {byte(), byte(), byte(), byte()}</c></p> - </item> - <item> - <p><c>sslsocket()</c></p> - </item> - <item> - <p><c>protocol() = sslv2 | sslv3 | tlsv1</c></p> + <p><c>socketoption() = [{property(), term()}] - defaults to + [{mode,list},{packet, 0},{header, 0},{active, true}]. + </c></p> + + <p>For valid options + see <seealso marker="kernel:inet">inet(3) </seealso> and + <seealso marker="kernel:gen_tcp">gen_tcp(3) </seealso>. + </p> + + <p> <c>ssloption() = {verify, verify_type()} | + {fail_if_no_peer_cert, boolean()} + {depth, integer()} | + {certfile, path()} | {keyfile, path()} | {password, string()} | + {cacertfile, path()} | {dhfile, path()} | {ciphers, ciphers()} | + {ssl_imp, ssl_imp()} | {reuse_sessions, boolean()} | {reuse_session, fun()} + </c></p> + + <p><c>transportoption() = {CallbackModule, DataTag, ClosedTag} + - defaults to {gen_tcp, tcp, tcp_closed}. Ssl may be + run over any reliable transport protocol that has + an equivalent API to gen_tcp's.</c></p> + + <p><c> CallbackModule = + atom()</c> + </p> <p><c> DataTag = + atom() - tag used in socket data message.</c></p> + <p><c> ClosedTag = atom() - tag used in + socket close message.</c></p> + + <p><c>verify_type() = verify_none | verify_peer</c></p> + + <p><c>path() = string() - representing a file path.</c></p> + + <p><c>host() = hostname() | ipaddress()</c></p> + + <p><c>hostname() = string()</c></p> + + <p><c> + ip_address() = {N1,N2,N3,N4} % IPv4 + | {K1,K2,K3,K4,K5,K6,K7,K8} % IPv6 </c></p> + + <p><c>sslsocket() - opaque to the user. </c></p> + + <p><c>protocol() = sslv3 | tlsv1 </c></p> + + <p><c>ciphers() = [ciphersuite()] | string() (according to old API)</c></p> + + <p><c>ciphersuite() = + {key_exchange(), cipher(), hash()}</c></p> + + <p><c>key_exchange() = rsa | dhe_dss | dhe_rsa + </c></p> + + <p><c>cipher() = rc4_128 | des_cbc | '3des_ede_cbc' + | aes_128_cbc | aes_256_cbc </c></p> + + <p> <c>hash() = md5 | sha + </c></p> + + <p><c>ssl_imp() = new | old - default is new.</c></p> + + </section> + +<section> + <title>SSL OPTION DESCRIPTIONS</title> + + <taglist> + <tag>{verify, verify_type()}</tag> + <item> If <c>verify_none</c> is specified x509-certificate + path validation errors at the client side + will not automatically cause the connection to fail, as + it will if the verify type is <c>verify_peer</c>. See also + the option verify_fun. + Servers only do the path validation if <c>verify_peer</c> is set to + true, as it then will + send a certificate request to + the client (this message is not sent if the verify option is + <c>verify_none</c>) and you may then also want to specify + the option <c>fail_if_no_peer_cert</c>. </item> - <item> - <p><c></c></p> + + <tag>{fail_if_no_peer_cert, boolean()}</tag> + <item>Used together with {verify, verify_peer} by a ssl server. + If set to true, + the server will fail if the client does not have a certificate + to send, e.i sends a empty certificate, if set to false it will + only fail if the client sends a invalid certificate (an empty + certificate is considered valid). + </item> + + <tag>{verify_fun, fun(ErrorList) -> boolean()}</tag> + <item>Used by the ssl client to determine if + x509-certificate path validations errors are acceptable or + if the connection should fail. Defaults to: + +<code> +fun(ErrorList) -> + case lists:foldl(fun({bad_cert,unknown_ca}, Acc) -> + Acc; + (Other, Acc) -> + [Other | Acc] + end, [], ErrorList) of + [] -> + true; + [_|_] -> + false + end +end +</code> + I.e. by default if the only error found was that the CA-certificate + holder was unknown this will be accepted. + + Possible errors in the error list are: + {bad_cert, cert_expired}, {bad_cert, invalid_issuer}, + {bad_cert, invalid_signature}, {bad_cert, name_not_permitted}, + {bad_cert, unknown_ca}, + {bad_cert, cert_expired}, {bad_cert, invalid_issuer}, + {bad_cert, invalid_signature}, {bad_cert, name_not_permitted}, + {bad_cert, cert_revoked} (not implemented yet), + {bad_cert, unknown_critical_extension} or {bad_cert, term()} </item> - </list> - <p>The socket option <c>{backlog, integer()}</c> is for - <c>listen/2</c> only, and the option <c>{port, integer()}</c> - is for <c>connect/3/4</c> only. - </p> - <p>The following socket options are set by default: <c>{mode, list}</c>, <c>{packet, 0}</c>, <c>{header, 0}</c>, <c>{nodelay, false}</c>, <c>{active, true}</c>, <c>{backlog, 5}</c>, - <c>{ip, {0,0,0,0}}</c>, and <c>{port, 0}</c>. - </p> - <p>Note that the options <c>{mode, binary}</c> and <c>binary</c> - are equivalent. Similarly <c>{mode, list}</c> and the absence of - option <c>binary</c> are equivalent. - </p> - <p>The ssl options are for setting specific SSL parameters as follows: - </p> - <list type="bulleted"> - <item> - <p><c>{verify, code()}</c> Specifies type of verification: - 0 = do not verify peer; 1 = verify peer, 2 = verify peer, - fail if no peer certificate. The default value is 0. - </p> + + + <tag>{validate_extensions_fun, fun()}</tag> + <item> + This options makes it possible to supply a fun to validate + possible application specific certificate extensions + during the certificat path validation. This option + will be better documented onec the public_key API is more + mature. + </item> + + <tag>{depth, integer()}</tag> + <item>Specifies the maximum + verification depth, i.e. how far in a chain of certificates the + verification process can proceed before the verification is + considered to fail. Peer certificate = 0, CA certificate = 1, + higher level CA certificate = 2, etc. The value 2 thus means + that a chain can at most contain peer cert, CA cert, next CA + cert, and an additional CA cert. The default value is 1. </item> - <item> - <p><c>{depth, depth()}</c> Specifies the maximum - verification depth, i.e. how far in a chain of certificates - the verification process can proceed before the verification - is considered to fail. - </p> - <p>Peer certificate = 0, CA certificate = 1, higher level CA - certificate = 2, etc. The value 2 thus means that a chain - can at most contain peer cert, CA cert, next CA cert, and an - additional CA cert. - </p> - <p>The default value is 1. - </p> + + <tag>{certfile, path()}</tag> + <item>Path to a file containing the + user's certificate. Optional for clients but note + that some servers requires that the client can certify + itself. </item> + <tag>{keyfile, path()}</tag> + <item>Path to file containing user's + private PEM encoded key. As PEM-files may contain several + entries this option defaults to the same file as given by + certfile option.</item> + <tag>{password, string()}</tag> + <item>String containing the user's password. + Only used if the private keyfile is password protected. + </item> + <tag>{cacertfile, path()}</tag> + <item>Path to file containing PEM encoded + CA certificates (trusted certificates used for verifying a peer + certificate). May be omitted if you do not want to verify + the peer.</item> + + <tag>{dhfile, path()}</tag> + <item>Path to file containing PEM encoded Diffie Hellman parameters, + for the server to use if a cipher suite using Diffie Hellman key exchange + is negotiated. If not specified hardcode parameters will be used. </item> - <item> - <p><c>{certfile, path()}</c> Path to a file containing the - user's certificate. - chain of PEM encoded certificates.</p> + + <tag>{ciphers, ciphers()}</tag> + <item>The function <c>ciphers_suites/0</c> can + be used to find all available ciphers. </item> - <item> - <p><c>{keyfile, path()}</c> Path to file containing user's - private PEM encoded key.</p> + + <tag>{ssl_imp, ssl_imp()}</tag> + <item>Specify which ssl implementation you want to use. Defaults to + new. </item> - <item> - <p><c>{password, string()}</c> String containing the user's - password. Only used if the private keyfile is password protected.</p> + + <tag>{reuse_sessions, boolean()}</tag> + <item>Specifies if ssl sessions should be reused + when possible. </item> - <item> - <p><c>{cacertfile, path()}</c> Path to file containing PEM encoded - CA certificates (trusted certificates used for verifying a peer - certificate).</p> + + <tag>{reuse_session, fun(SuggestedSessionId, + PeerCert, Compression, CipherSuite) -> boolean()}</tag> + <item>Enables the ssl server to have a local policy + for deciding if a session should be reused or not, + only meaning full if <c>reuse_sessions</c> is set to true. + SuggestedSessionId is a binary(), PeerCert is a DER encoded + certificate, Compression is an enumeration integer + and CipherSuite of type ciphersuite(). </item> - <item> - <p><c>{ciphers, string()}</c> String of ciphers as a colon - separated list of ciphers. The function <c>ciphers/0</c> can - be used to find all available ciphers.</p> + + <tag>{secure_renegotiate, boolean()}</tag> + <item>Specifies if to reject renegotiation attempt that does + not live up to RFC 5746. By default secure_renegotiate is + set to false e.i. secure renegotiation will be used if possible + but it will fallback to unsecure renegotiation if the peer + does not support RFC 5746. </item> - </list> - <p>The type <c>sslsocket()</c> is opaque to the user. - </p> - <p>The owner of a socket is the one that created it by a call to - <c>transport_accept/[1,2]</c>, <c>connect/[3,4]</c>, - or <c>listen/2</c>. - </p> - <p>When a socket is in active mode (the default), data from the + + </taglist> + </section> + + <section> + <title>General</title> + + <p>When a ssl socket is in active mode (the default), data from the socket is delivered to the owner of the socket in the form of messages: - </p> + </p> <list type="bulleted"> - <item> - <p><c>{ssl, Socket, Data}</c></p> + <item>{ssl, Socket, Data} </item> - <item> - <p><c>{ssl_closed, Socket}</c></p> + <item>{ssl_closed, Socket} </item> <item> - <p><c>{ssl_error, Socket, Reason}</c></p> + {ssl_error, Socket, Reason} </item> </list> + <p>A <c>Timeout</c> argument specifies a timeout in milliseconds. The default value for a <c>Timeout</c> argument is <c>infinity</c>. - </p> - <p>Functions listed below may return the value <c>{error, closed}</c>, which only indicates that the SSL socket is - considered closed for the operation in question. It is for - instance possible to have <c>{error, closed}</c> returned from - an call to <c>send/2</c>, and a subsequent call to <c>recv/3</c> - returning <c>{ok, Data}</c>. - </p> - <p>Hence a return value of <c>{error, closed}</c> must not be - interpreted as if the socket was completely closed. On the - contrary, in order to free all resources occupied by an SSL - socket, <c>close/1</c> must be called, or else the process owning - the socket has to terminate. - </p> - <p>For each SSL socket there is an Erlang process representing the - socket. When a socket is opened, that process links to the - calling client process. Implementations that want to detect - abnormal exits from the socket process by receiving <c>{'EXIT', Pid, Reason}</c> messages, should use the function <c>pid/1</c> - to retrieve the process identifier from the socket, in order to - be able to match exit messages properly.</p> + </p> </section> + <funcs> <func> - <name>ciphers() -> {ok, string()} | {error, enotstarted}</name> - <fsummary>Get supported ciphers.</fsummary> - <desc> - <p>Returns a string consisting of colon separated cipher - designations that are supported by the current SSL library - implementation. - </p> - <p>The SSL application has to be started to return the string - of ciphers.</p> - </desc> + <name>cipher_suites() -></name> + <name>cipher_suites(Type) -> ciphers()</name> + <fsummary> Returns a list of supported cipher suites</fsummary> + <type> + <v>Type = erlang | openssl</v> + + </type> + <desc><p>Returns a list of supported cipher suites. + cipher_suites() is equivalent to cipher_suites(erlang). + Type openssl is provided for backwards compatibility with + old ssl that used openssl. + </p> + </desc> </func> + <func> - <name>close(Socket) -> ok | {error, Reason}</name> - <fsummary>Close a socket returned by <c>transport_accept/[1,2]</c>, <c>connect/3/4</c>, or <c>listen/2</c>.</fsummary> + <name>connect(Socket, SslOptions) -> </name> + <name>connect(Socket, SslOptions, Timeout) -> {ok, SslSocket} + | {error, Reason}</name> + <fsummary> Upgrades a gen_tcp, or + equivalent, connected socket to a ssl socket. </fsummary> <type> - <v>Socket = sslsocket()</v> + <v>Socket = socket()</v> + <v>SslOptions = [ssloption()]</v> + <v>Timeout = integer() | infinity</v> + <v>SslSocket = sslsocket()</v> + <v>Reason = term()</v> </type> - <desc> - <p>Closes a socket returned by <c>transport_accept/[1,2]</c>, - <c>connect/[3,4]</c>, or <c>listen/2</c></p> - </desc> + <desc> <p>Upgrades a gen_tcp, or equivalent, + connected socket to a ssl socket e.i performs the + client-side ssl handshake.</p> + </desc> </func> + <func> - <name>connect(Address, Port, Options) -> {ok, Socket} | {error, Reason}</name> - <name>connect(Address, Port, Options, Timeout) -> {ok, Socket} | {error, Reason}</name> - <fsummary>Connect to <c>Port</c>at <c>Address</c>.</fsummary> + <name>connect(Host, Port, Options) -></name> + <name>connect(Host, Port, Options, Timeout) -> + {ok, SslSocket} | {error, Reason}</name> + <fsummary>Opens an ssl connection to Host, Port. </fsummary> <type> - <v>Address = address()</v> - <v>Port = integer()</v> - <v>Options = [connect_option()]</v> - <v>connect_option() = {mode, list} | {mode, binary} | binary | {packet, packettype()} | {header, integer()} | {nodelay, boolean()} | {active, activetype()} | {ip, ipaddress()} | {port, integer()} | {verify, code()} | {depth, depth()} | {certfile, path()} | {keyfile, path()} | {password, string()} | {cacertfile, path()} | {ciphers, string()}</v> - <v>Timeout = integer()</v> - <v>Socket = sslsocket()</v> + <v>Host = host()</v> + <v>Port = integer()</v> + <v>Options = [option()]</v> + <v>Timeout = integer() | infinity</v> + <v>SslSocket = sslsocket()</v> + <v>Reason = term()</v> </type> - <desc> - <p>Connects to <c>Port</c> at <c>Address</c>. If the optional - <c>Timeout</c> argument is specified, and a connection could not - be established within the given time, <c>{error, timeout}</c> is - returned. The default value for <c>Timeout</c> is <c>infinity</c>. - </p> - <p>The <c>ip</c> and <c>port</c> options are for binding to a - particular <em>local</em> address and port, respectively.</p> - </desc> + <desc> <p>Opens an ssl connection to Host, Port.</p> </desc> </func> + <func> - <name>connection_info(Socket) -> {ok, {Protocol, Cipher}} | {error, Reason}</name> - <fsummary>Get current protocol version and cipher.</fsummary> + <name>close(SslSocket) -> ok | {error, Reason}</name> + <fsummary>Close a ssl connection</fsummary> <type> - <v>Socket = sslsocket()</v> - <v>Protocol = protocol()</v> - <v>Cipher = string()</v> + <v>SslSocket = sslsocket()</v> + <v>Reason = term()</v> </type> - <desc> - <p>Gets the chosen protocol version and cipher for an established - connection (accepted och connected). </p> + <desc><p>Close a ssl connection.</p> + </desc> + </func> + + <func> + <name>controlling_process(SslSocket, NewOwner) -> + ok | {error, Reason}</name> + + <fsummary>Assigns a new controlling process to the + ssl-socket.</fsummary> + + <type> + <v>SslSocket = sslsocket()</v> + <v>NewOwner = pid()</v> + <v>Reason = term()</v> + </type> + <desc><p>Assigns a new controlling process to the ssl-socket. A + controlling process is the owner of a ssl-socket, and receives + all messages from the socket.</p> </desc> </func> + <func> - <name>controlling_process(Socket, NewOwner) -> ok | {error, Reason}</name> - <fsummary>Assign a new controlling process to the socket.</fsummary> + <name>connection_info(SslSocket) -> + {ok, {ProtocolVersion, CipherSuite}} | {error, Reason} </name> + <fsummary>Returns the negotiated protocol version and cipher suite. + </fsummary> <type> - <v>Socket = sslsocket()</v> - <v>NewOwner = pid()</v> + <v>CipherSuite = ciphersuite()</v> + <v>ProtocolVersion = protocol()</v> </type> - <desc> - <p>Assigns a new controlling process to <c>Socket</c>. A controlling - process is the owner of a socket, and receives all messages from - the socket.</p> + <desc><p>Returns the negotiated protocol version and cipher suite.</p> </desc> </func> - <func> - <name>format_error(ErrorCode) -> string()</name> + + <func> + <name>format_error(Reason) -> string()</name> <fsummary>Return an error string.</fsummary> <type> - <v>ErrorCode = term()</v> + <v>Reason = term()</v> </type> <desc> - <p>Returns a diagnostic string describing an error.</p> + <p>Presents the error returned by an ssl function as a printable string.</p> </desc> </func> + <func> - <name>getopts(Socket, OptionsTags) -> {ok, Options} | {error, Reason}</name> - <fsummary>Get options set for socket</fsummary> + <name>getopts(Socket) -> </name> + <name>getopts(Socket, OptionNames) -> + {ok, [socketoption()]} | {error, Reason}</name> + <fsummary>Get the value of the specified options.</fsummary> <type> - <v>Socket = sslsocket()</v> - <v>OptionTags = [optiontag()]()</v> + <v>Socket = sslsocket()</v> + <v>OptionNames = [property()]</v> </type> <desc> - <p>Returns the options the tags of which are <c>OptionTags</c> for - for the socket <c>Socket</c>. </p> + <p>Get the value of the specified socket options, if no + options are specified all options are returned. + </p> </desc> </func> + <func> - <name>listen(Port, Options) -> {ok, ListenSocket} | {error, Reason}</name> - <fsummary>Set up a socket to listen on a port on the local host.</fsummary> + <name>listen(Port, Options) -> + {ok, ListenSocket} | {error, Reason}</name> + <fsummary>Creates a ssl listen socket.</fsummary> <type> - <v>Port = integer()</v> - <v>Options = [listen_option()]</v> - <v>listen_option() = {mode, list} | {mode, binary} | binary | {packet, packettype()} | {header, integer()} | {active, activetype()} | {backlog, integer()} | {ip, ipaddress()} | {verify, code()} | {depth, depth()} | {certfile, path()} | {keyfile, path()} | {password, string()} | {cacertfile, path()} | {ciphers, string()}</v> - <v>ListenSocket = sslsocket()</v> + <v>Port = integer()</v> + <v>Options = options()</v> + <v>ListenSocket = sslsocket()</v> </type> <desc> - <p>Sets up a socket to listen on port <c>Port</c> at the local host. - If <c>Port</c> is zero, <c>listen/2</c> picks an available port - number (use <c>port/1</c> to retrieve it). - </p> - <p>The listen queue size defaults to 5. If a different value is - wanted, the option <c>{backlog, Size}</c> should be added to the - list of options. - </p> - <p>An empty <c>Options</c> list is considered an error, and - <c>{error, enooptions}</c> is returned. - </p> - <p>The returned <c>ListenSocket</c> can only be used in calls to - <c>transport_accept/[1,2]</c>.</p> + <p>Creates a ssl listen socket.</p> </desc> </func> + <func> - <name>peercert(Socket) -> {ok, Cert} | {error, Reason}</name> + <name>peercert(Socket) -> {ok, Cert} | {error, Reason}</name> <fsummary>Return the peer certificate.</fsummary> - <type> + <type> <v>Socket = sslsocket()</v> - <v>Cert = binary()()</v> - <v>Subject = term()()</v> + <v>Cert = binary()</v> </type> <desc> - <p>Returns the DER encoded peer certificate, the certificate can be decoded with - <c>public_key:pkix_decode_cert/2</c>. - </p> + <p>The peer certificate is returned as a DER encoded binary. + The certificate can be decoded with <c>public_key:pkix_decode_cert/2</c>. + </p> </desc> </func> <func> - <name>peername(Socket) -> {ok, {Address, Port}} | {error, Reason}</name> + <name>peername(Socket) -> {ok, {Address, Port}} | + {error, Reason}</name> <fsummary>Return peer address and port.</fsummary> <type> <v>Socket = sslsocket()</v> @@ -372,67 +446,47 @@ <p>Returns the address and port number of the peer.</p> </desc> </func> + <func> - <name>pid(Socket) -> pid()</name> - <fsummary>Return the pid of the socket process.</fsummary> - <type> - <v>Socket = sslsocket()</v> - </type> - <desc> - <p>Returns the pid of the socket process. The returned pid should - only be used for receiving exit messages.</p> - </desc> - </func> - <func> - <name>recv(Socket, Length) -> {ok, Data} | {error, Reason}</name> - <name>recv(Socket, Length, Timeout) -> {ok, Data} | {error, Reason}</name> - <fsummary>Receive data on socket.</fsummary> + <name>recv(Socket, Length) -> </name> + <name>recv(Socket, Length, Timeout) -> {ok, Data} | {error, + Reason}</name> + <fsummary>Receive data on a socket.</fsummary> <type> <v>Socket = sslsocket()</v> - <v>Length = integer() >= 0</v> + <v>Length = integer()</v> <v>Timeout = integer()</v> - <v>Data = bytes() | binary()</v> + <v>Data = [char()] | binary()</v> </type> <desc> - <p>Receives data on socket <c>Socket</c> when the socket is in - passive mode, i.e. when the option <c>{active, false}</c> - has been specified. - </p> - <p>A notable return value is <c>{error, closed}</c> which - indicates that the socket is closed. - </p> - <p>A positive value of the <c>Length</c> argument is only - valid when the socket is in raw mode (option <c>{packet, 0}</c> is set, and the option <c>binary</c> is <em>not</em> - set); otherwise it should be set to 0, whence all available - bytes are returned. - </p> - <p>If the optional <c>Timeout</c> parameter is specified, and - no data was available within the given time, <c>{error, timeout}</c> is returned. The default value for - <c>Timeout</c> is <c>infinity</c>.</p> + <p>This function receives a packet from a socket in passive + mode. A closed socket is indicated by a return value + <c>{error, closed}</c>.</p> + <p>The <c>Length</c> argument is only meaningful when + the socket is in <c>raw</c> mode and denotes the number of + bytes to read. If <c>Length</c> = 0, all available bytes are + returned. If <c>Length</c> > 0, exactly <c>Length</c> + bytes are returned, or an error; possibly discarding less + than <c>Length</c> bytes of data when the socket gets closed + from the other side.</p> + <p>The optional <c>Timeout</c> parameter specifies a timeout in + milliseconds. The default value is <c>infinity</c>.</p> </desc> </func> + <func> - <name>seed(Data) -> ok | {error, Reason}</name> - <fsummary>Seed the ssl random generator.</fsummary> + <name>renegotiate(Socket) -> ok | {error, Reason}</name> + <fsummary> Initiates a new handshake.</fsummary> <type> - <v>Data = iolist() | binary()</v> + <v>Socket = sslsocket()</v> </type> - <desc> - <p>Seeds the ssl random generator. - </p> - <p>It is strongly advised to seed the random generator after - the ssl application has been started, and before any - connections are established. Although the port program - interfacing to the OpenSSL libraries does a "random" seeding - of its own in order to make everything work properly, that - seeding is by no means random for the world since it has a - constant value which is known to everyone reading the source - code of the seeding. - </p> - <p>A notable return value is <c>{error, edata}}</c> indicating that - <c>Data</c> was not a binary nor an iolist.</p> + <desc><p>Initiates a new handshake. A notable return value is + <c>{error, renegotiation_rejected}</c> indicating that the peer + refused to go through with the renegotiation but the connection + is still active using the previously negotiated session.</p> </desc> </func> + <func> <name>send(Socket, Data) -> ok | {error, Reason}</name> <fsummary>Write data to a socket.</fsummary> @@ -458,26 +512,65 @@ <c>Socket</c>. </p> </desc> </func> + <func> - <name>ssl_accept(Socket) -> ok | {error, Reason}</name> - <name>ssl_accept(Socket, Timeout) -> ok | {error, Reason}</name> - <fsummary>Perform server-side SSL handshake and key exchange</fsummary> + <name>shutdown(Socket, How) -> ok | {error, Reason}</name> + <fsummary>Immediately close a socket</fsummary> <type> <v>Socket = sslsocket()</v> + <v>How = read | write | read_write</v> + <v>Reason = reason()</v> + </type> + <desc> + <p>Immediately close a socket in one or two directions.</p> + <p><c>How == write</c> means closing the socket for writing, + reading from it is still possible.</p> + <p>To be able to handle that the peer has done a shutdown on + the write side, the <c>{exit_on_close, false}</c> option + is useful.</p> + </desc> + </func> + + <func> + <name>ssl_accept(ListenSocket) -> </name> + <name>ssl_accept(ListenSocket, Timeout) -> ok | {error, Reason}</name> + <fsummary>Perform server-side SSL handshake</fsummary> + <type> + <v>ListenSocket = sslsocket()</v> <v>Timeout = integer()</v> - <v>Reason = atom()</v> + <v>Reason = term()</v> </type> <desc> <p>The <c>ssl_accept</c> function establish the SSL connection on the server side. It should be called directly after <c>transport_accept</c>, in the spawned server-loop.</p> - <p>Note that the ssl connection is not complete until <c>ssl_accept</c> - has returned <c>true</c>, and if an error is returned, the socket - is unavailable and for instance <c>close/1</c> will crash.</p> </desc> </func> + + <func> + <name>ssl_accept(ListenSocket, SslOptions) -> </name> + <name>ssl_accept(ListenSocket, SslOptions, Timeout) -> {ok, Socket} | {error, Reason}</name> + <fsummary>Perform server-side SSL handshake</fsummary> + <type> + <v>ListenSocket = socket()</v> + <v>SslOptions = ssloptions()</v> + <v>Timeout = integer()</v> + <v>Reason = term()</v> + </type> + <desc> + <p> Upgrades a gen_tcp, or + equivalent, socket to a ssl socket e.i performs the + ssl server-side handshake.</p> + <p><note>Note that the listen socket should be in {active, false} mode + before telling the client that the server is ready to upgrade + and calling this function, otherwise the upgrade may + or may not succeed depending on timing.</note></p> + </desc> + </func> + <func> - <name>sockname(Socket) -> {ok, {Address, Port}} | {error, Reason}</name> + <name>sockname(Socket) -> {ok, {Address, Port}} | + {error, Reason}</name> <fsummary>Return the local address and port.</fsummary> <type> <v>Socket = sslsocket()</v> @@ -489,217 +582,84 @@ <c>Socket</c>.</p> </desc> </func> + <func> - <name>transport_accept(Socket) -> {ok, NewSocket} | {error, Reason}</name> - <name>transport_accept(Socket, Timeout) -> {ok, NewSocket} | {error, Reason}</name> - <fsummary>Accept an incoming connection and prepare for <c>ssl_accept</c></fsummary> + <name>start() -> </name> + <name>start(Type) -> ok | {error, Reason}</name> + <fsummary>Starts the Ssl application. </fsummary> + <type> + <v>Type = permanent | transient | temporary</v> + </type> + <desc> + <p>Starts the Ssl application. Default type + is temporary. + <seealso marker="kernel:application">application(3)</seealso></p> + </desc> + </func> + <func> + <name>stop() -> ok </name> + <fsummary>Stops the Ssl application.</fsummary> + <desc> + <p>Stops the Ssl application. + <seealso marker="kernel:application">application(3)</seealso></p> + </desc> + </func> + + <func> + <name>transport_accept(Socket) -></name> + <name>transport_accept(Socket, Timeout) -> + {ok, NewSocket} | {error, Reason}</name> + <fsummary>Accept an incoming connection and + prepare for <c>ssl_accept</c></fsummary> <type> <v>Socket = NewSocket = sslsocket()</v> <v>Timeout = integer()</v> - <v>Reason = atom()</v> + <v>Reason = reason()</v> </type> <desc> <p>Accepts an incoming connection request on a listen socket. - <c>ListenSocket</c> must be a socket returned from <c>listen/2</c>. - The socket returned should be passed to <c>ssl_accept</c> to - complete ssl handshaking and establishing the connection.</p> + <c>ListenSocket</c> must be a socket returned from + <c>listen/2</c>. The socket returned should be passed to + <c>ssl_accept</c> to complete ssl handshaking and + establishing the connection.</p> <warning> <p>The socket returned can only be used with <c>ssl_accept</c>, no traffic can be sent or received before that call.</p> </warning> - <p>The accepted socket inherits the options set for <c>ListenSocket</c> - in <c>listen/2</c>.</p> - <p>The default value for <c>Timeout</c> is <c>infinity</c>. If - <c>Timeout</c> is specified, and no connection is accepted within - the given time, <c>{error, timeout}</c> is returned.</p> + <p>The accepted socket inherits the options set for + <c>ListenSocket</c> in <c>listen/2</c>.</p> + <p>The default + value for <c>Timeout</c> is <c>infinity</c>. If + <c>Timeout</c> is specified, and no connection is accepted + within the given time, <c>{error, timeout}</c> is + returned.</p> </desc> </func> + <func> - <name>version() -> {ok, {SSLVsn, CompVsn, LibVsn}}</name> - <fsummary>Return the version of SSL.</fsummary> + <name>versions() -> + [{SslAppVer, SupportedSslVer, AvailableSslVsn}]</name> + <fsummary>Returns version information relevant for the + ssl application.</fsummary> <type> - <v>SSLVsn = CompVsn = LibVsn = string()()</v> + <v>SslAppVer = string()</v> + <v>SupportedSslVer = [protocol()]</v> + <v>AvailableSslVsn = [protocol()]</v> </type> <desc> - <p>Returns the SSL application version (<c>SSLVsn</c>), the library - version used when compiling the SSL application port program - (<c>CompVsn</c>), and the actual library version used when - dynamically linking in runtime (<c>LibVsn</c>). - </p> - <p>If the SSL application has not been started, <c>CompVsn</c> and - <c>LibVsn</c> are empty strings. - </p> + <p> + Returns version information relevant for the + ssl application.</p> </desc> </func> - </funcs> - - <section> - <title>ERRORS</title> - <p>The possible error reasons and the corresponding diagnostic strings - returned by <c>format_error/1</c> are either the same as those defined - in the <c>inet(3)</c> reference manual, or as follows: - </p> - <taglist> - <tag><c>closed</c></tag> - <item> - <p>Connection closed for the operation in question. - </p> - </item> - <tag><c>ebadsocket</c></tag> - <item> - <p>Connection not found (internal error). - </p> - </item> - <tag><c>ebadstate</c></tag> - <item> - <p>Connection not in connect state (internal error). - </p> - </item> - <tag><c>ebrokertype</c></tag> - <item> - <p>Wrong broker type (internal error). - </p> - </item> - <tag><c>ecacertfile</c></tag> - <item> - <p>Own CA certificate file is invalid. - </p> - </item> - <tag><c>ecertfile</c></tag> - <item> - <p>Own certificate file is invalid. - </p> - </item> - <tag><c>echaintoolong</c></tag> - <item> - <p>The chain of certificates provided by peer is too long. - </p> - </item> - <tag><c>ecipher</c></tag> - <item> - <p>Own list of specified ciphers is invalid. - </p> - </item> - <tag><c>ekeyfile</c></tag> - <item> - <p>Own private key file is invalid. - </p> - </item> - <tag><c>ekeymismatch</c></tag> - <item> - <p>Own private key does not match own certificate. - </p> - </item> - <tag><c>enoissuercert</c></tag> - <item> - <p>Cannot find certificate of issuer of certificate provided - by peer. - </p> - </item> - <tag><c>enoservercert</c></tag> - <item> - <p>Attempt to do accept without having set own certificate. - </p> - </item> - <tag><c>enotlistener</c></tag> - <item> - <p>Attempt to accept on a non-listening socket. - </p> - </item> - <tag><c>enoproxysocket</c></tag> - <item> - <p>No proxy socket found (internal error). - </p> - </item> - <tag><c>enooptions</c></tag> - <item> - <p>The list of options is empty. - </p> - </item> - <tag><c>enotstarted</c></tag> - <item> - <p>The SSL application has not been started. - </p> - </item> - <tag><c>eoptions</c></tag> - <item> - <p>Invalid list of options. - </p> - </item> - <tag><c>epeercert</c></tag> - <item> - <p>Certificate provided by peer is in error. - </p> - </item> - <tag><c>epeercertexpired</c></tag> - <item> - <p>Certificate provided by peer has expired. - </p> - </item> - <tag><c>epeercertinvalid</c></tag> - <item> - <p>Certificate provided by peer is invalid. - </p> - </item> - <tag><c>eselfsignedcert</c></tag> - <item> - <p>Certificate provided by peer is self signed. - </p> - </item> - <tag><c>esslaccept</c></tag> - <item> - <p>Server SSL handshake procedure between client and server failed. - </p> - </item> - <tag><c>esslconnect</c></tag> - <item> - <p>Client SSL handshake procedure between client and server failed. - </p> - </item> - <tag><c>esslerrssl</c></tag> - <item> - <p>SSL protocol failure. Typically because of a fatal alert - from peer. - </p> - </item> - <tag><c>ewantconnect</c></tag> - <item> - <p>Protocol wants to connect, which is not supported in - this version of the SSL application. - </p> - </item> - <tag><c>ex509lookup</c></tag> - <item> - <p>Protocol wants X.509 lookup, which is not supported in - this version of the SSL application. - </p> - </item> - <tag><c>{badcall, Call}</c></tag> - <item> - <p>Call not recognized for current mode (active or passive) and - state of socket. - </p> - </item> - <tag><c>{badcast, Cast}</c></tag> - <item> - <p>Call not recognized for current mode (active or passive) and - state of socket. - </p> - </item> - <tag><c>{badinfo, Info}</c></tag> - <item> - <p>Call not recognized for current mode (active or passive) and - state of socket. - </p> - </item> - </taglist> - </section> - + </funcs> + <section> <title>SEE ALSO</title> - <p>gen_tcp(3), inet(3) public_key(3) </p> + <p><seealso marker="kernel:inet">inet(3) </seealso> and + <seealso marker="kernel:gen_tcp">gen_tcp(3) </seealso> + </p> </section> - -</erlref> +</erlref> diff --git a/lib/ssl/doc/src/ssl_app.xml b/lib/ssl/doc/src/ssl_app.xml index ae8bd87781..2ba6f48611 100644 --- a/lib/ssl/doc/src/ssl_app.xml +++ b/lib/ssl/doc/src/ssl_app.xml @@ -4,7 +4,7 @@ <appref> <header> <copyright> - <year>1999</year><year>2009</year> + <year>1999</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,45 +13,20 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>ssl</title> - <prepared>Peter Högfeldt</prepared> - <responsible>Peter Högfeldt</responsible> - <docno></docno> - <approved>Peter Högfeldt</approved> - <checked>Peter Högfeldt</checked> - <date>2005-03-10</date> - <rev>E</rev> <file>ssl_app.sgml</file> </header> <app>ssl</app> - <appsummary>The SSL Application</appsummary> - <description> - <p>The Secure Socket Layer (SSL) application provides secure - socket communication over TCP/IP. - </p> - </description> - - <section> - <title>Warning</title> - <p>In previous versions of Erlang/OTP SSL it was advised, as a - work-around, to set the operating system environment variable - <c>SSL_CERT_FILE</c> to point at a file containing CA - certificates. That variable is no longer needed, and is not - recognised by Erlang/OTP SSL any more. - </p> - <p>However, the OpenSSL package does interpret that environment - variable. Hence a setting of that variable might have - unpredictable effects on the Erlang/OTP SSL application. It is - therefore adviced to not used that environment variable at all.</p> - </section> + <appsummary>The SSL application provides secure communication over + sockets.</appsummary> <section> <title>Environment</title> @@ -61,115 +36,43 @@ </p> <p>Note that the environment parameters can be set on the command line, for instance,</p> - <p><c>erl ... -ssl protocol_version '[sslv2,sslv3]' ...</c>. + <p><c>erl ... -ssl protocol_version '[sslv3, tlsv1]' ...</c>. </p> <taglist> - <tag><c><![CDATA[ephemeral_rsa = true | false <optional>]]></c></tag> - <item> - <p>Enables all SSL servers (those that listen and accept) - to use ephemeral RSA key generation when a clients connect with - weak handshake cipher specifications, that need equally weak - ciphers from the server (i.e. obsolete restrictions on export - ciphers). Default is <c>false</c>. - </p> - </item> - <tag><c><![CDATA[debug = true | false <optional>]]></c></tag> - <item> - <p>Causes debug information to be written to standard - output. Default is <c>false</c>. - </p> - </item> - <tag><c><![CDATA[debugdir = path() | false <optional>]]></c></tag> - <item> - <p>Causes debug information output controlled by <c>debug</c> - and <c>msgdebug</c> to be printed to a file named - <c><![CDATA[ssl_esock.<pid>.log]]></c> in the directory specified by - <c>debugdir</c>, where <c><![CDATA[<pid>]]></c> is the operating system - specific textual representation of the process identifier - of the external port program of the SSL application. Default - is <c>false</c>, i.e. no log file is produced. - </p> - </item> - <tag><c><![CDATA[msgdebug = true | false <optional>]]></c></tag> + <tag><c><![CDATA[protocol_version = [sslv3|tlsv1] <optional>]]></c>.</tag> <item> - <p>Sets <c>debug = true</c> and causes also the contents - of low level messages to be printed to standard output. - Default is <c>false</c>. - </p> + <p>Protocol that will be supported by started clients and + servers. If this option is not set it will default to all + protocols currently supported by the erlang ssl application. + Note that this option may be overridden by the version option + to ssl:connect/[2,3] and ssl:listen/2. + </p> </item> - <tag><c><![CDATA[port_program = string() | false <optional>]]></c></tag> - <item> - <p>Name of port program. The default is <c>ssl_esock</c>. - </p> - </item> - <tag><c><![CDATA[protocol_version = [sslv2|sslv3|tlsv1] <optional>]]></c>.</tag> + + <tag><c><![CDATA[session_lifetime = integer() <optional>]]></c></tag> <item> - <p>Name of protocols to use. If this option is not set, - all protocols are assumed, i.e. the default value is - <c>[sslv2, sslv3, tlsv1]</c>. - </p> + <p>The lifetime of session data in seconds. + </p> </item> - <tag><c><![CDATA[proxylsport = integer() | false <optional>]]></c></tag> + + <tag><c><![CDATA[session_cb = atom() <optional>]]></c></tag> <item> - <p>Define the port number of the listen port of the - SSL port program. Almost never is this option needed. + <p> + Name of session cache callback module that implements + the ssl_session_cache_api behavior, defaults to + ssl_session_cache.erl. </p> </item> - <tag><c><![CDATA[proxylsbacklog = integer() | false <optional>]]></c></tag> + + <tag><c><![CDATA[session_cb_init_args = list() <optional>]]></c></tag> <item> - <p>Set the listen queue size of the listen port of the - SSL port program. The default is 128. - </p> + <p> + List of arguments to the init function in session cache + callback module, defaults to []. + </p> </item> - </taglist> - </section> - <section> - <title>OpenSSL libraries</title> - <p>The current implementation of the Erlang SSL application is - based on the <em>OpenSSL</em> package version 0.9.7 or higher. - There are source and binary releases on the web. - </p> - <p>Source releases of OpenSSL can be downloaded from the <url href="http://www.openssl.org">OpenSSL</url> project home page, - or mirror sites listed there. - </p> - <p>The same URL also contains links to some compiled binaries and - libraries of OpenSSL (see the <c>Related/Binaries</c> menu) of - which the <url href="http://www.shininglightpro.com/search.php?searchname=Win32+OpenSSL">Shining Light Productions Win32 and OpenSSL</url> pages are of - interest for the Win32 user. - </p> - <p>For some Unix flavours there are binary packages available - on the net. - </p> - <p>If you cannot find a suitable binary OpenSSL package, you - have to fetch an OpenSSL source release and compile it. - </p> - <p>You then have to compile and install the libraries - <c>libcrypto.so</c> and <c>libssl.so</c> (Unix), or the - libraries <c>libeay32.dll</c> and <c>ssleay32.dll</c> (Win32). - </p> - <p>For Unix The <c>ssl_esock</c> port program is delivered linked - to OpenSSL libraries in <c>/usr/local/lib</c>, but the default - dynamic linking will also accept libraries in <c>/lib</c> and - <c>/usr/lib</c>. - </p> - <p>If that is not applicable to the particular Unix operating - system used, the example <c>Makefile</c> in the SSL - <c>priv/obj</c> directory, should be used as a guide to - relinking the final version of the port program. - </p> - <p>For <c>Win32</c> it is only required that the libraries can be - found from the <c>PATH</c> environment variable, or that they - reside in the appropriate <c>SYSTEM32</c> directory; hence no - particular relinking is need. Hence no example <c>Makefile</c> - for Win32 is provided.</p> - </section> - - <section> - <title>Restrictions</title> - <p>Users must be aware of export restrictions and patent rights - concerning cryptographic software. - </p> + </taglist> </section> <section> @@ -178,5 +81,3 @@ </section> </appref> - - diff --git a/lib/ssl/doc/src/ssl_distribution.xml b/lib/ssl/doc/src/ssl_distribution.xml index c743cd67a3..4067fb8a22 100644 --- a/lib/ssl/doc/src/ssl_distribution.xml +++ b/lib/ssl/doc/src/ssl_distribution.xml @@ -32,7 +32,13 @@ <file>ssl_distribution.xml</file> </header> <p>This chapter describes how the Erlang distribution can use - SSL to get additional verification and security.</p> + SSL to get additional verification and security. + + <note><p>Note this + documentation is written for the old ssl implementation and + will be updated for the new one once this functionallity is + supported by the new implementation.</p></note> + </p> <section> <title>Introduction</title> diff --git a/lib/ssl/doc/src/ssl_protocol.xml b/lib/ssl/doc/src/ssl_protocol.xml index 3dc2332795..726b9a4eeb 100644 --- a/lib/ssl/doc/src/ssl_protocol.xml +++ b/lib/ssl/doc/src/ssl_protocol.xml @@ -13,337 +13,138 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> - <title>The SSL Protocol</title> - <prepared>Peter Högfeldt</prepared> - <docno></docno> - <date>2003-04-28</date> - <rev>PA2</rev> + <title>Transport Layer Security (TLS) and its predecessor, Secure Socket Layer (SSL)</title> <file>ssl_protocol.xml</file> </header> - <p>Here we provide a short introduction to the SSL protocol. We only - consider those part of the protocol that are important from a - programming point of view. - </p> - <p>For a very good general introduction to SSL and TLS see the book - <cite id="rescorla"></cite>. - </p> - <p><em>Outline:</em></p> - <list type="bulleted"> - <item>Two types of connections - connection: handshake, data transfer, and - shutdown - - SSL/TLS protocol - server must have certificate - what the the - server sends to the client - client may verify the server - - server may ask client for certificate - what the client sends to - the server - server may then verify the client - verification - - certificate chains - root certificates - public keys - key - agreement - purpose of certificate - references</item> - </list> + + <p>The erlang ssl application currently supports SSL 3.0 and TLS 1.0 + RFC 2246, and will in the future also support later versions of TLS. + SSL 2.0 is not supported. + </p> - <section> - <title>SSL Connections</title> - <p>The SSL protocol is implemented on top of the TCP/IP protocol. - From an endpoint view it also has the same type of connections - as that protocol, almost always created by calls to socket - interface functions <em>listen</em>, <em>accept</em> and - <em>connect</em>. The endpoints are <em>servers</em> and - <em>clients</em>. - </p> - <p>A <em>server</em><em>listen</em>s for connections on a - specific address and port. This is done once. The server then - <em>accept</em>s each connections on that same address and - port. This is typically done indefinitely many times. - </p> - <p>A <em>client</em> connects to a server on a specific address - and port. For each purpose this is done once. - </p> - <p>For a plain TCP/IP connection the establishment of a connection - (through an accept or a connect) is followed by data transfer between - the client and server, finally ended by a connection close. - </p> - <p>An SSL connection also consists of data transfer and connection - close, However, the data transfer contains encrypted data, and - in order to establish the encryption parameters, the data - transfer is preceded by an SSL <em>handshake</em>. In this - handshake the server plays a dominant role, and the main - instrument used in achieving a valid SSL connection is the - server's <em>certificate</em>. We consider certificates in the - next section, and the SSL handshake in a subsequent section.</p> - </section> + <p>By default erlang ssl is run over the TCP/IP protocol even + though you could plug in an other reliable transport protocol + with the same API as gen_tcp.</p> + + <p>If a client and server wants to use an upgrade mechanism, such as + defined by RFC2817, to upgrade a regular TCP/IP connection to a ssl + connection the erlang ssl API supports this. This can be useful for + things such as supporting HTTP and HTTPS on the same port and + implementing virtual hosting. + </p> <section> - <title>Certificates</title> - <p>A certificate is similar to a driver's license, or a - passport. The holder of the certificate is called the - <em>subject</em>. First of all the certificate identifies the - subject in terms of the name of the subject, its postal address, - country name, company name (if applicable), etc. - </p> - <p>Although a driver's license is always issued by a well-known and - distinct authority, a certificate may have an <em>issuer</em> - that is not so well-known. Therefore a certificate also always - contains information on the issuer of the certificate. That - information is of the same type as the information on the - subject. The issuer of a certificate also signs the certificate - with a <em>digital signature</em> (the signature is an inherent - part of the certificate), which allow others to verify that the - issuer really is the issuer of the certificate. - </p> - <p>Now that a certificate can be checked by verifying the - signature of the issuer, the question is how to trust the - issuer. The answer to this question is to require that there is - a certificate for the issuer as well. That issuer has in turn an - issuer, which must also have a certificate, and so on. This - <em>certificate chain</em> has to have en end, which then must - be a certificate that is trusted by other means. We shall cover - this problem of <em>authentication</em> in a subsequent - section. - </p> + <title>Security overview</title> + + <p>To achive authentication and privacy the client and server will + perform a TLS Handshake procedure before transmitting or receiving + any data. During the handshake they agree on a protocol version and + cryptographic algorithms, they generate shared secrets using public + key cryptographics and optionally authenticate each other with + digital certificates.</p> </section> - + <section> - <title>Encryption Algorithms</title> - <p>An encryption algorithm is a mathematical algorithm for - encryption and decryption of messages (arrays of bytes, - say). The algorithm as such is always required to be publicly - known, otherwise its strength cannot be evaluated, and hence it - cannot be used reliably. The secrecy of an encrypted message is - not achieved by the secrecy of the algorithm used, but by the - secrecy of the <em>keys</em> used as input to the encryption and - decryption algorithms. For an account of cryptography in general - see <cite id="schneier"></cite>. - </p> - <p>There are two classes of encryption algorithms: <em>symmetric key</em> algorithms and <em>public key</em> algorithms. Both - types of algorithms are used in the SSL protocol. - </p> - <p>In the sequel we assume holders of keys keep them secret (except - public keys) and that they in that sense are trusted. How a - holder of a secret key is proved to be the one it claims to be - is a question of <em>authentication</em>, which, in the context - of the SSL protocol, is described in a section further below. - </p> - - <section> - <title>Symmetric Key Algorithms</title> - <p>A <em>symmetric key</em> algorithm has one key only. The key - is used for both encryption and decryption. Obviously the key - of a symmetric key algorithm must always be kept secret by the - users of the key. DES is an example of a symmetric key - algorithm. - </p> - <p>Symmetric key algorithms are fast compared to public key - algorithms. They are therefore typically used for encrypting - bulk data. - </p> - </section> - - <section> - <title>Public Key Algorithms</title> - <p>A <em>public key</em> algorithm has two keys. Any of the two - keys can be used for encryption. A message encrypted with one - of the keys, can only be decrypted with the other key. One of - the keys is public (known to the world), while the other key - is private (i.e. kept secret) by the owner of the two keys. - </p> - <p>RSA is an example of a public key algorithm. - </p> - <p>Public key algorithms are slow compared to symmetric key - algorithms, and they are therefore seldom used for bulk data - encryption. They are therefore only used in cases where the - fact that one key is public and the other is private, provides - features that cannot be provided by symmetric algorithms. - </p> - </section> - - <section> - <title>Digital Signature Algorithms</title> - <p>An interesting feature of a public key algorithm is that its - public and private keys can both be used for encryption. - Anyone can use the public key to encrypt a message, and send - that message to the owner of the private key, and be sure of - that only the holder of the private key can decrypt the - message. - </p> - <p>On the other hand, the owner of the private key can encrypt a - message with the private key, thus obtaining an encrypted - message that can decrypted by anyone having the public key. - </p> - <p>The last approach can be used as a digital signature - algorithm. The holder of the private key signs an array of - bytes by performing a specified well-known <em>message digest algorithm</em> to compute a hash of the array, encrypts the - hash value with its private key, an then presents the original - array, the name of the digest algorithm, and the encryption of - the hash value as a <em>signed array of bytes</em>. - </p> - <p>Now anyone having the public key, can decrypt the encrypted - hash value with that key, compute the hash with the specified - digest algorithm, and check that the hash values compare equal - in order to verify that the original array was indeed signed - by the holder of the private key. - </p> - <p>What we have accounted for so far is by no means all that can - be said about digital signatures (see <cite id="schneier"></cite>for - further details). - </p> - </section> - - <section> - <title>Message Digests Algorithms</title> - <p>A message digest algorithm is a hash function that accepts - an array bytes of arbitrary but finite length of input, and - outputs an array of bytes of fixed length. Such an algorithm - is also required to be very hard to invert. - </p> - <p>MD5 (16 bytes output) and SHA1 (20 bytes output) are examples - of message digest algorithms. - </p> - </section> + <title>Data Privacy and Integrity</title> + + <p>A <em>symmetric key</em> algorithm has one key only. The key is + used for both encryption and decryption. These algoritms are fast + compared to public key algorithms (using two keys, a public and a + private one) and are therefore typically used for encrypting bulk + data. + </p> + + <p>The keys for the symmetric encryption are generated uniquely + for each connection and are based on a secret negotiated + in the TLS handshake. </p> + + <p>The TLS handsake protocol and data transfer is run on top of + the TLS Record Protocol that uses a keyed-hash MAC (Message + Authenticity Code), or HMAC, to protect the message's data + integrity. From the TLS RFC "A Message Authentication Code is a + one-way hash computed from a message and some secret data. It is + difficult to forge without knowing the secret data. Its purpose is + to detect if the message has been altered." + </p> + </section> - <section> - <title>SSL Handshake</title> - <p>The main purpose of the handshake performed before an an SSL - connection is established is to negotiate the encryption - algorithm and key to be used for the bulk data transfer between - the client and the server. We are writing <em>the</em> key, - since the algorithm to choose for bulk encryption one of the - symmetric algorithms. - </p> - <p>There is thus only one key to agree upon, and obviously that - key has to be kept secret between the client and the server. To - obtain that the handshake has to be encrypted as well. - </p> - <p>The SSL protocol requires that the server always sends its - certificate to the client in the beginning of the handshake. The - client then retrieves the server's public key from the - certificate, which means that the client can use the server's - public key to encrypt messages to the server, and the server can - decrypt those messages with its private key. Similarly, the - server can encrypt messages to the client with its private key, - and the client can decrypt messages with the server's public - key. It is thus is with the server's public and private keys - that messages in the handshake are encrypted and decrypted, and - hence the key agreed upon for symmetric encryption of bulk data - can be kept secret (there are more things to consider to really - keep it secret, see <cite id="rescorla"></cite>). - </p> - <p>The above indicates that the server does not care who is - connecting, and that only the client has the possibility to - properly identify the server based on the server's certificate. - That is indeed true in the minimal use of the protocol, but it - is possible to instruct the server to request the certificate of - the client, in order to have a means to identify the client, but - it is by no means required to establish an SSL connection. - </p> - <p>If a server request the client certificate, it verifies, as a - part of the protocol, that the client really holds the private - key of the certificate by sending the client a string of bytes - to encrypt with its private key, which the server then decrypts - with the client's public key, the result of which is compared - with the original string of bytes (a similar procedure is always - performed by the client when it has received the server's - certificate). - </p> - <p>The way clients and servers <em>authenticate</em> each other, - i.e. proves that their respective peers are what they claim to - be, is the topic of the next section. - </p> - </section> + <section> + <title>Digital Certificates</title> + <p>A certificate is similar to a driver's license, or a + passport. The holder of the certificate is called the + <em>subject</em>. The certificate is signed + with the private key of the issuer of the certificate. A chain + of trust is build by having the issuer in its turn being + certified by an other certificate and so on until you reach the + so called root certificate that is self signed e.i. issued + by itself.</p> + + <p>Certificates are issued by <em>certification + authorities</em> (<em>CA</em>s) only. There are a handful of + top CAs in the world that issue root certificates. You can + examine the certificates of several of them by clicking + through the menus of your web browser. + </p> + </section> + + <section> + <title>Authentication of Sender</title> + + <p>Authentication of the sender is done by public key path + validation as defined in RFC 3280. Simplified that means that + each certificate in the certificate chain is issued by the one + before, the certificates attributes are valid ones, and the + root cert is a trusted cert that is present in the trusted + certs database kept by the peer.</p> + + <p>The server will always send a certificate chain as part of + the TLS handshake, but the client will only send one if + the server requests it. If the client does not have + an appropriate certificate it may send an "empty" certificate + to the server.</p> + + <p>The client may choose to accept some path evaluation errors + for instance a web browser may ask the user if they want to + accept an unknown CA root certificate. The server, if it request + a certificate, will on the other hand not accept any path validation + errors. It is configurable if the server should accept + or reject an "empty" certificate as response to + a certificate request.</p> + </section> + + <section> + <title>TLS Sessions</title> + + <p>From the TLS RFC "A TLS session is an association between a + client and a server. Sessions are created by the handshake + protocol. Sessions define a set of cryptographic security + parameters, which can be shared among multiple + connections. Sessions are used to avoid the expensive negotiation + of new security parameters for each connection."</p> - <section> - <title>Authentication</title> - <p>As we have already seen the reception of a certificate from a - peer is not enough to prove that the peer is authentic. More - certificates are needed, and we have to consider how certificates - are issued and on what grounds. - </p> - <p>Certificates are issued by <em>certification authorities</em> - (<em>CA</em>s) only. They issue certificates both for other CAs - and ordinary users (which are not CAs). - </p> - <p>Certain CAs are <em>top CAs</em>, i.e. they do not have a - certificate issued by another CA. Instead they issue their own - certificate, where the subject and issuer part of the - certificate are identical (such a certificate is called a - self-signed certificate). A top CA has to be well-known, and has - to have a publicly available policy telling on what grounds it - issues certificates. - </p> - <p>There are a handful of top CAs in the world. You can examine the - certificates of several of them by clicking through the menus of - your web browser. - </p> - <p>A top CA typically issues certificates for other CAs, called - <em>intermediate CAs</em>, but possibly also to ordinary users. Thus - the certificates derivable from a top CA constitute a tree, where - the leaves of the tree are ordinary user certificates. - </p> - <p>A <em>certificate chain</em> is an ordered sequence of - certificates, <c>C1, C2, ..., Cn</c>, say, where <c>C1</c> is a - top CA certificate, and where <c>Cn</c> is an ordinary user - certificate, and where the holder of <c>C1</c> is the issuer of - <c>C2</c>, the holder of <c>C2</c> is the issuer of <c>C3</c>, - ..., and the holder of <c>Cn-1</c> is the issuer of <c>Cn</c>, - the ordinary user certificate. The holders of <c>C2, C3, ..., Cn-1</c> are then intermediate CAs. - </p> - <p>Now to verify that a certificate chain is unbroken we have to - take the public key from each certificate <c>Ck</c>, and apply - that key to decrypt the signature of certificate <c>Ck-1</c>, - thus obtaining the message digest computed by the holder of the - <c>Ck</c> certificate, compute the real message digest of the - <c>Ck-1</c> certificate and compare the results. If they compare - equal the link of the chain between <c>Ck</c> and <c>Ck-1</c> is - considered to unbroken. This is done for each link k = 1, 2, - ..., n-1. If all links are found to be unbroken, the user - certificate <c>Cn</c> is considered authenticated. - </p> + <p>Session data is by default kept by the ssl application in a + memory storage hence session data will be lost at application + restart or takeover. Users may define their own callback module + to handle session data storage if persistent data storage is + required. Session data will also be invalidated after 24 hours + from it was saved, for security reasons. It is of course + possible to configure the amount of time the session data should be + saved.</p> - <section> - <title>Trusted Certificates</title> - <p>Now that there is a way to authenticate a certificate by - checking that all links of a certificate chain are unbroken, - the question is how you can be sure to trust the certificates - in the chain, and in particular the top CA certificate of the - chain. - </p> - <p>To provide an answer to that question consider the - perspective of a client, which have just received the - certificate of the server. In order to authenticate the server - the client has to construct a certificate chain and to prove - that the chain is unbroken. The client has to have a set of CA - certificates (top CA or intermediate CA certificates) not - obtained from the server, but obtained by other means. Those - certificates are kept <c>locally</c> by the client, and are - trusted by the client. - </p> - <p>More specifically, the client does not really have to have - top CA certificates in its local storage. In order to - authenticate a server it is sufficient for the client to - posses the trusted certificate of the issuer of the server - certificate. - </p> - <p>Now that is not the whole story. A server can send an - (incomplete) certificate chain to its client, and then the - task of the client is to construct a certificate chain that - begins with a trusted certificate and ends with the server's - certificate. (A client can also send a chain to its server, - provided the server requested the client's certificate.) - </p> - <p>All this means that an unbroken certificate chain begins with - a trusted certificate (top CA or not), and ends with the peer - certificate. That is the end of the chain is obtained from the - peer, but the beginning of the chain is obtained from local - storage, which is considered trusted. - </p> - </section> - </section> -</chapter> + <p>Ssl clients will by default try to reuse an available session, + ssl servers will by default agree to reuse sessions when clients + ask to do so.</p> + + </section> + </chapter> diff --git a/lib/ssl/doc/src/ssl_session_cache_api.xml b/lib/ssl/doc/src/ssl_session_cache_api.xml new file mode 100644 index 0000000000..7b70c6cf34 --- /dev/null +++ b/lib/ssl/doc/src/ssl_session_cache_api.xml @@ -0,0 +1,158 @@ +<?xml version="1.0" encoding="latin1" ?> +<!DOCTYPE erlref SYSTEM "erlref.dtd"> + +<erlref> + <header> + <copyright> + <year>1999</year><year>2010</year> + <holder>Ericsson AB. All Rights Reserved.</holder> + </copyright> + <legalnotice> + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + </legalnotice> + <title>ssl</title> + <file>ssl_session_cache_api.xml</file> + </header> + <module>ssl_session_cache_api</module> + <modulesummary>Defines the API for the TLS session cache so + that the datastorge scheme can be replaced by + defining a new callback module implementing this API.</modulesummary> + + <section> + <title>Common Data Types</title> + + <p>The following data types are used in the functions below: + </p> + + <p><c>cache_ref() = opaque()</c></p> + + <p><c>key() = {partialkey(), session_id()}</c></p> + + <p><c>partialkey() = opaque()</c></p> + + <p><c>session_id() = binary()</c></p> + + <p><c>session() = opaque()</c></p> + + </section> + + <funcs> + + <func> + <name>delete(Cache, Key) -> _</name> + <fsummary></fsummary> + <type> + <v> Cache = cache_ref()</v> + <v> Key = key()</v> + </type> + <desc> + <p> Delets a cache entry. Will only be called from the cache + handling process. + </p> + </desc> + </func> + + <func> + <name>foldl(Fun, Acc0, Cache) -> Acc</name> + <fsummary></fsummary> + <type> + <v></v> + </type> + <desc> + <p>Calls Fun(Elem, AccIn) on successive elements of the + cache, starting with AccIn == Acc0. Fun/2 must return a new + accumulator which is passed to the next call. The function returns + the final value of the accumulator. Acc0 is returned if the cache is + empty. + </p> + </desc> + </func> + + <func> + <name>init() -> opaque() </name> + <fsummary>Return cache reference</fsummary> + <type> + <v></v> + </type> + <desc> + <p>Performes possible initializations of the cache and returns + a reference to it that will be used as parameter to the other + api functions. Will be called by the cache handling processes + init function, hence puting the same requierments on it as + a normal process init function. + </p> + </desc> + </func> + + <func> + <name>lookup(Cache, Key) -> Entry</name> + <fsummary> Looks up a cach entry.</fsummary> + <type> + <v> Cache = cache_ref()</v> + <v> Key = key()</v> + <v> Entry = session() | undefined </v> + </type> + <desc> + <p>Looks up a cach entry. Should be callable from any + process. + </p> + </desc> + </func> + + <func> + <name>select_session(Cache, PartialKey) -> [session()]</name> + <fsummary>>Selects sessions that could be reused.</fsummary> + <type> + <v> Cache = cache_ref()</v> + <v> PartialKey = partialkey()</v> + <v> Session = session()</v> + </type> + <desc> + <p>Selects sessions that could be reused. Should be callable + from any process. + </p> + </desc> + </func> + + <func> + <name>terminate(Cache) -> _</name> + <fsummary>Called by the process that handles the cache when it + is aboute to terminat.</fsummary> + <type> + <v>Cache = term() - as returned by init/0</v> + </type> + <desc> + <p>Takes care of possible cleanup that is needed when the + cache handling process terminates. + </p> + </desc> + </func> + + <func> + <name>update(Cache, Key, Session) -> _</name> + <fsummary> Caches a new session or updates a already cached one.</fsummary> + <type> + <v> Cache = cache_ref()</v> + <v> Key = key()</v> + <v> Session = session()</v> + </type> + <desc> + <p> Caches a new session or updates a already cached one. Will + only be called from the cache handling process. + </p> + </desc> + </func> + + </funcs> + +</erlref> diff --git a/lib/ssl/doc/src/usersguide.xml b/lib/ssl/doc/src/usersguide.xml index 98071f5742..6528c00a0b 100644 --- a/lib/ssl/doc/src/usersguide.xml +++ b/lib/ssl/doc/src/usersguide.xml @@ -4,7 +4,7 @@ <part xmlns:xi="http://www.w3.org/2001/XInclude"> <header> <copyright> - <year>2000</year><year>2009</year> + <year>2000</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,43 +13,27 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>SSL User's Guide</title> <prepared>OTP Team</prepared> - <docno></docno> <date>2003-05-26</date> - <rev>B</rev> <file>usersguide.sgml</file> </header> <description> <p>The <em>SSL</em> application provides secure communication over sockets. </p> - <p>This product includes software developed by the OpenSSL Project for - use in the OpenSSL Toolkit (http://www.openssl.org/). - </p> - <p>This product includes cryptographic software written by Eric Young - ([email protected]). - </p> - <p>This product includes software written by Tim Hudson - ([email protected]). - </p> - <p>For full OpenSSL and SSLeay license texts, see <seealso marker="licenses#licenses">Licenses</seealso>. - </p> </description> <xi:include href="ssl_protocol.xml"/> <xi:include href="using_ssl.xml"/> - <xi:include href="pkix_certs.xml"/> - <xi:include href="create_certs.xml"/> <xi:include href="ssl_distribution.xml"/> - <xi:include href="licenses.xml"/> </part> diff --git a/lib/ssl/doc/src/using_ssl.xml b/lib/ssl/doc/src/using_ssl.xml index ba74dcfef4..4bdd8f97b4 100644 --- a/lib/ssl/doc/src/using_ssl.xml +++ b/lib/ssl/doc/src/using_ssl.xml @@ -21,93 +21,129 @@ </legalnotice> - <title>Using the SSL application</title> - <prepared>Peter Högfeldt</prepared> - <docno></docno> - <date>2003-04-23</date> - <rev>PA2</rev> + <title>Using the SSL API</title> <file>using_ssl.xml</file> </header> - <p>Here we provide an introduction to using the Erlang/OTP SSL - application, which is accessed through the <c>ssl</c> interface - module. - </p> - <p>We also present example code in the Erlang module - <c>client_server</c>, also provided in the directory - <c>ssl-X.Y.Z/examples</c>, with source code in <c>src</c> and the - compiled module in <c>ebin</c> of that directory. - </p> <section> - <title>The ssl Module</title> - <p>The <c>ssl</c> module provides the user interface to the Erlang/OTP - SSL application. The interface functions provided are very similar - to those provided by the <c>gen_tcp</c> and <c>inet</c> modules. - </p> - <p>Servers use the interface functions <c>listen</c> and - <c>accept</c>. The <c>listen</c> function specifies a TCP port - to to listen to, and each call to the <c>accept</c> function - establishes an incoming connection. - </p> - <p>Clients use the <c>connect</c> function which specifies the address - and port of a server to connect to, and a successful call establishes - such a connection. - </p> - <p>The <c>listen</c> and <c>connect</c> functions have almost all - the options that the corresponding functions in <c>gen_tcp/</c> have, - but there are also additional options specific to the SSL protocol. - </p> - <p>The most important SSL specific option is the <c>cacertfile</c> - option which specifies a local file containing trusted CA - certificates which are and used for peer authentication. This - option is used by clients and servers in case they want to - authenticate their peers. - </p> - <p>The <c>certfile</c> option specifies a local path to a file - containing the certificate of the holder of the connection - endpoint. In case of a server endpoint this option is mandatory - since the contents of the sever certificate is needed in the - the handshake preceding the establishment of a connection. - </p> - <p>Similarly, the <c>keyfile</c> option points to a local file - containing the private key of the holder of the endpoint. If the - <c>certfile</c> option is present, this option has to be - specified as well, unless the private key is provided in the - same file as specified by the <c>certfile</c> option (a - certificate and a private key can thus coexist in the same file). - </p> - <p>The <c>verify</c> option specifies how the peer should be verified: - </p> - <taglist> - <tag>0</tag> - <item>Do not verify the peer,</item> - <tag>1</tag> - <item>Verify peer,</item> - <tag>2</tag> - <item>Verify peer, fail the verification if the peer has no - certificate. </item> - </taglist> - <p>The <c>depth</c> option specifies the maximum length of the - verification certificate chain. Depth = 0 means the peer - certificate, depth = 1 the CA certificate, depth = 2 the next CA - certificate etc. If the verification process does not find a - trusted CA certificate within the maximum length, the verification - fails. - </p> - <p>The <c>ciphers</c> option specifies which ciphers to use (a - string of colon separated cipher names). To obtain a list of - available ciphers, evaluate the <c>ssl:ciphers/0</c> function - (the SSL application has to be running). - </p> - </section> + <title>General information</title> + <p>To see relevant version information for ssl you can + call ssl:versions/0</p> + + <p>To see all supported cipher suites + call ssl:cipher_suites/0. Note that available cipher suites + for a connection will depend on your certificate. It is also + possible to specify a specific cipher suite(s) that you + want your connection to use. Default is to use the strongest + available.</p> - <section> - <title>A Client-Server Example</title> - <p>Here is a simple client server example. - </p> - <codeinclude file="../../examples/src/client_server.erl" tag="" type="erl"></codeinclude> </section> -</chapter> - - + + <section> + <title>Setting up connections</title> + + <p>Here follows some small example of how to set up client/server connections + using the erlang shell. The returned value of the sslsocket has been abbreviated with + <c>[...]</c> as it can be fairly large and is opaque.</p> + + <section> + <title>Minmal example</title> + + <note><p> The minimal setup is not the most secure setup of ssl.</p> + </note> + + <p> Start server side</p> + <code type="erl">1 server> ssl:start(). +ok</code> + + <p>Create a ssl listen socket</p> + <code type="erl">2 server> {ok, ListenSocket} = +ssl:listen(9999, [{certfile, "cert.pem"}, {keyfile, "key.pem"},{reuseaddr, true}]). +{ok,{sslsocket, [...]}}</code> + + <p>Do a transport accept on the ssl listen socket</p> + <code type="erl">3 server> {ok, Socket} = ssl:transport_accept(ListenSocket). +{ok,{sslsocket, [...]}}</code> + <p>Start client side</p> + <code type="erl">1 client> ssl:start(). +ok</code> + + <code type="erl">2 client> {ok, Socket} = ssl:connect("localhost", 9999, [], infinity). +{ok,{sslsocket, [...]}}</code> + + <p>Do the ssl handshake</p> + <code type="erl">4 server> ok = ssl:ssl_accept(Socket). +ok</code> + + <p>Send a messag over ssl</p> + <code type="erl">5 server> ssl:send(Socket, "foo"). +ok</code> + + <p>Flush the shell message queue to see that we got the message + sent on the server side</p> + <code type="erl">3 client> flush(). +Shell got {ssl,{sslsocket,[...]},"foo"} +ok</code> + </section> + + <section> + <title>Upgrade example</title> + + <note><p> To upgrade a TCP/IP connection to a ssl connection the + client and server have to aggre to do so. Agreement + may be accompliced by using a protocol such the one used by HTTP + specified in RFC 2817.</p> </note> + + <p>Start server side</p> + <code type="erl">1 server> ssl:start(). +ok</code> + + <p>Create a normal tcp listen socket</p> + <code type="erl">2 server> {ok, ListenSocket} = gen_tcp:listen(9999, [{reuseaddr, true}]). +{ok, #Port<0.475>}</code> + + <p>Accept client connection</p> + <code type="erl">3 server> {ok, Socket} = gen_tcp:accept(ListenSocket). +{ok, #Port<0.476>}</code> + + <p>Start client side</p> + <code type="erl">1 client> ssl:start(). +ok</code> + + <code type="erl">2 client> {ok, Socket} = gen_tcp:connect("localhost", 9999, [], infinity).</code> + + <p>Make sure active is set to false before trying + to upgrade a connection to a ssl connection, otherwhise + ssl handshake messages may be deliverd to the wrong process.</p> + <code type="erl">4 server> inet:setopts(Socket, [{active, false}]). +ok</code> + + <p>Do the ssl handshake.</p> + <code type="erl">5 server> {ok, SSLSocket} = ssl:ssl_accept(Socket, [{cacertfile, "cacerts.pem"}, +{certfile, "cert.pem"}, {keyfile, "key.pem"}]). +{ok,{sslsocket,[...]}}</code> + + <p> Upgrade to a ssl connection. Note that the client and server + must agree upon the upgrade and the server must call + ssl:accept/2 before the client calls ssl:connect/3.</p> + <code type="erl">3 client>{ok, SSLSocket} = ssl:connect(Socket, [{cacertfile, "cacerts.pem"}, +{certfile, "cert.pem"}, {keyfile, "key.pem"}], infinity). +{ok,{sslsocket,[...]}}</code> + + <p>Send a messag over ssl</p> + <code type="erl">4 client> ssl:send(SSLSocket, "foo"). +ok</code> + + <p>Set active true on the ssl socket</p> + <code type="erl">4 server> ssl:setopts(SSLSocket, [{active, true}]). +ok</code> + + <p>Flush the shell message queue to see that we got the message + sent on the client side</p> + <code type="erl">5 server> flush(). +Shell got {ssl,{sslsocket,[...]},"foo"} +ok</code> + </section> + </section> + </chapter> diff --git a/lib/ssl/pkix/Makefile b/lib/ssl/pkix/Makefile deleted file mode 100644 index 260361c025..0000000000 --- a/lib/ssl/pkix/Makefile +++ /dev/null @@ -1,121 +0,0 @@ -# -# %CopyrightBegin% -# -# Copyright Ericsson AB 2003-2009. All Rights Reserved. -# -# The contents of this file are subject to the Erlang Public License, -# Version 1.1, (the "License"); you may not use this file except in -# compliance with the License. You should have received a copy of the -# Erlang Public License along with this software. If not, it can be -# retrieved online at http://www.erlang.org/. -# -# Software distributed under the License is distributed on an "AS IS" -# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -# the License for the specific language governing rights and limitations -# under the License. -# -# %CopyrightEnd% -# - -# - -include $(ERL_TOP)/make/target.mk -include $(ERL_TOP)/make/$(TARGET)/otp.mk - -# ---------------------------------------------------- -# Application version -# ---------------------------------------------------- -include ../vsn.mk -VSN=$(SSL_VSN) - -# ---------------------------------------------------- -# Release directory specification -# ---------------------------------------------------- -RELSYSDIR = $(RELEASE_PATH)/lib/ssl-$(VSN) - -# ---------------------------------------------------- -# Common Macros -# ---------------------------------------------------- - -.SUFFIXES: .asn1 -.PRECIOUS: %.erl - -ASN_TOP = OTP-PKIX -ASN_MODULES = PKIX1Explicit88 PKIX1Implicit88 PKIX1Algorithms88 \ - PKIXAttributeCertificate SSL-PKIX -ASN_ASNS = $(ASN_MODULES:%=%.asn1) -ASN_ERLS = $(ASN_TOP).erl -ASN_HRLS = $(ASN_TOP).hrl -ASN_CONFIGS = OTP-PKIX.asn1config -ASN_DBS = $(ASN_MODULES:%=%.asn1db) -ASN_TABLES = $(ASN_MODULES:%=%.table) - -GEN_MODULES = ssl_pkix_oid $(ORBER_TMP_FIX_ERL) -GEN_ERLS = $(GEN_MODULES:%=%.erl) -ERL_MODULES = $(ASN_TOP) $(GEN_MODULES) - -TARGET_FILES= $(ERL_MODULES:%=$(EBIN)/%.$(EMULATOR)) - -HRL_FILES = $(ASN_HRLS:%=$(INCLUDE)/%) - -ORBER_TMP_FIX_HRL = PKIX1Algorithms88.hrl PKIX1Explicit88.hrl \ - PKIX1Implicit88.hrl PKIXAttributeCertificate.hrl - -INCLUDE = ../include -EBIN = ../ebin - -# ---------------------------------------------------- -# FLAGS -# ---------------------------------------------------- -EXTRA_ERLC_FLAGS = -ERL_COMPILE_FLAGS += $(EXTRA_ERLC_FLAGS) - -ASN_FLAGS = -bber_bin +der +compact_bit_string +optimize +noobj +asn1config +inline - -# ---------------------------------------------------- -# Targets -# ---------------------------------------------------- - -debug opt: $(TARGET_FILES) $(HRL_FILES) - -clean: - -rm -f $(ASN_ERLS) $(GEN_ERLS) $(ASN_HRLS) $(HRL_FILES) $(ASN_DBS) \ - $(ASN_TABLES) $(TARGET_FILES) *.beam *~ - -docs: - -%.erl: %.set.asn - erlc $(ASN_FLAGS) $< - -ssl_pkix_oid.erl: mk_ssl_pkix_oid.beam $(EBIN)/OTP-PKIX.beam - erl -pa $(EBIN) -noshell -s mk_ssl_pkix_oid make -s erlang halt - -$(HRL_FILES): $(ASN_HRLS) - cp -p $(ASN_HRLS) $(INCLUDE) - -# ---------------------------------------------------- -# Release Target -# ---------------------------------------------------- -include $(ERL_TOP)/make/otp_release_targets.mk - -release_spec: opt - $(INSTALL_DIR) $(RELSYSDIR)/include - $(INSTALL_DATA) $(HRL_FILES) $(RELSYSDIR)/include - $(INSTALL_DIR) $(RELSYSDIR)/pkix - $(INSTALL_DATA) $(ASN_ASNS) $(ASN_ERLS) $(ASN_HRLS) $(ASN_CONFIGS) \ - $(ORBER_TMP_FIX_HRL) $(GEN_ERLS) mk_ssl_pkix_oid.erl $(RELSYSDIR)/pkix - $(INSTALL_DIR) $(RELSYSDIR)/ebin - $(INSTALL_DATA) $(TARGET_FILES) $(RELSYSDIR)/ebin - -release_docs_spec: - -# -# Dependencies - -$(EBIN)/OTP-PKIX.beam: OTP-PKIX.erl OTP-PKIX.hrl -OTP-PKIX.erl OTP-PKIX.hrl: OTP-PKIX.asn1db -OTP-PKIX.asn1db: PKIX1Algorithms88.asn1 \ - PKIX1Explicit88.asn1 \ - PKIX1Implicit88.asn1 \ - PKIXAttributeCertificate.asn1 \ - SSL-PKIX.asn1 diff --git a/lib/ssl/pkix/OTP-PKIX.asn1config b/lib/ssl/pkix/OTP-PKIX.asn1config deleted file mode 100644 index 0caa158f52..0000000000 --- a/lib/ssl/pkix/OTP-PKIX.asn1config +++ /dev/null @@ -1,2 +0,0 @@ -{exclusive_decode,{'OTP-PKIX', - [{decode_TBSCert_exclusive,['Certificate',[{tbsCertificate,undecoded}]]}]}}. diff --git a/lib/ssl/pkix/OTP-PKIX.set.asn b/lib/ssl/pkix/OTP-PKIX.set.asn deleted file mode 100644 index 1c3483d519..0000000000 --- a/lib/ssl/pkix/OTP-PKIX.set.asn +++ /dev/null @@ -1,6 +0,0 @@ -SSL-PKIX.asn1 -PKIX1Explicit88.asn1 -PKIX1Implicit88.asn1 -PKIXAttributeCertificate.asn1 -PKIX1Algorithms88.asn1 -PKCS-1.asn1 diff --git a/lib/ssl/pkix/PKCS-1.asn1 b/lib/ssl/pkix/PKCS-1.asn1 deleted file mode 100755 index 547cc2e072..0000000000 --- a/lib/ssl/pkix/PKCS-1.asn1 +++ /dev/null @@ -1,54 +0,0 @@ -PKCS-1 { - iso(1) member-body(2) us(840) rsadsi(113549) pkcs(1) pkcs-1(1) - modules(0) pkcs-1(1) -} - - -DEFINITIONS IMPLICIT TAGS ::= BEGIN - --- EXPORTS ALL -- - -IMPORTS - AlgorithmIdentifier - FROM PKIX1Explicit88 {iso(1) identified-organization(3) - dod(6) internet(1) security(5) mechanisms(5) - pkix(7) id-mod(0) id-pkix1-explicit-88(1)} ; - -pkcs-1 OBJECT IDENTIFIER ::= { - iso(1) member-body(2) us(840) rsadsi(113549) pkcs(1) 1 } - -RSAPrivateKey ::= SEQUENCE { - version Version, - modulus INTEGER, -- n - publicExponent INTEGER, -- e - privateExponent INTEGER, -- d - prime1 INTEGER, -- p - prime2 INTEGER, -- q - exponent1 INTEGER, -- d mod (p-1) - exponent2 INTEGER, -- d mod (q-1) - coefficient INTEGER, -- (inverse of q) mod p - otherPrimeInfos OtherPrimeInfos OPTIONAL -} - -Version ::= INTEGER { two-prime(0), multi(1) } - (CONSTRAINED BY { - -- version must be multi if otherPrimeInfos present -- - }) - -OtherPrimeInfos ::= SEQUENCE SIZE(1..MAX) OF OtherPrimeInfo - -OtherPrimeInfo ::= SEQUENCE { - prime INTEGER, -- ri - exponent INTEGER, -- di - coefficient INTEGER -- ti -} - -DigestInfo ::= SEQUENCE { - digestAlgorithm DigestAlgorithmIdentifier, - digest OCTET STRING -} - -DigestAlgorithmIdentifier ::= AlgorithmIdentifier - -END -- PKCS1Definitions - diff --git a/lib/ssl/pkix/PKIX1Algorithms88.asn1 b/lib/ssl/pkix/PKIX1Algorithms88.asn1 deleted file mode 100644 index e78de69b0e..0000000000 --- a/lib/ssl/pkix/PKIX1Algorithms88.asn1 +++ /dev/null @@ -1,274 +0,0 @@ - PKIX1Algorithms88 { iso(1) identified-organization(3) dod(6) - internet(1) security(5) mechanisms(5) pkix(7) id-mod(0) - id-mod-pkix1-algorithms(17) } - - DEFINITIONS EXPLICIT TAGS ::= BEGIN - - -- EXPORTS All; - - -- IMPORTS NONE; - - -- - -- One-way Hash Functions - -- - - md2 OBJECT IDENTIFIER ::= { - iso(1) member-body(2) us(840) rsadsi(113549) - digestAlgorithm(2) 2 } - - md5 OBJECT IDENTIFIER ::= { - iso(1) member-body(2) us(840) rsadsi(113549) - digestAlgorithm(2) 5 } - - id-sha1 OBJECT IDENTIFIER ::= { - iso(1) identified-organization(3) oiw(14) secsig(3) - algorithms(2) 26 } - - -- - -- DSA Keys and Signatures - -- - - -- OID for DSA public key - - id-dsa OBJECT IDENTIFIER ::= { - iso(1) member-body(2) us(840) x9-57(10040) x9algorithm(4) 1 } - - -- encoding for DSA public key - - DSAPublicKey ::= INTEGER -- public key, y - - Dss-Parms ::= SEQUENCE { - p INTEGER, - q INTEGER, - g INTEGER } - - -- OID for DSA signature generated with SHA-1 hash - - id-dsa-with-sha1 OBJECT IDENTIFIER ::= { - iso(1) member-body(2) us(840) x9-57 (10040) x9algorithm(4) 3 } - - -- encoding for DSA signature generated with SHA-1 hash - - Dss-Sig-Value ::= SEQUENCE { - r INTEGER, - s INTEGER } - - -- - -- RSA Keys and Signatures - -- - - -- arc for RSA public key and RSA signature OIDs - - pkcs-1 OBJECT IDENTIFIER ::= { - iso(1) member-body(2) us(840) rsadsi(113549) pkcs(1) 1 } - - -- OID for RSA public keys - - rsaEncryption OBJECT IDENTIFIER ::= { pkcs-1 1 } - - -- OID for RSA signature generated with MD2 hash - - md2WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 2 } - - -- OID for RSA signature generated with MD5 hash - - md5WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 4 } - - -- OID for RSA signature generated with SHA-1 hash - - sha1WithRSAEncryption OBJECT IDENTIFIER ::= { pkcs-1 5 } - - -- encoding for RSA public key - - RSAPublicKey ::= SEQUENCE { - modulus INTEGER, -- n - publicExponent INTEGER } -- e - - -- - -- Diffie-Hellman Keys - -- - - dhpublicnumber OBJECT IDENTIFIER ::= { - iso(1) member-body(2) us(840) ansi-x942(10046) - number-type(2) 1 } - - -- encoding for DSA public key - - DHPublicKey ::= INTEGER -- public key, y = g^x mod p - - DomainParameters ::= SEQUENCE { - p INTEGER, -- odd prime, p=jq +1 - g INTEGER, -- generator, g - q INTEGER, -- factor of p-1 - j INTEGER OPTIONAL, -- subgroup factor, j>= 2 - validationParms ValidationParms OPTIONAL } - - ValidationParms ::= SEQUENCE { - seed BIT STRING, - pgenCounter INTEGER } - - -- - -- KEA Keys - -- - - id-keyExchangeAlgorithm OBJECT IDENTIFIER ::= - { 2 16 840 1 101 2 1 1 22 } - - KEA-Parms-Id ::= OCTET STRING - - -- - -- Elliptic Curve Keys, Signatures, and Curves - -- - - ansi-X9-62 OBJECT IDENTIFIER ::= { - iso(1) member-body(2) us(840) 10045 } - - FieldID ::= SEQUENCE { -- Finite field - fieldType OBJECT IDENTIFIER, - parameters ANY DEFINED BY fieldType } - - -- Arc for ECDSA signature OIDS - - id-ecSigType OBJECT IDENTIFIER ::= { ansi-X9-62 signatures(4) } - - -- OID for ECDSA signatures with SHA-1 - - ecdsa-with-SHA1 OBJECT IDENTIFIER ::= { id-ecSigType 1 } - - -- OID for an elliptic curve signature - -- format for the value of an ECDSA signature value - - ECDSA-Sig-Value ::= SEQUENCE { - r INTEGER, - s INTEGER } - - -- recognized field type OIDs are defined in the following arc - - id-fieldType OBJECT IDENTIFIER ::= { ansi-X9-62 fieldType(1) } - - -- where fieldType is prime-field, the parameters are of type Prime-p - - prime-field OBJECT IDENTIFIER ::= { id-fieldType 1 } - - Prime-p ::= INTEGER -- Finite field F(p), where p is an odd prime - - -- where fieldType is characteristic-two-field, the parameters are - -- of type Characteristic-two - - characteristic-two-field OBJECT IDENTIFIER ::= { id-fieldType 2 } - - Characteristic-two ::= SEQUENCE { - m INTEGER, -- Field size 2^m - basis OBJECT IDENTIFIER, - parameters ANY DEFINED BY basis } - - -- recognized basis type OIDs are defined in the following arc - - id-characteristic-two-basis OBJECT IDENTIFIER ::= { - characteristic-two-field basisType(3) } - - -- gnbasis is identified by OID gnBasis and indicates - -- parameters are NULL - - gnBasis OBJECT IDENTIFIER ::= { id-characteristic-two-basis 1 } - - -- parameters for this basis are NULL - - -- trinomial basis is identified by OID tpBasis and indicates - -- parameters of type Pentanomial - - tpBasis OBJECT IDENTIFIER ::= { id-characteristic-two-basis 2 } - - -- Trinomial basis representation of F2^m - -- Integer k for reduction polynomial xm + xk + 1 - - Trinomial ::= INTEGER - - -- for pentanomial basis is identified by OID ppBasis and indicates - -- parameters of type Pentanomial - - ppBasis OBJECT IDENTIFIER ::= { id-characteristic-two-basis 3 } - - -- Pentanomial basis representation of F2^m - -- reduction polynomial integers k1, k2, k3 - -- f(x) = x**m + x**k3 + x**k2 + x**k1 + 1 - - Pentanomial ::= SEQUENCE { - k1 INTEGER, - k2 INTEGER, - k3 INTEGER } - - -- The object identifiers gnBasis, tpBasis and ppBasis name - -- three kinds of basis for characteristic-two finite fields - - FieldElement ::= OCTET STRING -- Finite field element - - ECPoint ::= OCTET STRING -- Elliptic curve point - - -- Elliptic Curve parameters may be specified explicitly, - -- specified implicitly through a "named curve", or - -- inherited from the CA - - EcpkParameters ::= CHOICE { - ecParameters ECParameters, - namedCurve OBJECT IDENTIFIER, - implicitlyCA NULL } - - ECParameters ::= SEQUENCE { -- Elliptic curve parameters - version ECPVer, - fieldID FieldID, - curve Curve, - base ECPoint, -- Base point G - order INTEGER, -- Order n of the base point - cofactor INTEGER OPTIONAL } -- The integer h = #E(Fq)/n - - ECPVer ::= INTEGER {ecpVer1(1)} - - Curve ::= SEQUENCE { - a FieldElement, -- Elliptic curve coefficient a - b FieldElement, -- Elliptic curve coefficient b - seed BIT STRING OPTIONAL } - - id-publicKeyType OBJECT IDENTIFIER ::= { ansi-X9-62 keyType(2) } - - id-ecPublicKey OBJECT IDENTIFIER ::= { id-publicKeyType 1 } - - -- Named Elliptic Curves in ANSI X9.62. - - ellipticCurve OBJECT IDENTIFIER ::= { ansi-X9-62 curves(3) } - - c-TwoCurve OBJECT IDENTIFIER ::= { - ellipticCurve characteristicTwo(0) } - - c2pnb163v1 OBJECT IDENTIFIER ::= { c-TwoCurve 1 } - c2pnb163v2 OBJECT IDENTIFIER ::= { c-TwoCurve 2 } - c2pnb163v3 OBJECT IDENTIFIER ::= { c-TwoCurve 3 } - c2pnb176w1 OBJECT IDENTIFIER ::= { c-TwoCurve 4 } - c2tnb191v1 OBJECT IDENTIFIER ::= { c-TwoCurve 5 } - c2tnb191v2 OBJECT IDENTIFIER ::= { c-TwoCurve 6 } - c2tnb191v3 OBJECT IDENTIFIER ::= { c-TwoCurve 7 } - c2onb191v4 OBJECT IDENTIFIER ::= { c-TwoCurve 8 } - c2onb191v5 OBJECT IDENTIFIER ::= { c-TwoCurve 9 } - c2pnb208w1 OBJECT IDENTIFIER ::= { c-TwoCurve 10 } - c2tnb239v1 OBJECT IDENTIFIER ::= { c-TwoCurve 11 } - c2tnb239v2 OBJECT IDENTIFIER ::= { c-TwoCurve 12 } - c2tnb239v3 OBJECT IDENTIFIER ::= { c-TwoCurve 13 } - c2onb239v4 OBJECT IDENTIFIER ::= { c-TwoCurve 14 } - c2onb239v5 OBJECT IDENTIFIER ::= { c-TwoCurve 15 } - c2pnb272w1 OBJECT IDENTIFIER ::= { c-TwoCurve 16 } - c2pnb304w1 OBJECT IDENTIFIER ::= { c-TwoCurve 17 } - c2tnb359v1 OBJECT IDENTIFIER ::= { c-TwoCurve 18 } - c2pnb368w1 OBJECT IDENTIFIER ::= { c-TwoCurve 19 } - c2tnb431r1 OBJECT IDENTIFIER ::= { c-TwoCurve 20 } - - primeCurve OBJECT IDENTIFIER ::= { ellipticCurve prime(1) } - - prime192v1 OBJECT IDENTIFIER ::= { primeCurve 1 } - prime192v2 OBJECT IDENTIFIER ::= { primeCurve 2 } - prime192v3 OBJECT IDENTIFIER ::= { primeCurve 3 } - prime239v1 OBJECT IDENTIFIER ::= { primeCurve 4 } - prime239v2 OBJECT IDENTIFIER ::= { primeCurve 5 } - prime239v3 OBJECT IDENTIFIER ::= { primeCurve 6 } - prime256v1 OBJECT IDENTIFIER ::= { primeCurve 7 } - - END diff --git a/lib/ssl/pkix/PKIX1Algorithms88.hrl b/lib/ssl/pkix/PKIX1Algorithms88.hrl deleted file mode 100644 index a11793618d..0000000000 --- a/lib/ssl/pkix/PKIX1Algorithms88.hrl +++ /dev/null @@ -1,94 +0,0 @@ -%% Generated by the Erlang ASN.1 compiler version:1.4.4.8 -%% Purpose: Erlang record definitions for each named and unnamed -%% SEQUENCE and SET, and macro definitions for each value -%% definition,in module PKIX1Algorithms88 - - - --record('Dss-Parms',{ -p, q, g}). - --record('Dss-Sig-Value',{ -r, s}). - --record('RSAPublicKey',{ -modulus, publicExponent}). - --record('DomainParameters',{ -p, g, q, j = asn1_NOVALUE, validationParms = asn1_NOVALUE}). - --record('ValidationParms',{ -seed, pgenCounter}). - --record('FieldID',{ -fieldType, parameters}). - --record('ECDSA-Sig-Value',{ -r, s}). - --record('Characteristic-two',{ -m, basis, parameters}). - --record('Pentanomial',{ -k1, k2, k3}). - --record('ECParameters',{ -version, fieldID, curve, base, order, cofactor = asn1_NOVALUE}). - --record('Curve',{ -a, b, seed = asn1_NOVALUE}). - --define('md2', {1,2,840,113549,2,2}). --define('md5', {1,2,840,113549,2,5}). --define('id-sha1', {1,3,14,3,2,26}). --define('id-dsa', {1,2,840,10040,4,1}). --define('id-dsa-with-sha1', {1,2,840,10040,4,3}). --define('pkcs-1', {1,2,840,113549,1,1}). --define('rsaEncryption', {1,2,840,113549,1,1,1}). --define('md2WithRSAEncryption', {1,2,840,113549,1,1,2}). --define('md5WithRSAEncryption', {1,2,840,113549,1,1,4}). --define('sha1WithRSAEncryption', {1,2,840,113549,1,1,5}). --define('dhpublicnumber', {1,2,840,10046,2,1}). --define('id-keyExchangeAlgorithm', {2,16,840,1,101,2,1,1,22}). --define('ansi-X9-62', {1,2,840,10045}). --define('id-ecSigType', {1,2,840,10045,4}). --define('ecdsa-with-SHA1', {1,2,840,10045,4,1}). --define('id-fieldType', {1,2,840,10045,1}). --define('prime-field', {1,2,840,10045,1,1}). --define('characteristic-two-field', {1,2,840,10045,1,2}). --define('id-characteristic-two-basis', {1,2,840,10045,1,2,3}). --define('gnBasis', {1,2,840,10045,1,2,3,1}). --define('tpBasis', {1,2,840,10045,1,2,3,2}). --define('ppBasis', {1,2,840,10045,1,2,3,3}). --define('id-publicKeyType', {1,2,840,10045,2}). --define('id-ecPublicKey', {1,2,840,10045,2,1}). --define('ellipticCurve', {1,2,840,10045,3}). --define('c-TwoCurve', {1,2,840,10045,3,0}). --define('c2pnb163v1', {1,2,840,10045,3,0,1}). --define('c2pnb163v2', {1,2,840,10045,3,0,2}). --define('c2pnb163v3', {1,2,840,10045,3,0,3}). --define('c2pnb176w1', {1,2,840,10045,3,0,4}). --define('c2tnb191v1', {1,2,840,10045,3,0,5}). --define('c2tnb191v2', {1,2,840,10045,3,0,6}). --define('c2tnb191v3', {1,2,840,10045,3,0,7}). --define('c2onb191v4', {1,2,840,10045,3,0,8}). --define('c2onb191v5', {1,2,840,10045,3,0,9}). --define('c2pnb208w1', {1,2,840,10045,3,0,10}). --define('c2tnb239v1', {1,2,840,10045,3,0,11}). --define('c2tnb239v2', {1,2,840,10045,3,0,12}). --define('c2tnb239v3', {1,2,840,10045,3,0,13}). --define('c2onb239v4', {1,2,840,10045,3,0,14}). --define('c2onb239v5', {1,2,840,10045,3,0,15}). --define('c2pnb272w1', {1,2,840,10045,3,0,16}). --define('c2pnb304w1', {1,2,840,10045,3,0,17}). --define('c2tnb359v1', {1,2,840,10045,3,0,18}). --define('c2pnb368w1', {1,2,840,10045,3,0,19}). --define('c2tnb431r1', {1,2,840,10045,3,0,20}). --define('primeCurve', {1,2,840,10045,3,1}). --define('prime192v1', {1,2,840,10045,3,1,1}). --define('prime192v2', {1,2,840,10045,3,1,2}). --define('prime192v3', {1,2,840,10045,3,1,3}). --define('prime239v1', {1,2,840,10045,3,1,4}). --define('prime239v2', {1,2,840,10045,3,1,5}). --define('prime239v3', {1,2,840,10045,3,1,6}). --define('prime256v1', {1,2,840,10045,3,1,7}). diff --git a/lib/ssl/pkix/PKIX1Explicit88.asn1 b/lib/ssl/pkix/PKIX1Explicit88.asn1 deleted file mode 100644 index 9b8068fed0..0000000000 --- a/lib/ssl/pkix/PKIX1Explicit88.asn1 +++ /dev/null @@ -1,619 +0,0 @@ -PKIX1Explicit88 { iso(1) identified-organization(3) dod(6) internet(1) - security(5) mechanisms(5) pkix(7) id-mod(0) id-pkix1-explicit(18) } - -DEFINITIONS EXPLICIT TAGS ::= - -BEGIN - --- EXPORTS ALL -- - --- IMPORTS NONE -- - --- UNIVERSAL Types defined in 1993 and 1998 ASN.1 --- and required by this specification - --- UniversalString ::= [UNIVERSAL 28] IMPLICIT OCTET STRING - -- UniversalString is defined in ASN.1:1993 - --- BMPString ::= [UNIVERSAL 30] IMPLICIT OCTET STRING - -- BMPString is the subtype of UniversalString and models - -- the Basic Multilingual Plane of ISO/IEC/ITU 10646-1 - --- UTF8String ::= [UNIVERSAL 12] IMPLICIT OCTET STRING - -- The content of this type conforms to RFC 2279. - --- PKIX specific OIDs - -id-pkix OBJECT IDENTIFIER ::= - { iso(1) identified-organization(3) dod(6) internet(1) - security(5) mechanisms(5) pkix(7) } - --- PKIX arcs - -id-pe OBJECT IDENTIFIER ::= { id-pkix 1 } - -- arc for private certificate extensions -id-qt OBJECT IDENTIFIER ::= { id-pkix 2 } - -- arc for policy qualifier types -id-kp OBJECT IDENTIFIER ::= { id-pkix 3 } - -- arc for extended key purpose OIDS -id-ad OBJECT IDENTIFIER ::= { id-pkix 48 } - -- arc for access descriptors - --- policyQualifierIds for Internet policy qualifiers - -id-qt-cps OBJECT IDENTIFIER ::= { id-qt 1 } - -- OID for CPS qualifier -id-qt-unotice OBJECT IDENTIFIER ::= { id-qt 2 } - -- OID for user notice qualifier - --- access descriptor definitions - -id-ad-ocsp OBJECT IDENTIFIER ::= { id-ad 1 } -id-ad-caIssuers OBJECT IDENTIFIER ::= { id-ad 2 } -id-ad-timeStamping OBJECT IDENTIFIER ::= { id-ad 3 } -id-ad-caRepository OBJECT IDENTIFIER ::= { id-ad 5 } - --- attribute data types - -Attribute ::= SEQUENCE { - type AttributeType, - values SET OF AttributeValue } - -- at least one value is required - -AttributeType ::= OBJECT IDENTIFIER - -AttributeValue ::= ANY - -AttributeTypeAndValue ::= SEQUENCE { - type AttributeType, - value AttributeValue } - --- suggested naming attributes: Definition of the following --- information object set may be augmented to meet local --- requirements. Note that deleting members of the set may --- prevent interoperability with conforming implementations. --- presented in pairs: the AttributeType followed by the --- type definition for the corresponding AttributeValue ---Arc for standard naming attributes -id-at OBJECT IDENTIFIER ::= { joint-iso-ccitt(2) ds(5) 4 } - --- Naming attributes of type X520name - -id-at-name AttributeType ::= { id-at 41 } -id-at-surname AttributeType ::= { id-at 4 } -id-at-givenName AttributeType ::= { id-at 42 } -id-at-initials AttributeType ::= { id-at 43 } -id-at-generationQualifier AttributeType ::= { id-at 44 } - -X520name ::= CHOICE { - teletexString TeletexString (SIZE (1..ub-name)), - printableString PrintableString (SIZE (1..ub-name)), - universalString UniversalString (SIZE (1..ub-name)), - utf8String UTF8String (SIZE (1..ub-name)), - bmpString BMPString (SIZE (1..ub-name)) } - --- Naming attributes of type X520CommonName - -id-at-commonName AttributeType ::= { id-at 3 } - -X520CommonName ::= CHOICE { - teletexString TeletexString (SIZE (1..ub-common-name)), - printableString PrintableString (SIZE (1..ub-common-name)), - universalString UniversalString (SIZE (1..ub-common-name)), - utf8String UTF8String (SIZE (1..ub-common-name)), - bmpString BMPString (SIZE (1..ub-common-name)) } - --- Naming attributes of type X520LocalityName - -id-at-localityName AttributeType ::= { id-at 7 } - -X520LocalityName ::= CHOICE { - teletexString TeletexString (SIZE (1..ub-locality-name)), - printableString PrintableString (SIZE (1..ub-locality-name)), - universalString UniversalString (SIZE (1..ub-locality-name)), - utf8String UTF8String (SIZE (1..ub-locality-name)), - bmpString BMPString (SIZE (1..ub-locality-name)) } - --- Naming attributes of type X520StateOrProvinceName - -id-at-stateOrProvinceName AttributeType ::= { id-at 8 } - -X520StateOrProvinceName ::= CHOICE { - teletexString TeletexString (SIZE (1..ub-state-name)), - printableString PrintableString (SIZE (1..ub-state-name)), - universalString UniversalString (SIZE (1..ub-state-name)), - utf8String UTF8String (SIZE (1..ub-state-name)), - bmpString BMPString (SIZE(1..ub-state-name)) } - --- Naming attributes of type X520OrganizationName - -id-at-organizationName AttributeType ::= { id-at 10 } - -X520OrganizationName ::= CHOICE { - teletexString TeletexString - (SIZE (1..ub-organization-name)), - printableString PrintableString - (SIZE (1..ub-organization-name)), - universalString UniversalString - (SIZE (1..ub-organization-name)), - utf8String UTF8String - (SIZE (1..ub-organization-name)), - bmpString BMPString - (SIZE (1..ub-organization-name)) } - --- Naming attributes of type X520OrganizationalUnitName - -id-at-organizationalUnitName AttributeType ::= { id-at 11 } - -X520OrganizationalUnitName ::= CHOICE { - teletexString TeletexString - (SIZE (1..ub-organizational-unit-name)), - printableString PrintableString - (SIZE (1..ub-organizational-unit-name)), - universalString UniversalString - (SIZE (1..ub-organizational-unit-name)), - utf8String UTF8String - (SIZE (1..ub-organizational-unit-name)), - bmpString BMPString - (SIZE (1..ub-organizational-unit-name)) } - --- Naming attributes of type X520Title - -id-at-title AttributeType ::= { id-at 12 } - -X520Title ::= CHOICE { - teletexString TeletexString (SIZE (1..ub-title)), - printableString PrintableString (SIZE (1..ub-title)), - universalString UniversalString (SIZE (1..ub-title)), - utf8String UTF8String (SIZE (1..ub-title)), - bmpString BMPString (SIZE (1..ub-title)) } - --- Naming attributes of type X520dnQualifier - -id-at-dnQualifier AttributeType ::= { id-at 46 } - -X520dnQualifier ::= PrintableString - --- Naming attributes of type X520countryName (digraph from IS 3166) - -id-at-countryName AttributeType ::= { id-at 6 } - -X520countryName ::= PrintableString (SIZE (2)) - --- Naming attributes of type X520SerialNumber - -id-at-serialNumber AttributeType ::= { id-at 5 } - -X520SerialNumber ::= PrintableString (SIZE (1..ub-serial-number)) - --- Naming attributes of type X520Pseudonym - -id-at-pseudonym AttributeType ::= { id-at 65 } - -X520Pseudonym ::= CHOICE { - teletexString TeletexString (SIZE (1..ub-pseudonym)), - printableString PrintableString (SIZE (1..ub-pseudonym)), - universalString UniversalString (SIZE (1..ub-pseudonym)), - utf8String UTF8String (SIZE (1..ub-pseudonym)), - bmpString BMPString (SIZE (1..ub-pseudonym)) } - --- Naming attributes of type DomainComponent (from RFC 2247) - -id-domainComponent AttributeType ::= - { 0 9 2342 19200300 100 1 25 } - -DomainComponent ::= IA5String - --- Legacy attributes - -pkcs-9 OBJECT IDENTIFIER ::= - { iso(1) member-body(2) us(840) rsadsi(113549) pkcs(1) 9 } - -id-emailAddress AttributeType ::= { pkcs-9 1 } - -EmailAddress ::= IA5String (SIZE (1..ub-emailaddress-length)) - --- naming data types -- - -Name ::= CHOICE { -- only one possibility for now -- - rdnSequence RDNSequence } - -RDNSequence ::= SEQUENCE OF RelativeDistinguishedName - -DistinguishedName ::= RDNSequence - -RelativeDistinguishedName ::= - SET SIZE (1 .. MAX) OF AttributeTypeAndValue - --- Directory string type -- - -DirectoryString ::= CHOICE { - teletexString TeletexString (SIZE (1..MAX)), - printableString PrintableString (SIZE (1..MAX)), - universalString UniversalString (SIZE (1..MAX)), - utf8String UTF8String (SIZE (1..MAX)), - bmpString BMPString (SIZE (1..MAX)) } - --- certificate and CRL specific structures begin here - -Certificate ::= SEQUENCE { - tbsCertificate TBSCertificate, - signatureAlgorithm AlgorithmIdentifier, - signature BIT STRING } - -TBSCertificate ::= SEQUENCE { - version [0] Version DEFAULT v1, - serialNumber CertificateSerialNumber, - signature AlgorithmIdentifier, - issuer Name, - validity Validity, - subject Name, - subjectPublicKeyInfo SubjectPublicKeyInfo, - issuerUniqueID [1] IMPLICIT UniqueIdentifier OPTIONAL, - -- If present, version MUST be v2 or v3 - subjectUniqueID [2] IMPLICIT UniqueIdentifier OPTIONAL, - -- If present, version MUST be v2 or v3 - extensions [3] Extensions OPTIONAL - -- If present, version MUST be v3 -- } - -Version ::= INTEGER { v1(0), v2(1), v3(2) } - -CertificateSerialNumber ::= INTEGER - -Validity ::= SEQUENCE { - notBefore Time, - notAfter Time } - -Time ::= CHOICE { - utcTime UTCTime, - generalTime GeneralizedTime } - -UniqueIdentifier ::= BIT STRING - -SubjectPublicKeyInfo ::= SEQUENCE { - algorithm AlgorithmIdentifier, - subjectPublicKey BIT STRING } - -Extensions ::= SEQUENCE SIZE (1..MAX) OF Extension - -Extension ::= SEQUENCE { - extnID OBJECT IDENTIFIER, - critical BOOLEAN DEFAULT FALSE, - extnValue OCTET STRING } - --- CRL structures - -CertificateList ::= SEQUENCE { - tbsCertList TBSCertList, - signatureAlgorithm AlgorithmIdentifier, - signature BIT STRING } - -TBSCertList ::= SEQUENCE { - version Version OPTIONAL, - -- if present, MUST be v2 - signature AlgorithmIdentifier, - issuer Name, - thisUpdate Time, - nextUpdate Time OPTIONAL, - revokedCertificates SEQUENCE OF SEQUENCE { - userCertificate CertificateSerialNumber, - revocationDate Time, - crlEntryExtensions Extensions OPTIONAL - -- if present, MUST be v2 - } OPTIONAL, - crlExtensions [0] Extensions OPTIONAL } - -- if present, MUST be v2 - --- Version, Time, CertificateSerialNumber, and Extensions were --- defined earlier for use in the certificate structure - -AlgorithmIdentifier ::= SEQUENCE { - algorithm OBJECT IDENTIFIER, - parameters ANY DEFINED BY algorithm OPTIONAL } - -- contains a value of the type - -- registered for use with the - -- algorithm object identifier value - --- X.400 address syntax starts here - -ORAddress ::= SEQUENCE { - built-in-standard-attributes BuiltInStandardAttributes, - built-in-domain-defined-attributes - BuiltInDomainDefinedAttributes OPTIONAL, - -- see also teletex-domain-defined-attributes - extension-attributes ExtensionAttributes OPTIONAL } - --- Built-in Standard Attributes - -BuiltInStandardAttributes ::= SEQUENCE { - country-name CountryName OPTIONAL, - administration-domain-name AdministrationDomainName OPTIONAL, - network-address [0] IMPLICIT NetworkAddress OPTIONAL, - -- see also extended-network-address - terminal-identifier [1] IMPLICIT TerminalIdentifier OPTIONAL, - private-domain-name [2] PrivateDomainName OPTIONAL, - organization-name [3] IMPLICIT OrganizationName OPTIONAL, - -- see also teletex-organization-name - numeric-user-identifier [4] IMPLICIT NumericUserIdentifier - OPTIONAL, - personal-name [5] IMPLICIT PersonalName OPTIONAL, - -- see also teletex-personal-name - organizational-unit-names [6] IMPLICIT OrganizationalUnitNames - OPTIONAL } - -- see also teletex-organizational-unit-names - -CountryName ::= [APPLICATION 1] CHOICE { - x121-dcc-code NumericString - (SIZE (ub-country-name-numeric-length)), - iso-3166-alpha2-code PrintableString - (SIZE (ub-country-name-alpha-length)) } - -AdministrationDomainName ::= [APPLICATION 2] CHOICE { - numeric NumericString (SIZE (0..ub-domain-name-length)), - printable PrintableString (SIZE (0..ub-domain-name-length)) } - -NetworkAddress ::= X121Address -- see also extended-network-address - -X121Address ::= NumericString (SIZE (1..ub-x121-address-length)) - -TerminalIdentifier ::= PrintableString (SIZE -(1..ub-terminal-id-length)) - -PrivateDomainName ::= CHOICE { - numeric NumericString (SIZE (1..ub-domain-name-length)), - printable PrintableString (SIZE (1..ub-domain-name-length)) } - -OrganizationName ::= PrintableString - (SIZE (1..ub-organization-name-length)) - -- see also teletex-organization-name - -NumericUserIdentifier ::= NumericString - (SIZE (1..ub-numeric-user-id-length)) - -PersonalName ::= SET { - surname [0] IMPLICIT PrintableString - (SIZE (1..ub-surname-length)), - given-name [1] IMPLICIT PrintableString - (SIZE (1..ub-given-name-length)) OPTIONAL, - initials [2] IMPLICIT PrintableString - (SIZE (1..ub-initials-length)) OPTIONAL, - generation-qualifier [3] IMPLICIT PrintableString - (SIZE (1..ub-generation-qualifier-length)) - OPTIONAL } - -- see also teletex-personal-name - -OrganizationalUnitNames ::= SEQUENCE SIZE (1..ub-organizational-units) - OF OrganizationalUnitName - -- see also teletex-organizational-unit-names - -OrganizationalUnitName ::= PrintableString (SIZE - (1..ub-organizational-unit-name-length)) - --- Built-in Domain-defined Attributes - -BuiltInDomainDefinedAttributes ::= SEQUENCE SIZE - (1..ub-domain-defined-attributes) OF - BuiltInDomainDefinedAttribute - -BuiltInDomainDefinedAttribute ::= SEQUENCE { - type PrintableString (SIZE - (1..ub-domain-defined-attribute-type-length)), - value PrintableString (SIZE - (1..ub-domain-defined-attribute-value-length)) } - --- Extension Attributes - -ExtensionAttributes ::= SET SIZE (1..ub-extension-attributes) OF - ExtensionAttribute - -ExtensionAttribute ::= SEQUENCE { - extension-attribute-type [0] IMPLICIT INTEGER - (0..ub-extension-attributes), - extension-attribute-value [1] - ANY DEFINED BY extension-attribute-type } - --- Extension types and attribute values - -common-name INTEGER ::= 1 - -CommonName ::= PrintableString (SIZE (1..ub-common-name-length)) - -teletex-common-name INTEGER ::= 2 - -TeletexCommonName ::= TeletexString (SIZE (1..ub-common-name-length)) - -teletex-organization-name INTEGER ::= 3 - -TeletexOrganizationName ::= - TeletexString (SIZE (1..ub-organization-name-length)) - -teletex-personal-name INTEGER ::= 4 - -TeletexPersonalName ::= SET { - surname [0] IMPLICIT TeletexString - (SIZE (1..ub-surname-length)), - given-name [1] IMPLICIT TeletexString - (SIZE (1..ub-given-name-length)) OPTIONAL, - initials [2] IMPLICIT TeletexString - (SIZE (1..ub-initials-length)) OPTIONAL, - generation-qualifier [3] IMPLICIT TeletexString - (SIZE (1..ub-generation-qualifier-length)) - OPTIONAL } - -teletex-organizational-unit-names INTEGER ::= 5 - -TeletexOrganizationalUnitNames ::= SEQUENCE SIZE - (1..ub-organizational-units) OF TeletexOrganizationalUnitName - -TeletexOrganizationalUnitName ::= TeletexString - (SIZE (1..ub-organizational-unit-name-length)) - -pds-name INTEGER ::= 7 - -PDSName ::= PrintableString (SIZE (1..ub-pds-name-length)) - -physical-delivery-country-name INTEGER ::= 8 - -PhysicalDeliveryCountryName ::= CHOICE { - x121-dcc-code NumericString (SIZE -(ub-country-name-numeric-length)), - iso-3166-alpha2-code PrintableString - (SIZE (ub-country-name-alpha-length)) } - -postal-code INTEGER ::= 9 - -PostalCode ::= CHOICE { - numeric-code NumericString (SIZE (1..ub-postal-code-length)), - printable-code PrintableString (SIZE (1..ub-postal-code-length)) } - -physical-delivery-office-name INTEGER ::= 10 - -PhysicalDeliveryOfficeName ::= PDSParameter - -physical-delivery-office-number INTEGER ::= 11 - -PhysicalDeliveryOfficeNumber ::= PDSParameter - -extension-OR-address-components INTEGER ::= 12 - -ExtensionORAddressComponents ::= PDSParameter - -physical-delivery-personal-name INTEGER ::= 13 - -PhysicalDeliveryPersonalName ::= PDSParameter - -physical-delivery-organization-name INTEGER ::= 14 - -PhysicalDeliveryOrganizationName ::= PDSParameter - -extension-physical-delivery-address-components INTEGER ::= 15 - -ExtensionPhysicalDeliveryAddressComponents ::= PDSParameter - -unformatted-postal-address INTEGER ::= 16 - -UnformattedPostalAddress ::= SET { - printable-address SEQUENCE SIZE (1..ub-pds-physical-address-lines) - OF PrintableString (SIZE (1..ub-pds-parameter-length)) - OPTIONAL, - teletex-string TeletexString - (SIZE (1..ub-unformatted-address-length)) OPTIONAL } - -street-address INTEGER ::= 17 - -StreetAddress ::= PDSParameter - -post-office-box-address INTEGER ::= 18 - -PostOfficeBoxAddress ::= PDSParameter - -poste-restante-address INTEGER ::= 19 - -PosteRestanteAddress ::= PDSParameter - -unique-postal-name INTEGER ::= 20 - -UniquePostalName ::= PDSParameter - -local-postal-attributes INTEGER ::= 21 - -LocalPostalAttributes ::= PDSParameter - -PDSParameter ::= SET { - printable-string PrintableString - (SIZE(1..ub-pds-parameter-length)) OPTIONAL, - teletex-string TeletexString - (SIZE(1..ub-pds-parameter-length)) OPTIONAL } - -extended-network-address INTEGER ::= 22 - -ExtendedNetworkAddress ::= CHOICE { - e163-4-address SEQUENCE { - number [0] IMPLICIT NumericString - (SIZE (1..ub-e163-4-number-length)), - sub-address [1] IMPLICIT NumericString - (SIZE (1..ub-e163-4-sub-address-length)) - OPTIONAL }, - psap-address [0] IMPLICIT PresentationAddress } - -PresentationAddress ::= SEQUENCE { - pSelector [0] EXPLICIT OCTET STRING OPTIONAL, - sSelector [1] EXPLICIT OCTET STRING OPTIONAL, - tSelector [2] EXPLICIT OCTET STRING OPTIONAL, - nAddresses [3] EXPLICIT SET SIZE (1..MAX) OF OCTET STRING } - -terminal-type INTEGER ::= 23 - -TerminalType ::= INTEGER { - telex (3), - teletex (4), - g3-facsimile (5), - g4-facsimile (6), - ia5-terminal (7), - videotex (8) } (0..ub-integer-options) - --- Extension Domain-defined Attributes - -teletex-domain-defined-attributes INTEGER ::= 6 - -TeletexDomainDefinedAttributes ::= SEQUENCE SIZE - (1..ub-domain-defined-attributes) OF TeletexDomainDefinedAttribute - -TeletexDomainDefinedAttribute ::= SEQUENCE { - type TeletexString - (SIZE (1..ub-domain-defined-attribute-type-length)), - value TeletexString - (SIZE (1..ub-domain-defined-attribute-value-length)) } - --- specifications of Upper Bounds MUST be regarded as mandatory --- from Annex B of ITU-T X.411 Reference Definition of MTS Parameter --- Upper Bounds - --- Upper Bounds -ub-name INTEGER ::= 32768 -ub-common-name INTEGER ::= 64 -ub-locality-name INTEGER ::= 128 -ub-state-name INTEGER ::= 128 -ub-organization-name INTEGER ::= 64 -ub-organizational-unit-name INTEGER ::= 64 -ub-title INTEGER ::= 64 -ub-serial-number INTEGER ::= 64 -ub-match INTEGER ::= 128 -ub-emailaddress-length INTEGER ::= 128 -ub-common-name-length INTEGER ::= 64 -ub-country-name-alpha-length INTEGER ::= 2 -ub-country-name-numeric-length INTEGER ::= 3 -ub-domain-defined-attributes INTEGER ::= 4 -ub-domain-defined-attribute-type-length INTEGER ::= 8 -ub-domain-defined-attribute-value-length INTEGER ::= 128 -ub-domain-name-length INTEGER ::= 16 -ub-extension-attributes INTEGER ::= 256 -ub-e163-4-number-length INTEGER ::= 15 -ub-e163-4-sub-address-length INTEGER ::= 40 -ub-generation-qualifier-length INTEGER ::= 3 -ub-given-name-length INTEGER ::= 16 -ub-initials-length INTEGER ::= 5 -ub-integer-options INTEGER ::= 256 -ub-numeric-user-id-length INTEGER ::= 32 -ub-organization-name-length INTEGER ::= 64 -ub-organizational-unit-name-length INTEGER ::= 32 -ub-organizational-units INTEGER ::= 4 -ub-pds-name-length INTEGER ::= 16 -ub-pds-parameter-length INTEGER ::= 30 -ub-pds-physical-address-lines INTEGER ::= 6 -ub-postal-code-length INTEGER ::= 16 -ub-pseudonym INTEGER ::= 128 -ub-surname-length INTEGER ::= 40 -ub-terminal-id-length INTEGER ::= 24 -ub-unformatted-address-length INTEGER ::= 180 -ub-x121-address-length INTEGER ::= 16 - --- Note - upper bounds on string types, such as TeletexString, are --- measured in characters. Excepting PrintableString or IA5String, a --- significantly greater number of octets will be required to hold --- such a value. As a minimum, 16 octets, or twice the specified --- upper bound, whichever is the larger, should be allowed for --- TeletexString. For UTF8String or UniversalString at least four --- times the upper bound should be allowed. - -END diff --git a/lib/ssl/pkix/PKIX1Explicit88.hrl b/lib/ssl/pkix/PKIX1Explicit88.hrl deleted file mode 100644 index 5940c1e245..0000000000 --- a/lib/ssl/pkix/PKIX1Explicit88.hrl +++ /dev/null @@ -1,163 +0,0 @@ -%% Generated by the Erlang ASN.1 compiler version:1.4.4.8 -%% Purpose: Erlang record definitions for each named and unnamed -%% SEQUENCE and SET, and macro definitions for each value -%% definition,in module PKIX1Explicit88 - - - --record('Attribute',{ -type, values}). - --record('AttributeTypeAndValue',{ -type, value}). - --record('Certificate',{ -tbsCertificate, signatureAlgorithm, signature}). - --record('TBSCertificate',{ -version = asn1_DEFAULT, serialNumber, signature, issuer, validity, subject, subjectPublicKeyInfo, issuerUniqueID = asn1_NOVALUE, subjectUniqueID = asn1_NOVALUE, extensions = asn1_NOVALUE}). - --record('Validity',{ -notBefore, notAfter}). - --record('SubjectPublicKeyInfo',{ -algorithm, subjectPublicKey}). - --record('Extension',{ -extnID, critical = asn1_DEFAULT, extnValue}). - --record('CertificateList',{ -tbsCertList, signatureAlgorithm, signature}). - --record('TBSCertList',{ -version = asn1_NOVALUE, signature, issuer, thisUpdate, nextUpdate = asn1_NOVALUE, revokedCertificates = asn1_NOVALUE, crlExtensions = asn1_NOVALUE}). - --record('TBSCertList_revokedCertificates_SEQOF',{ -userCertificate, revocationDate, crlEntryExtensions = asn1_NOVALUE}). - --record('AlgorithmIdentifier',{ -algorithm, parameters = asn1_NOVALUE}). - --record('ORAddress',{ -'built-in-standard-attributes', 'built-in-domain-defined-attributes' = asn1_NOVALUE, 'extension-attributes' = asn1_NOVALUE}). - --record('BuiltInStandardAttributes',{ -'country-name' = asn1_NOVALUE, 'administration-domain-name' = asn1_NOVALUE, 'network-address' = asn1_NOVALUE, 'terminal-identifier' = asn1_NOVALUE, 'private-domain-name' = asn1_NOVALUE, 'organization-name' = asn1_NOVALUE, 'numeric-user-identifier' = asn1_NOVALUE, 'personal-name' = asn1_NOVALUE, 'organizational-unit-names' = asn1_NOVALUE}). - --record('PersonalName',{ -surname, 'given-name' = asn1_NOVALUE, initials = asn1_NOVALUE, 'generation-qualifier' = asn1_NOVALUE}). - --record('BuiltInDomainDefinedAttribute',{ -type, value}). - --record('ExtensionAttribute',{ -'extension-attribute-type', 'extension-attribute-value'}). - --record('TeletexPersonalName',{ -surname, 'given-name' = asn1_NOVALUE, initials = asn1_NOVALUE, 'generation-qualifier' = asn1_NOVALUE}). - --record('UnformattedPostalAddress',{ -'printable-address' = asn1_NOVALUE, 'teletex-string' = asn1_NOVALUE}). - --record('PDSParameter',{ -'printable-string' = asn1_NOVALUE, 'teletex-string' = asn1_NOVALUE}). - --record('ExtendedNetworkAddress_e163-4-address',{ -number, 'sub-address' = asn1_NOVALUE}). - --record('PresentationAddress',{ -pSelector = asn1_NOVALUE, sSelector = asn1_NOVALUE, tSelector = asn1_NOVALUE, nAddresses}). - --record('TeletexDomainDefinedAttribute',{ -type, value}). - --define('id-pkix', {1,3,6,1,5,5,7}). --define('id-pe', {1,3,6,1,5,5,7,1}). --define('id-qt', {1,3,6,1,5,5,7,2}). --define('id-kp', {1,3,6,1,5,5,7,3}). --define('id-ad', {1,3,6,1,5,5,7,48}). --define('id-qt-cps', {1,3,6,1,5,5,7,2,1}). --define('id-qt-unotice', {1,3,6,1,5,5,7,2,2}). --define('id-ad-ocsp', {1,3,6,1,5,5,7,48,1}). --define('id-ad-caIssuers', {1,3,6,1,5,5,7,48,2}). --define('id-ad-timeStamping', {1,3,6,1,5,5,7,48,3}). --define('id-ad-caRepository', {1,3,6,1,5,5,7,48,5}). --define('id-at', {2,5,4}). --define('id-at-name', {2,5,4,41}). --define('id-at-surname', {2,5,4,4}). --define('id-at-givenName', {2,5,4,42}). --define('id-at-initials', {2,5,4,43}). --define('id-at-generationQualifier', {2,5,4,44}). --define('id-at-commonName', {2,5,4,3}). --define('id-at-localityName', {2,5,4,7}). --define('id-at-stateOrProvinceName', {2,5,4,8}). --define('id-at-organizationName', {2,5,4,10}). --define('id-at-organizationalUnitName', {2,5,4,11}). --define('id-at-title', {2,5,4,12}). --define('id-at-dnQualifier', {2,5,4,46}). --define('id-at-countryName', {2,5,4,6}). --define('id-at-serialNumber', {2,5,4,5}). --define('id-at-pseudonym', {2,5,4,65}). --define('id-domainComponent', {0,9,2342,19200300,100,1,25}). --define('pkcs-9', {1,2,840,113549,1,9}). --define('id-emailAddress', {1,2,840,113549,1,9,1}). --define('common-name', 1). --define('teletex-common-name', 2). --define('teletex-organization-name', 3). --define('teletex-personal-name', 4). --define('teletex-organizational-unit-names', 5). --define('pds-name', 7). --define('physical-delivery-country-name', 8). --define('postal-code', 9). --define('physical-delivery-office-name', 10). --define('physical-delivery-office-number', 11). --define('extension-OR-address-components', 12). --define('physical-delivery-personal-name', 13). --define('physical-delivery-organization-name', 14). --define('extension-physical-delivery-address-components', 15). --define('unformatted-postal-address', 16). --define('street-address', 17). --define('post-office-box-address', 18). --define('poste-restante-address', 19). --define('unique-postal-name', 20). --define('local-postal-attributes', 21). --define('extended-network-address', 22). --define('terminal-type', 23). --define('teletex-domain-defined-attributes', 6). --define('ub-name', 32768). --define('ub-common-name', 64). --define('ub-locality-name', 128). --define('ub-state-name', 128). --define('ub-organization-name', 64). --define('ub-organizational-unit-name', 64). --define('ub-title', 64). --define('ub-serial-number', 64). --define('ub-match', 128). --define('ub-emailaddress-length', 128). --define('ub-common-name-length', 64). --define('ub-country-name-alpha-length', 2). --define('ub-country-name-numeric-length', 3). --define('ub-domain-defined-attributes', 4). --define('ub-domain-defined-attribute-type-length', 8). --define('ub-domain-defined-attribute-value-length', 128). --define('ub-domain-name-length', 16). --define('ub-extension-attributes', 256). --define('ub-e163-4-number-length', 15). --define('ub-e163-4-sub-address-length', 40). --define('ub-generation-qualifier-length', 3). --define('ub-given-name-length', 16). --define('ub-initials-length', 5). --define('ub-integer-options', 256). --define('ub-numeric-user-id-length', 32). --define('ub-organization-name-length', 64). --define('ub-organizational-unit-name-length', 32). --define('ub-organizational-units', 4). --define('ub-pds-name-length', 16). --define('ub-pds-parameter-length', 30). --define('ub-pds-physical-address-lines', 6). --define('ub-postal-code-length', 16). --define('ub-pseudonym', 128). --define('ub-surname-length', 40). --define('ub-terminal-id-length', 24). --define('ub-unformatted-address-length', 180). --define('ub-x121-address-length', 16). diff --git a/lib/ssl/pkix/PKIX1Implicit88.asn1 b/lib/ssl/pkix/PKIX1Implicit88.asn1 deleted file mode 100644 index ced270baf6..0000000000 --- a/lib/ssl/pkix/PKIX1Implicit88.asn1 +++ /dev/null @@ -1,349 +0,0 @@ -PKIX1Implicit88 { iso(1) identified-organization(3) dod(6) internet(1) - security(5) mechanisms(5) pkix(7) id-mod(0) id-pkix1-implicit(19) } - -DEFINITIONS IMPLICIT TAGS ::= - -BEGIN - --- EXPORTS ALL -- - -IMPORTS - id-pe, id-kp, id-qt-unotice, id-qt-cps, - -- delete following line if "new" types are supported -- - -- BMPString, - -- UTF8String, end "new" types -- - ORAddress, Name, RelativeDistinguishedName, - CertificateSerialNumber, Attribute, DirectoryString - FROM PKIX1Explicit88 { iso(1) identified-organization(3) - dod(6) internet(1) security(5) mechanisms(5) pkix(7) - id-mod(0) id-pkix1-explicit(18) }; - - --- ISO arc for standard certificate and CRL extensions - -id-ce OBJECT IDENTIFIER ::= {joint-iso-ccitt(2) ds(5) 29} - --- authority key identifier OID and syntax - -id-ce-authorityKeyIdentifier OBJECT IDENTIFIER ::= { id-ce 35 } - -AuthorityKeyIdentifier ::= SEQUENCE { - keyIdentifier [0] KeyIdentifier OPTIONAL, - authorityCertIssuer [1] GeneralNames OPTIONAL, - authorityCertSerialNumber [2] CertificateSerialNumber OPTIONAL } - -- authorityCertIssuer and authorityCertSerialNumber MUST both - -- be present or both be absent - -KeyIdentifier ::= OCTET STRING - --- subject key identifier OID and syntax - -id-ce-subjectKeyIdentifier OBJECT IDENTIFIER ::= { id-ce 14 } - -SubjectKeyIdentifier ::= KeyIdentifier - --- key usage extension OID and syntax - -id-ce-keyUsage OBJECT IDENTIFIER ::= { id-ce 15 } - -KeyUsage ::= BIT STRING { - digitalSignature (0), - nonRepudiation (1), - keyEncipherment (2), - dataEncipherment (3), - keyAgreement (4), - keyCertSign (5), - cRLSign (6), - encipherOnly (7), - decipherOnly (8) } - --- private key usage period extension OID and syntax - -id-ce-privateKeyUsagePeriod OBJECT IDENTIFIER ::= { id-ce 16 } - -PrivateKeyUsagePeriod ::= SEQUENCE { - notBefore [0] GeneralizedTime OPTIONAL, - notAfter [1] GeneralizedTime OPTIONAL } - -- either notBefore or notAfter MUST be present - --- certificate policies extension OID and syntax - -id-ce-certificatePolicies OBJECT IDENTIFIER ::= { id-ce 32 } - -anyPolicy OBJECT IDENTIFIER ::= { id-ce-certificatePolicies 0 } - -CertificatePolicies ::= SEQUENCE SIZE (1..MAX) OF PolicyInformation - -PolicyInformation ::= SEQUENCE { - policyIdentifier CertPolicyId, - policyQualifiers SEQUENCE SIZE (1..MAX) OF - PolicyQualifierInfo OPTIONAL } - -CertPolicyId ::= OBJECT IDENTIFIER - -PolicyQualifierInfo ::= SEQUENCE { - policyQualifierId PolicyQualifierId, - qualifier ANY DEFINED BY policyQualifierId } - --- Implementations that recognize additional policy qualifiers MUST --- augment the following definition for PolicyQualifierId - -PolicyQualifierId ::= - OBJECT IDENTIFIER ( id-qt-cps | id-qt-unotice ) - --- CPS pointer qualifier - -CPSuri ::= IA5String - --- user notice qualifier - -UserNotice ::= SEQUENCE { - noticeRef NoticeReference OPTIONAL, - explicitText DisplayText OPTIONAL} - -NoticeReference ::= SEQUENCE { - organization DisplayText, - noticeNumbers SEQUENCE OF INTEGER } - -DisplayText ::= CHOICE { - ia5String IA5String (SIZE (1..200)), - visibleString VisibleString (SIZE (1..200)), - bmpString BMPString (SIZE (1..200)), - utf8String UTF8String (SIZE (1..200)) } - --- policy mapping extension OID and syntax - -id-ce-policyMappings OBJECT IDENTIFIER ::= { id-ce 33 } - -PolicyMappings ::= SEQUENCE SIZE (1..MAX) OF SEQUENCE { - issuerDomainPolicy CertPolicyId, - subjectDomainPolicy CertPolicyId } - --- subject alternative name extension OID and syntax - -id-ce-subjectAltName OBJECT IDENTIFIER ::= { id-ce 17 } - -SubjectAltName ::= GeneralNames - -GeneralNames ::= SEQUENCE SIZE (1..MAX) OF GeneralName - -GeneralName ::= CHOICE { - otherName [0] AnotherName, - rfc822Name [1] IA5String, - dNSName [2] IA5String, - x400Address [3] ORAddress, - directoryName [4] Name, - ediPartyName [5] EDIPartyName, - uniformResourceIdentifier [6] IA5String, - iPAddress [7] OCTET STRING, - registeredID [8] OBJECT IDENTIFIER } - --- AnotherName replaces OTHER-NAME ::= TYPE-IDENTIFIER, as --- TYPE-IDENTIFIER is not supported in the '88 ASN.1 syntax - -AnotherName ::= SEQUENCE { - type-id OBJECT IDENTIFIER, - value [0] EXPLICIT ANY DEFINED BY type-id } - -EDIPartyName ::= SEQUENCE { - nameAssigner [0] DirectoryString OPTIONAL, - partyName [1] DirectoryString } - --- issuer alternative name extension OID and syntax - -id-ce-issuerAltName OBJECT IDENTIFIER ::= { id-ce 18 } - -IssuerAltName ::= GeneralNames - -id-ce-subjectDirectoryAttributes OBJECT IDENTIFIER ::= { id-ce 9 } - -SubjectDirectoryAttributes ::= SEQUENCE SIZE (1..MAX) OF Attribute - --- basic constraints extension OID and syntax - -id-ce-basicConstraints OBJECT IDENTIFIER ::= { id-ce 19 } - -BasicConstraints ::= SEQUENCE { - cA BOOLEAN DEFAULT FALSE, - pathLenConstraint INTEGER (0..MAX) OPTIONAL } - --- name constraints extension OID and syntax - -id-ce-nameConstraints OBJECT IDENTIFIER ::= { id-ce 30 } - -NameConstraints ::= SEQUENCE { - permittedSubtrees [0] GeneralSubtrees OPTIONAL, - excludedSubtrees [1] GeneralSubtrees OPTIONAL } - -GeneralSubtrees ::= SEQUENCE SIZE (1..MAX) OF GeneralSubtree - -GeneralSubtree ::= SEQUENCE { - base GeneralName, - minimum [0] BaseDistance DEFAULT 0, - maximum [1] BaseDistance OPTIONAL } - -BaseDistance ::= INTEGER (0..MAX) - --- policy constraints extension OID and syntax - -id-ce-policyConstraints OBJECT IDENTIFIER ::= { id-ce 36 } - -PolicyConstraints ::= SEQUENCE { - requireExplicitPolicy [0] SkipCerts OPTIONAL, - inhibitPolicyMapping [1] SkipCerts OPTIONAL } - -SkipCerts ::= INTEGER (0..MAX) - --- CRL distribution points extension OID and syntax - -id-ce-cRLDistributionPoints OBJECT IDENTIFIER ::= {id-ce 31} - -CRLDistributionPoints ::= SEQUENCE SIZE (1..MAX) OF DistributionPoint - -DistributionPoint ::= SEQUENCE { - distributionPoint [0] DistributionPointName OPTIONAL, - reasons [1] ReasonFlags OPTIONAL, - cRLIssuer [2] GeneralNames OPTIONAL } - -DistributionPointName ::= CHOICE { - fullName [0] GeneralNames, - nameRelativeToCRLIssuer [1] RelativeDistinguishedName } - -ReasonFlags ::= BIT STRING { - unused (0), - keyCompromise (1), - cACompromise (2), - affiliationChanged (3), - superseded (4), - cessationOfOperation (5), - certificateHold (6), - privilegeWithdrawn (7), - aACompromise (8) } - --- extended key usage extension OID and syntax - -id-ce-extKeyUsage OBJECT IDENTIFIER ::= {id-ce 37} - -ExtKeyUsageSyntax ::= SEQUENCE SIZE (1..MAX) OF KeyPurposeId - - -KeyPurposeId ::= OBJECT IDENTIFIER - --- permit unspecified key uses - -anyExtendedKeyUsage OBJECT IDENTIFIER ::= { id-ce-extKeyUsage 0 } - --- extended key purpose OIDs - -id-kp-serverAuth OBJECT IDENTIFIER ::= { id-kp 1 } -id-kp-clientAuth OBJECT IDENTIFIER ::= { id-kp 2 } -id-kp-codeSigning OBJECT IDENTIFIER ::= { id-kp 3 } -id-kp-emailProtection OBJECT IDENTIFIER ::= { id-kp 4 } -id-kp-timeStamping OBJECT IDENTIFIER ::= { id-kp 8 } -id-kp-OCSPSigning OBJECT IDENTIFIER ::= { id-kp 9 } - --- inhibit any policy OID and syntax - -id-ce-inhibitAnyPolicy OBJECT IDENTIFIER ::= { id-ce 54 } - -InhibitAnyPolicy ::= SkipCerts - --- freshest (delta)CRL extension OID and syntax - -id-ce-freshestCRL OBJECT IDENTIFIER ::= { id-ce 46 } - -FreshestCRL ::= CRLDistributionPoints - --- authority info access - -id-pe-authorityInfoAccess OBJECT IDENTIFIER ::= { id-pe 1 } - -AuthorityInfoAccessSyntax ::= - SEQUENCE SIZE (1..MAX) OF AccessDescription - -AccessDescription ::= SEQUENCE { - accessMethod OBJECT IDENTIFIER, - accessLocation GeneralName } - --- subject info access - -id-pe-subjectInfoAccess OBJECT IDENTIFIER ::= { id-pe 11 } - -SubjectInfoAccessSyntax ::= - SEQUENCE SIZE (1..MAX) OF AccessDescription - --- CRL number extension OID and syntax - -id-ce-cRLNumber OBJECT IDENTIFIER ::= { id-ce 20 } - -CRLNumber ::= INTEGER (0..MAX) - --- issuing distribution point extension OID and syntax - -id-ce-issuingDistributionPoint OBJECT IDENTIFIER ::= { id-ce 28 } - -IssuingDistributionPoint ::= SEQUENCE { - distributionPoint [0] DistributionPointName OPTIONAL, - onlyContainsUserCerts [1] BOOLEAN DEFAULT FALSE, - onlyContainsCACerts [2] BOOLEAN DEFAULT FALSE, - onlySomeReasons [3] ReasonFlags OPTIONAL, - indirectCRL [4] BOOLEAN DEFAULT FALSE, - onlyContainsAttributeCerts [5] BOOLEAN DEFAULT FALSE } - -id-ce-deltaCRLIndicator OBJECT IDENTIFIER ::= { id-ce 27 } - -BaseCRLNumber ::= CRLNumber - --- CRL reasons extension OID and syntax - -id-ce-cRLReasons OBJECT IDENTIFIER ::= { id-ce 21 } - -CRLReason ::= ENUMERATED { - unspecified (0), - keyCompromise (1), - cACompromise (2), - affiliationChanged (3), - superseded (4), - cessationOfOperation (5), - certificateHold (6), - removeFromCRL (8), - privilegeWithdrawn (9), - aACompromise (10) } - --- certificate issuer CRL entry extension OID and syntax - -id-ce-certificateIssuer OBJECT IDENTIFIER ::= { id-ce 29 } - -CertificateIssuer ::= GeneralNames - --- hold instruction extension OID and syntax - -id-ce-holdInstructionCode OBJECT IDENTIFIER ::= { id-ce 23 } - -HoldInstructionCode ::= OBJECT IDENTIFIER - --- ANSI x9 holdinstructions - --- ANSI x9 arc holdinstruction arc - -holdInstruction OBJECT IDENTIFIER ::= - {joint-iso-itu-t(2) member-body(2) us(840) x9cm(10040) 2} - --- ANSI X9 holdinstructions referenced by this standard - -id-holdinstruction-none OBJECT IDENTIFIER ::= - {holdInstruction 1} -- deprecated - -id-holdinstruction-callissuer OBJECT IDENTIFIER ::= - {holdInstruction 2} - -id-holdinstruction-reject OBJECT IDENTIFIER ::= - {holdInstruction 3} - --- invalidity date CRL entry extension OID and syntax - -id-ce-invalidityDate OBJECT IDENTIFIER ::= { id-ce 24 } - -InvalidityDate ::= GeneralizedTime - -END diff --git a/lib/ssl/pkix/PKIX1Implicit88.hrl b/lib/ssl/pkix/PKIX1Implicit88.hrl deleted file mode 100644 index 8fa1836284..0000000000 --- a/lib/ssl/pkix/PKIX1Implicit88.hrl +++ /dev/null @@ -1,93 +0,0 @@ -%% Generated by the Erlang ASN.1 compiler version:1.4.4.8 -%% Purpose: Erlang record definitions for each named and unnamed -%% SEQUENCE and SET, and macro definitions for each value -%% definition,in module PKIX1Implicit88 - - - --record('AuthorityKeyIdentifier',{ -keyIdentifier = asn1_NOVALUE, authorityCertIssuer = asn1_NOVALUE, authorityCertSerialNumber = asn1_NOVALUE}). - --record('PrivateKeyUsagePeriod',{ -notBefore = asn1_NOVALUE, notAfter = asn1_NOVALUE}). - --record('PolicyInformation',{ -policyIdentifier, policyQualifiers = asn1_NOVALUE}). - --record('PolicyQualifierInfo',{ -policyQualifierId, qualifier}). - --record('UserNotice',{ -noticeRef = asn1_NOVALUE, explicitText = asn1_NOVALUE}). - --record('NoticeReference',{ -organization, noticeNumbers}). - --record('PolicyMappings_SEQOF',{ -issuerDomainPolicy, subjectDomainPolicy}). - --record('AnotherName',{ -'type-id', value}). - --record('EDIPartyName',{ -nameAssigner = asn1_NOVALUE, partyName}). - --record('BasicConstraints',{ -cA = asn1_DEFAULT, pathLenConstraint = asn1_NOVALUE}). - --record('NameConstraints',{ -permittedSubtrees = asn1_NOVALUE, excludedSubtrees = asn1_NOVALUE}). - --record('GeneralSubtree',{ -base, minimum = asn1_DEFAULT, maximum = asn1_NOVALUE}). - --record('PolicyConstraints',{ -requireExplicitPolicy = asn1_NOVALUE, inhibitPolicyMapping = asn1_NOVALUE}). - --record('DistributionPoint',{ -distributionPoint = asn1_NOVALUE, reasons = asn1_NOVALUE, cRLIssuer = asn1_NOVALUE}). - --record('AccessDescription',{ -accessMethod, accessLocation}). - --record('IssuingDistributionPoint',{ -distributionPoint = asn1_NOVALUE, onlyContainsUserCerts = asn1_DEFAULT, onlyContainsCACerts = asn1_DEFAULT, onlySomeReasons = asn1_NOVALUE, indirectCRL = asn1_DEFAULT, onlyContainsAttributeCerts = asn1_DEFAULT}). - --define('id-ce', {2,5,29}). --define('id-ce-authorityKeyIdentifier', {2,5,29,35}). --define('id-ce-subjectKeyIdentifier', {2,5,29,14}). --define('id-ce-keyUsage', {2,5,29,15}). --define('id-ce-privateKeyUsagePeriod', {2,5,29,16}). --define('id-ce-certificatePolicies', {2,5,29,32}). --define('anyPolicy', {2,5,29,32,0}). --define('id-ce-policyMappings', {2,5,29,33}). --define('id-ce-subjectAltName', {2,5,29,17}). --define('id-ce-issuerAltName', {2,5,29,18}). --define('id-ce-subjectDirectoryAttributes', {2,5,29,9}). --define('id-ce-basicConstraints', {2,5,29,19}). --define('id-ce-nameConstraints', {2,5,29,30}). --define('id-ce-policyConstraints', {2,5,29,36}). --define('id-ce-cRLDistributionPoints', {2,5,29,31}). --define('id-ce-extKeyUsage', {2,5,29,37}). --define('anyExtendedKeyUsage', {2,5,29,37,0}). --define('id-kp-serverAuth', {1,3,6,1,5,5,7,3,1}). --define('id-kp-clientAuth', {1,3,6,1,5,5,7,3,2}). --define('id-kp-codeSigning', {1,3,6,1,5,5,7,3,3}). --define('id-kp-emailProtection', {1,3,6,1,5,5,7,3,4}). --define('id-kp-timeStamping', {1,3,6,1,5,5,7,3,8}). --define('id-kp-OCSPSigning', {1,3,6,1,5,5,7,3,9}). --define('id-ce-inhibitAnyPolicy', {2,5,29,54}). --define('id-ce-freshestCRL', {2,5,29,46}). --define('id-pe-authorityInfoAccess', {1,3,6,1,5,5,7,1,1}). --define('id-pe-subjectInfoAccess', {1,3,6,1,5,5,7,1,11}). --define('id-ce-cRLNumber', {2,5,29,20}). --define('id-ce-issuingDistributionPoint', {2,5,29,28}). --define('id-ce-deltaCRLIndicator', {2,5,29,27}). --define('id-ce-cRLReasons', {2,5,29,21}). --define('id-ce-certificateIssuer', {2,5,29,29}). --define('id-ce-holdInstructionCode', {2,5,29,23}). --define('holdInstruction', {2,2,840,10040,2}). --define('id-holdinstruction-none', {2,2,840,10040,2,1}). --define('id-holdinstruction-callissuer', {2,2,840,10040,2,2}). --define('id-holdinstruction-reject', {2,2,840,10040,2,3}). --define('id-ce-invalidityDate', {2,5,29,24}). diff --git a/lib/ssl/pkix/PKIXAttributeCertificate.asn1 b/lib/ssl/pkix/PKIXAttributeCertificate.asn1 deleted file mode 100644 index 7d93e6b37e..0000000000 --- a/lib/ssl/pkix/PKIXAttributeCertificate.asn1 +++ /dev/null @@ -1,189 +0,0 @@ - PKIXAttributeCertificate {iso(1) identified-organization(3) dod(6) - internet(1) security(5) mechanisms(5) pkix(7) id-mod(0) - id-mod-attribute-cert(12)} - - DEFINITIONS IMPLICIT TAGS ::= - - BEGIN - - -- EXPORTS ALL -- - - IMPORTS - - -- IMPORTed module OIDs MAY change if [PKIXPROF] changes - -- PKIX Certificate Extensions - Attribute, AlgorithmIdentifier, CertificateSerialNumber, - Extensions, UniqueIdentifier, - id-pkix, id-pe, id-kp, id-ad, id-at - FROM PKIX1Explicit88 {iso(1) identified-organization(3) - dod(6) internet(1) security(5) mechanisms(5) - pkix(7) id-mod(0) id-pkix1-explicit-88(1)} - - GeneralName, GeneralNames, id-ce - FROM PKIX1Implicit88 {iso(1) identified-organization(3) - dod(6) internet(1) security(5) mechanisms(5) - pkix(7) id-mod(0) id-pkix1-implicit-88(2)} ; - - id-pe-ac-auditIdentity OBJECT IDENTIFIER ::= { id-pe 4 } - id-pe-aaControls OBJECT IDENTIFIER ::= { id-pe 6 } - id-pe-ac-proxying OBJECT IDENTIFIER ::= { id-pe 10 } - id-ce-targetInformation OBJECT IDENTIFIER ::= { id-ce 55 } - - id-aca OBJECT IDENTIFIER ::= { id-pkix 10 } - id-aca-authenticationInfo OBJECT IDENTIFIER ::= { id-aca 1 } - id-aca-accessIdentity OBJECT IDENTIFIER ::= { id-aca 2 } - id-aca-chargingIdentity OBJECT IDENTIFIER ::= { id-aca 3 } - id-aca-group OBJECT IDENTIFIER ::= { id-aca 4 } - -- { id-aca 5 } is reserved - id-aca-encAttrs OBJECT IDENTIFIER ::= { id-aca 6 } - - id-at-role OBJECT IDENTIFIER ::= { id-at 72} - id-at-clearance OBJECT IDENTIFIER ::= - { joint-iso-ccitt(2) ds(5) module(1) - selected-attribute-types(5) clearance (55) } - - -- Uncomment this if using a 1988 level ASN.1 compiler - -- UTF8String ::= [UNIVERSAL 12] IMPLICIT OCTET STRING - - AttributeCertificate ::= SEQUENCE { - acinfo AttributeCertificateInfo, - signatureAlgorithm AlgorithmIdentifier, - signatureValue BIT STRING - } - - AttributeCertificateInfo ::= SEQUENCE { - version AttCertVersion, -- version is v2 - holder Holder, - issuer AttCertIssuer, - signature AlgorithmIdentifier, - serialNumber CertificateSerialNumber, - attrCertValidityPeriod AttCertValidityPeriod, - attributes SEQUENCE OF Attribute, - issuerUniqueID UniqueIdentifier OPTIONAL, - extensions Extensions OPTIONAL - } - - AttCertVersion ::= INTEGER { v2(1) } - - Holder ::= SEQUENCE { - baseCertificateID [0] IssuerSerial OPTIONAL, - -- the issuer and serial number of - -- the holder's Public Key Certificate - entityName [1] GeneralNames OPTIONAL, - -- the name of the claimant or role - objectDigestInfo [2] ObjectDigestInfo OPTIONAL - -- used to directly authenticate the - -- holder, for example, an executable - } - - ObjectDigestInfo ::= SEQUENCE { - digestedObjectType ENUMERATED { - publicKey (0), - publicKeyCert (1), - otherObjectTypes (2) }, - -- otherObjectTypes MUST NOT - -- MUST NOT be used in this profile - otherObjectTypeID OBJECT IDENTIFIER OPTIONAL, - digestAlgorithm AlgorithmIdentifier, - objectDigest BIT STRING - } - - AttCertIssuer ::= CHOICE { - v1Form GeneralNames, -- MUST NOT be used in this - -- profile - v2Form [0] V2Form -- v2 only - } - - V2Form ::= SEQUENCE { - issuerName GeneralNames OPTIONAL, - baseCertificateID [0] IssuerSerial OPTIONAL, - objectDigestInfo [1] ObjectDigestInfo OPTIONAL - -- issuerName MUST be present in this profile - -- baseCertificateID and objectDigestInfo MUST - -- NOT be present in this profile - } - - IssuerSerial ::= SEQUENCE { - issuer GeneralNames, - serial CertificateSerialNumber, - issuerUID UniqueIdentifier OPTIONAL - } - - AttCertValidityPeriod ::= SEQUENCE { - notBeforeTime GeneralizedTime, - notAfterTime GeneralizedTime - } - - Targets ::= SEQUENCE OF Target - - Target ::= CHOICE { - targetName [0] GeneralName, - targetGroup [1] GeneralName, - targetCert [2] TargetCert - } - - TargetCert ::= SEQUENCE { - targetCertificate IssuerSerial, - targetName GeneralName OPTIONAL, - certDigestInfo ObjectDigestInfo OPTIONAL - } - - IetfAttrSyntax ::= SEQUENCE { - policyAuthority[0] GeneralNames OPTIONAL, - values SEQUENCE OF CHOICE { - octets OCTET STRING, - oid OBJECT IDENTIFIER, - string UTF8String - } - } - - SvceAuthInfo ::= SEQUENCE { - service GeneralName, - ident GeneralName, - authInfo OCTET STRING OPTIONAL - } - - RoleSyntax ::= SEQUENCE { - roleAuthority [0] GeneralNames OPTIONAL, - roleName [1] GeneralName - } - - Clearance ::= SEQUENCE { - policyId [0] OBJECT IDENTIFIER, - classList [1] ClassList DEFAULT {unclassified}, - securityCategories - [2] SET OF SecurityCategory OPTIONAL - } - - ClassList ::= BIT STRING { - unmarked (0), - unclassified (1), - restricted (2), - confidential (3), - secret (4), - topSecret (5) - } - - SecurityCategory ::= SEQUENCE { - type [0] IMPLICIT OBJECT IDENTIFIER, - value [1] ANY DEFINED BY type - } - - AAControls ::= SEQUENCE { - pathLenConstraint INTEGER (0..MAX) OPTIONAL, - permittedAttrs [0] AttrSpec OPTIONAL, - excludedAttrs [1] AttrSpec OPTIONAL, - permitUnSpecified BOOLEAN DEFAULT TRUE - } - - AttrSpec::= SEQUENCE OF OBJECT IDENTIFIER - - ACClearAttrs ::= SEQUENCE { - acIssuer GeneralName, - acSerial INTEGER, - attrs SEQUENCE OF Attribute - } - - ProxyInfo ::= SEQUENCE OF Targets - - END diff --git a/lib/ssl/pkix/PKIXAttributeCertificate.hrl b/lib/ssl/pkix/PKIXAttributeCertificate.hrl deleted file mode 100644 index 99389c4852..0000000000 --- a/lib/ssl/pkix/PKIXAttributeCertificate.hrl +++ /dev/null @@ -1,64 +0,0 @@ -%% Generated by the Erlang ASN.1 compiler version:1.4.4.8 -%% Purpose: Erlang record definitions for each named and unnamed -%% SEQUENCE and SET, and macro definitions for each value -%% definition,in module PKIXAttributeCertificate - - - --record('AttributeCertificate',{ -acinfo, signatureAlgorithm, signatureValue}). - --record('AttributeCertificateInfo',{ -version, holder, issuer, signature, serialNumber, attrCertValidityPeriod, attributes, issuerUniqueID = asn1_NOVALUE, extensions = asn1_NOVALUE}). - --record('Holder',{ -baseCertificateID = asn1_NOVALUE, entityName = asn1_NOVALUE, objectDigestInfo = asn1_NOVALUE}). - --record('ObjectDigestInfo',{ -digestedObjectType, otherObjectTypeID = asn1_NOVALUE, digestAlgorithm, objectDigest}). - --record('V2Form',{ -issuerName = asn1_NOVALUE, baseCertificateID = asn1_NOVALUE, objectDigestInfo = asn1_NOVALUE}). - --record('IssuerSerial',{ -issuer, serial, issuerUID = asn1_NOVALUE}). - --record('AttCertValidityPeriod',{ -notBeforeTime, notAfterTime}). - --record('TargetCert',{ -targetCertificate, targetName = asn1_NOVALUE, certDigestInfo = asn1_NOVALUE}). - --record('IetfAttrSyntax',{ -policyAuthority = asn1_NOVALUE, values}). - --record('SvceAuthInfo',{ -service, ident, authInfo = asn1_NOVALUE}). - --record('RoleSyntax',{ -roleAuthority = asn1_NOVALUE, roleName}). - --record('Clearance',{ -policyId, classList = asn1_DEFAULT, securityCategories = asn1_NOVALUE}). - --record('SecurityCategory',{ -type, value}). - --record('AAControls',{ -pathLenConstraint = asn1_NOVALUE, permittedAttrs = asn1_NOVALUE, excludedAttrs = asn1_NOVALUE, permitUnSpecified = asn1_DEFAULT}). - --record('ACClearAttrs',{ -acIssuer, acSerial, attrs}). - --define('id-pe-ac-auditIdentity', {1,3,6,1,5,5,7,1,4}). --define('id-pe-aaControls', {1,3,6,1,5,5,7,1,6}). --define('id-pe-ac-proxying', {1,3,6,1,5,5,7,1,10}). --define('id-ce-targetInformation', {2,5,29,55}). --define('id-aca', {1,3,6,1,5,5,7,10}). --define('id-aca-authenticationInfo', {1,3,6,1,5,5,7,10,1}). --define('id-aca-accessIdentity', {1,3,6,1,5,5,7,10,2}). --define('id-aca-chargingIdentity', {1,3,6,1,5,5,7,10,3}). --define('id-aca-group', {1,3,6,1,5,5,7,10,4}). --define('id-aca-encAttrs', {1,3,6,1,5,5,7,10,6}). --define('id-at-role', {2,5,4,72}). --define('id-at-clearance', {2,5,1,5,55}). diff --git a/lib/ssl/pkix/README b/lib/ssl/pkix/README deleted file mode 100644 index 8be2c15de5..0000000000 --- a/lib/ssl/pkix/README +++ /dev/null @@ -1,49 +0,0 @@ -The files - - PKIX1Algorithms88.asn1 - PKIX1Explicit88.asn1 - PKIX1Implicit88.asn1 - PKIXAttributeCertificate.asn1 - -are from RFCs 3279, 3280 and 3281. - -We have edited PKIX1Explicit88.asn1, PKIX1Implicit88.asn1, and -PKIXAttributeCertificate.asn1 as follows: - - -1. Removal of definition of UniversalString and BMPString: - -diff -r1.1 PKIX1Explicit88.asn1 -15c15 -< UniversalString ::= [UNIVERSAL 28] IMPLICIT OCTET STRING ---- -> -- UniversalString ::= [UNIVERSAL 28] IMPLICIT OCTET STRING -18c18 -< BMPString ::= [UNIVERSAL 30] IMPLICIT OCTET STRING ---- -> -- BMPString ::= [UNIVERSAL 30] IMPLICIT OCTET STRING - - -2. Removal of definition of BMPString: - -diff -r1.1 PKIX1Implicit88.asn1 -13c13,14 -< BMPString, UTF8String, -- end "new" types -- ---- -> -- BMPString, -> UTF8String, -- end "new" types -- - - -3. Addition of definition of UTF8String, and correction of a typo. - -diff -r1.1 PKIXAttributeCertificate.asn1 -46c46 -< -- UTF8String ::= [UNIVERSAL 12] IMPLICIT OCTET STRING ---- -> UTF8String ::= [UNIVERSAL 12] IMPLICIT OCTET STRING -55c55 -< version AttCertVersion -- version is v2, ---- -> version AttCertVersion, -- version is v2 - -PKIX1Algorithms88.asn1 is unchanged. diff --git a/lib/ssl/pkix/SSL-PKIX.asn1 b/lib/ssl/pkix/SSL-PKIX.asn1 deleted file mode 100644 index ea6333f953..0000000000 --- a/lib/ssl/pkix/SSL-PKIX.asn1 +++ /dev/null @@ -1,704 +0,0 @@ -SSL-PKIX {iso(1) identified-organization(3) dod(6) internet(1) - private(4) enterprices(1) ericsson(193) otp(19) ssl(10) - pkix1(1)} - -DEFINITIONS EXPLICIT TAGS ::= - -BEGIN - --- EXPORTS ALL - -IMPORTS - -- Certificate (parts of) - Version, - CertificateSerialNumber, - --AlgorithmIdentifier, - Validity, - UniqueIdentifier, - - -- AttribyteTypeAndValue - Name, - AttributeType, - id-at-name, - id-at-surname, - id-at-givenName, - id-at-initials, - id-at-generationQualifier, X520name, - id-at-commonName, X520CommonName, - id-at-localityName, X520LocalityName, - id-at-stateOrProvinceName, X520StateOrProvinceName, - id-at-organizationName, X520OrganizationName, - id-at-organizationalUnitName, X520OrganizationalUnitName, - id-at-title, X520Title, - id-at-dnQualifier, X520dnQualifier, - id-at-countryName, X520countryName, - id-at-serialNumber, X520SerialNumber, - id-at-pseudonym, X520Pseudonym, - id-domainComponent, DomainComponent, - id-emailAddress, EmailAddress, - - -- Extension Attributes - common-name, CommonName, - teletex-common-name, TeletexCommonName, - teletex-personal-name, TeletexPersonalName, - pds-name, PDSName, - physical-delivery-country-name, PhysicalDeliveryCountryName, - postal-code, PostalCode, - physical-delivery-office-name, PhysicalDeliveryOfficeName, - physical-delivery-office-number, PhysicalDeliveryOfficeNumber, - extension-OR-address-components, ExtensionORAddressComponents, - physical-delivery-personal-name, PhysicalDeliveryPersonalName, - physical-delivery-organization-name, PhysicalDeliveryOrganizationName, - extension-physical-delivery-address-components, - ExtensionPhysicalDeliveryAddressComponents, - unformatted-postal-address, UnformattedPostalAddress, - street-address, StreetAddress, - post-office-box-address, PostOfficeBoxAddress, - poste-restante-address, PosteRestanteAddress, - unique-postal-name, UniquePostalName, - local-postal-attributes, LocalPostalAttributes, - extended-network-address, ExtendedNetworkAddress, - terminal-type, TerminalType, - teletex-domain-defined-attributes, TeletexDomainDefinedAttributes - - FROM PKIX1Explicit88 { iso(1) identified-organization(3) dod(6) - internet(1) security(5) mechanisms(5) pkix(7) id-mod(0) - id-pkix1-explicit(18) } - - -- Extensions - id-ce-authorityKeyIdentifier, AuthorityKeyIdentifier, - id-ce-subjectKeyIdentifier, SubjectKeyIdentifier, - id-ce-keyUsage, KeyUsage, - id-ce-privateKeyUsagePeriod, PrivateKeyUsagePeriod, - id-ce-certificatePolicies, CertificatePolicies, - id-ce-policyMappings, PolicyMappings, - id-ce-subjectAltName, SubjectAltName, - id-ce-issuerAltName, IssuerAltName, - id-ce-subjectDirectoryAttributes, SubjectDirectoryAttributes, - id-ce-basicConstraints, BasicConstraints, - id-ce-nameConstraints, NameConstraints, - id-ce-policyConstraints, PolicyConstraints, - id-ce-cRLDistributionPoints, CRLDistributionPoints, - id-ce-extKeyUsage, ExtKeyUsageSyntax, - id-ce-inhibitAnyPolicy, InhibitAnyPolicy, - id-ce-freshestCRL, FreshestCRL, - id-pe-authorityInfoAccess, AuthorityInfoAccessSyntax, - id-pe-subjectInfoAccess, SubjectInfoAccessSyntax, - id-ce-cRLNumber, CRLNumber, - id-ce-issuingDistributionPoint, IssuingDistributionPoint, - id-ce-deltaCRLIndicator, BaseCRLNumber, - id-ce-cRLReasons, CRLReason, - id-ce-certificateIssuer, CertificateIssuer, - id-ce-holdInstructionCode, HoldInstructionCode, - id-ce-invalidityDate, InvalidityDate - - FROM PKIX1Implicit88 { iso(1) identified-organization(3) dod(6) - internet(1) security(5) mechanisms(5) pkix(7) id-mod(0) - id-pkix1-implicit(19) } - - --Keys and Signatures - id-dsa, Dss-Parms, DSAPublicKey, - id-dsa-with-sha1, - md2WithRSAEncryption, - md5WithRSAEncryption, - sha1WithRSAEncryption, - rsaEncryption, RSAPublicKey, - dhpublicnumber, DomainParameters, DHPublicKey, - id-keyExchangeAlgorithm, KEA-Parms-Id, --KEA-PublicKey, - ecdsa-with-SHA1, - prime-field, Prime-p, - characteristic-two-field, --Characteristic-two, - gnBasis, - tpBasis, Trinomial, - ppBasis, Pentanomial, - id-ecPublicKey, EcpkParameters, ECPoint - FROM PKIX1Algorithms88 { iso(1) identified-organization(3) dod(6) - internet(1) security(5) mechanisms(5) pkix(7) id-mod(0) - id-mod-pkix1-algorithms(17) }; - --- --- Certificate --- - -SSLCertificate ::= SEQUENCE { - tbsCertificate TBSCertificate, - signatureAlgorithm SignatureAlgorithm, - signature BIT STRING } - -SSLTBSCertificate ::= SEQUENCE { - version [0] Version DEFAULT v1, - serialNumber CertificateSerialNumber, - signature SignatureAlgorithm, - issuer Name, - validity Validity, - subject Name, - subjectPublicKeyInfo SubjectPublicKeyInfo, - issuerUniqueID [1] IMPLICIT UniqueIdentifier OPTIONAL, - -- If present, version MUST be v2 or v3 - subjectUniqueID [2] IMPLICIT UniqueIdentifier OPTIONAL, - -- If present, version MUST be v2 or v3 - extensions [3] Extensions OPTIONAL - -- If present, version MUST be v3 -- } - - --- Attribute type and values --- - -ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= CLASS { - &id AttributeType UNIQUE, - &Type } - WITH SYNTAX { - ID &id - TYPE &Type } - -SSLAttributeTypeAndValue ::= SEQUENCE { - type ATTRIBUTE-TYPE-AND-VALUE-CLASS.&id - ({SupportedAttributeTypeAndValues}), - value ATTRIBUTE-TYPE-AND-VALUE-CLASS.&Type - ({SupportedAttributeTypeAndValues}{@type}) } - -SupportedAttributeTypeAndValues ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= - { name | surname | givenName | initials | generationQualifier | - commonName | localityName | stateOrProvinceName | organizationName | - organizationalUnitName | title | dnQualifier | countryName | - serialNumber | pseudonym | domainComponent | emailAddress } - -name ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-name - TYPE X520name } - -surname ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-surname - TYPE X520name } - -givenName ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-givenName - TYPE X520name } - -initials ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-initials - TYPE X520name } - -generationQualifier ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-generationQualifier - TYPE X520name } - -commonName ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-commonName - TYPE X520CommonName } - -localityName ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-localityName - TYPE X520LocalityName } - -stateOrProvinceName ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-stateOrProvinceName - TYPE X520StateOrProvinceName } - -organizationName ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-organizationName - TYPE X520OrganizationName } - -organizationalUnitName ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-organizationalUnitName - TYPE X520OrganizationalUnitName } - -title ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-title - TYPE X520Title } - -dnQualifier ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-dnQualifier - TYPE X520dnQualifier } - -countryName ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-countryName - TYPE X520countryName } - -serialNumber ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-serialNumber - TYPE X520SerialNumber } - -pseudonym ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-at-pseudonym - TYPE X520Pseudonym } - -domainComponent ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-domainComponent - TYPE DomainComponent } - -emailAddress ATTRIBUTE-TYPE-AND-VALUE-CLASS ::= { - ID id-emailAddress - TYPE EmailAddress } - --- --- Signature and Public Key Algorithms --- - -SSLSubjectPublicKeyInfo ::= SEQUENCE { - algorithm SEQUENCE { - algo PUBLIC-KEY-ALGORITHM-CLASS.&id - ({SupportedPublicKeyAlgorithms}), - parameters PUBLIC-KEY-ALGORITHM-CLASS.&Type - ({SupportedPublicKeyAlgorithms}{@.algo}) - OPTIONAL - }, - subjectPublicKey PUBLIC-KEY-ALGORITHM-CLASS.&PublicKeyType - ({SupportedPublicKeyAlgorithms}{@algorithm.algo}) } - --- The following is needed for conversion of SubjectPublicKeyInfo. - -SSLSubjectPublicKeyInfo-Any ::= SEQUENCE { - algorithm PublicKeyAlgorithm, - subjectPublicKey ANY } - - -SIGNATURE-ALGORITHM-CLASS ::= CLASS { - &id OBJECT IDENTIFIER UNIQUE, - &Type OPTIONAL } - WITH SYNTAX { - ID &id - [TYPE &Type] } - -PUBLIC-KEY-ALGORITHM-CLASS ::= CLASS { - &id OBJECT IDENTIFIER UNIQUE, - &Type OPTIONAL, - &PublicKeyType OPTIONAL } - WITH SYNTAX { - ID &id - [TYPE &Type] - [PUBLIC-KEY-TYPE &PublicKeyType] } - -SignatureAlgorithm ::= SEQUENCE { - algorithm SIGNATURE-ALGORITHM-CLASS.&id - ({SupportedSignatureAlgorithms}), - parameters SIGNATURE-ALGORITHM-CLASS.&Type - ({SupportedSignatureAlgorithms}{@algorithm}) - OPTIONAL } - -SignatureAlgorithm-Any ::= SEQUENCE { - algorithm OBJECT IDENTIFIER, - parameters ANY OPTIONAL } - -PublicKeyAlgorithm ::= SEQUENCE { - algorithm PUBLIC-KEY-ALGORITHM-CLASS.&id - ({SupportedPublicKeyAlgorithms}), - parameters PUBLIC-KEY-ALGORITHM-CLASS.&Type - ({SupportedPublicKeyAlgorithms}{@algorithm}) - OPTIONAL } - -SupportedSignatureAlgorithms SIGNATURE-ALGORITHM-CLASS ::= { - dsa-with-sha1 | md2-with-rsa-encryption | - md5-with-rsa-encryption | sha1-with-rsa-encryption | - ecdsa-with-sha1 } - -SupportedPublicKeyAlgorithms PUBLIC-KEY-ALGORITHM-CLASS ::= { - dsa | rsa-encryption | dh | kea | ec-public-key } - - -- DSA Keys and Signatures - - -- SubjectPublicKeyInfo: - - dsa PUBLIC-KEY-ALGORITHM-CLASS ::= { - ID id-dsa - TYPE Dss-Parms -- XXX Must be OPTIONAL - PUBLIC-KEY-TYPE DSAPublicKey } - - -- Certificate.signatureAlgorithm - - dsa-with-sha1 SIGNATURE-ALGORITHM-CLASS ::= { - ID id-dsa-with-sha1 - TYPE NULL } -- XXX Must be empty and not NULL - - -- - -- RSA Keys and Signatures - -- - - -- Certificate.signatureAlgorithm - - md2-with-rsa-encryption SIGNATURE-ALGORITHM-CLASS ::= { - ID md2WithRSAEncryption - TYPE NULL } - - md5-with-rsa-encryption SIGNATURE-ALGORITHM-CLASS ::= { - ID md5WithRSAEncryption - TYPE NULL } - - sha1-with-rsa-encryption SIGNATURE-ALGORITHM-CLASS ::= { - ID sha1WithRSAEncryption - TYPE NULL } - - -- Certificate.signature - -- See PKCS #1 (RFC 2313). XXX - - -- SubjectPublicKeyInfo: - - rsa-encryption PUBLIC-KEY-ALGORITHM-CLASS ::= { - ID rsaEncryption - TYPE NULL - PUBLIC-KEY-TYPE RSAPublicKey } - - -- - -- Diffie-Hellman Keys - -- - - -- SubjectPublicKeyInfo: - - dh PUBLIC-KEY-ALGORITHM-CLASS ::= { - ID dhpublicnumber - TYPE DomainParameters - PUBLIC-KEY-TYPE DHPublicKey } - - -- There are no Diffie-Hellman signature algorithms - - -- - -- KEA Keys - -- - - -- SubjectPublicKeyInfo: - - KEA-PublicKey ::= INTEGER - - kea PUBLIC-KEY-ALGORITHM-CLASS ::= { - ID id-keyExchangeAlgorithm - TYPE KEA-Parms-Id - PUBLIC-KEY-TYPE KEA-PublicKey } - - -- There are no KEA signature algorithms - - -- - -- Elliptic Curve Keys, Signatures, and Curves - -- - - -- Certificate.signatureAlgorithm - - ecdsa-with-sha1 SIGNATURE-ALGORITHM-CLASS ::= { - ID ecdsa-with-SHA1 - TYPE NULL } -- XXX Must be empty and not NULL - - FIELD-ID-CLASS ::= CLASS { - &id OBJECT IDENTIFIER UNIQUE, - &Type } - WITH SYNTAX { - ID &id - TYPE &Type } - - SSLFieldID ::= SEQUENCE { -- Finite field - fieldType FIELD-ID-CLASS.&id({SupportedFieldIds}), - parameters FIELD-ID-CLASS.&Type({SupportedFieldIds}{@fieldType}) } - - SupportedFieldIds FIELD-ID-CLASS ::= { - field-prime-field | field-characteristic-two } - - field-prime-field FIELD-ID-CLASS ::= { - ID prime-field - TYPE Prime-p } - - CHARACTERISTIC-TWO-CLASS ::= CLASS { - &id OBJECT IDENTIFIER UNIQUE, - &Type } - WITH SYNTAX { - ID &id - TYPE &Type } - - SSLCharacteristic-two ::= SEQUENCE { -- Finite field - m INTEGER, -- Field size 2^m - basis CHARACTERISTIC-TWO-CLASS.&id({SupportedCharacteristicTwos}), - parameters CHARACTERISTIC-TWO-CLASS.&Type - ({SupportedCharacteristicTwos}{@basis}) } - - SupportedCharacteristicTwos CHARACTERISTIC-TWO-CLASS ::= { - gn-basis | tp-basis | pp-basis } - - field-characteristic-two FIELD-ID-CLASS ::= { - ID characteristic-two-field - TYPE Characteristic-two } - - gn-basis CHARACTERISTIC-TWO-CLASS ::= { - ID gnBasis - TYPE NULL } - - tp-basis CHARACTERISTIC-TWO-CLASS ::= { - ID tpBasis - TYPE Trinomial } - - pp-basis CHARACTERISTIC-TWO-CLASS ::= { - ID ppBasis - TYPE Pentanomial } - - -- SubjectPublicKeyInfo.algorithm - - ec-public-key PUBLIC-KEY-ALGORITHM-CLASS ::= { - ID id-ecPublicKey - TYPE EcpkParameters - PUBLIC-KEY-TYPE ECPoint } - --- --- Extension Attributes --- - -EXTENSION-ATTRIBUTE-CLASS ::= CLASS { - &id INTEGER UNIQUE, - &Type } - WITH SYNTAX { - ID &id - TYPE &Type } - -SSLExtensionAttributes ::= SET SIZE (1..MAX) OF ExtensionAttribute - --- XXX Below we should have extension-attribute-type and extension- --- attribute-value but Erlang ASN1 does not like it. -SSLExtensionAttribute ::= SEQUENCE { - extensionAttributeType [0] IMPLICIT EXTENSION-ATTRIBUTE-CLASS.&id - ({SupportedExtensionAttributes}), - extensionAttributeValue [1] EXTENSION-ATTRIBUTE-CLASS.&Type - ({SupportedExtensionAttributes}{@extensionAttributeType}) } - -SupportedExtensionAttributes EXTENSION-ATTRIBUTE-CLASS ::= { - x400-common-name | - x400-teletex-common-name | - x400-teletex-personal-name | - x400-pds-name | - x400-physical-delivery-country-name | - x400-postal-code | - x400-physical-delivery-office-name | - x400-physical-delivery-office-number | - x400-extension-OR-address-components | - x400-physical-delivery-personal-name | - x400-physical-delivery-organization-name | - x400-extension-physical-delivery-address-components | - x400-unformatted-postal-address | - x400-street-address | - x400-post-office-box-address | - x400-poste-restante-address | - x400-unique-postal-name | - x400-local-postal-attributes | - x400-extended-network-address | - x400-terminal-type | - x400-teletex-domain-defined-attributes } - --- Extension types and attribute values - -x400-common-name EXTENSION-ATTRIBUTE-CLASS ::= { - ID common-name - TYPE CommonName } - -x400-teletex-common-name EXTENSION-ATTRIBUTE-CLASS ::= { - ID teletex-common-name - TYPE TeletexCommonName } - -x400-teletex-personal-name EXTENSION-ATTRIBUTE-CLASS ::= { - ID teletex-personal-name - TYPE TeletexPersonalName } - -x400-pds-name EXTENSION-ATTRIBUTE-CLASS ::= { - ID pds-name - TYPE PDSName } - -x400-physical-delivery-country-name EXTENSION-ATTRIBUTE-CLASS ::= { - ID physical-delivery-country-name - TYPE PhysicalDeliveryCountryName } - -x400-postal-code EXTENSION-ATTRIBUTE-CLASS ::= { - ID postal-code - TYPE PostalCode } - -x400-physical-delivery-office-name EXTENSION-ATTRIBUTE-CLASS ::= { - ID physical-delivery-office-name - TYPE PhysicalDeliveryOfficeName } - -x400-physical-delivery-office-number EXTENSION-ATTRIBUTE-CLASS ::= { - ID physical-delivery-office-number - TYPE PhysicalDeliveryOfficeNumber } - -x400-extension-OR-address-components EXTENSION-ATTRIBUTE-CLASS ::= { - ID extension-OR-address-components - TYPE ExtensionORAddressComponents } - -x400-physical-delivery-personal-name EXTENSION-ATTRIBUTE-CLASS ::= { - ID physical-delivery-personal-name - TYPE PhysicalDeliveryPersonalName } - -x400-physical-delivery-organization-name EXTENSION-ATTRIBUTE-CLASS ::= { - ID physical-delivery-organization-name - TYPE PhysicalDeliveryOrganizationName } - -x400-extension-physical-delivery-address-components - EXTENSION-ATTRIBUTE-CLASS ::= { - ID extension-physical-delivery-address-components - TYPE ExtensionPhysicalDeliveryAddressComponents } - -x400-unformatted-postal-address EXTENSION-ATTRIBUTE-CLASS ::= { - ID unformatted-postal-address - TYPE UnformattedPostalAddress } - -x400-street-address EXTENSION-ATTRIBUTE-CLASS ::= { - ID street-address - TYPE StreetAddress } - -x400-post-office-box-address EXTENSION-ATTRIBUTE-CLASS ::= { - ID post-office-box-address - TYPE PostOfficeBoxAddress } - -x400-poste-restante-address EXTENSION-ATTRIBUTE-CLASS ::= { - ID poste-restante-address - TYPE PosteRestanteAddress } - -x400-unique-postal-name EXTENSION-ATTRIBUTE-CLASS ::= { - ID unique-postal-name - TYPE UniquePostalName } - -x400-local-postal-attributes EXTENSION-ATTRIBUTE-CLASS ::= { - ID local-postal-attributes - TYPE LocalPostalAttributes } - -x400-extended-network-address EXTENSION-ATTRIBUTE-CLASS ::= { - ID extended-network-address - TYPE ExtendedNetworkAddress } - -x400-terminal-type EXTENSION-ATTRIBUTE-CLASS ::= { - ID terminal-type - TYPE TerminalType } - -x400-teletex-domain-defined-attributes EXTENSION-ATTRIBUTE-CLASS ::= { - ID teletex-domain-defined-attributes - TYPE TeletexDomainDefinedAttributes } - --- Extensions - -SSLExtensions ::= SEQUENCE SIZE (1..MAX) OF Extension - -EXTENSION-CLASS ::= CLASS { - &id OBJECT IDENTIFIER UNIQUE, - &Type OPTIONAL} - WITH SYNTAX { - ID &id - [TYPE &Type] } - -SSLExtension ::= SEQUENCE { - extnID EXTENSION-CLASS.&id({SupportedExtensions}), - critical BOOLEAN DEFAULT FALSE, - extnValue EXTENSION-CLASS.&Type({SupportedExtensions}{@extnID}) } - --- The following is needed for conversion between Extension and Extension-Cd - -ObjId ::= OBJECT IDENTIFIER -Boolean ::= BOOLEAN -Any ::= ANY - -Extension-Any ::= SEQUENCE { - extnID OBJECT IDENTIFIER, - critical BOOLEAN DEFAULT FALSE, - extnValue ANY } - -SupportedExtensions EXTENSION-CLASS ::= { authorityKeyIdentifier | - subjectKeyIdentifier | keyUsage | privateKeyUsagePeriod | - certificatePolicies | policyMappings | subjectAltName | - issuerAltName | subjectDirectoryAttributes | basicConstraints | - nameConstraints | policyConstraints | cRLDistributionPoints | - extKeyUsage | inhibitAnyPolicy | freshestCRL | authorityInfoAccess | - subjectInfoAccess | cRLNumber | issuingDistributionPoint | - deltaCRLIndicator | cRLReasons | certificateIssuer | - holdInstructionCode | invalidityDate } - -authorityKeyIdentifier EXTENSION-CLASS ::= { - ID id-ce-authorityKeyIdentifier - TYPE AuthorityKeyIdentifier } - -subjectKeyIdentifier EXTENSION-CLASS ::= { - ID id-ce-subjectKeyIdentifier - TYPE SubjectKeyIdentifier } - -keyUsage EXTENSION-CLASS ::= { - ID id-ce-keyUsage - TYPE KeyUsage } - -privateKeyUsagePeriod EXTENSION-CLASS ::= { - ID id-ce-privateKeyUsagePeriod - TYPE PrivateKeyUsagePeriod } - -certificatePolicies EXTENSION-CLASS ::= { - ID id-ce-certificatePolicies - TYPE CertificatePolicies } - -policyMappings EXTENSION-CLASS ::= { - ID id-ce-policyMappings - TYPE PolicyMappings } - -subjectAltName EXTENSION-CLASS ::= { - ID id-ce-subjectAltName - TYPE SubjectAltName } - -issuerAltName EXTENSION-CLASS ::= { - ID id-ce-issuerAltName - TYPE IssuerAltName } - -subjectDirectoryAttributes EXTENSION-CLASS ::= { - ID id-ce-subjectDirectoryAttributes - TYPE SubjectDirectoryAttributes } - -basicConstraints EXTENSION-CLASS ::= { - ID id-ce-basicConstraints - TYPE BasicConstraints } - -nameConstraints EXTENSION-CLASS ::= { - ID id-ce-nameConstraints - TYPE NameConstraints } - -policyConstraints EXTENSION-CLASS ::= { - ID id-ce-policyConstraints - TYPE PolicyConstraints } - -cRLDistributionPoints EXTENSION-CLASS ::= { - ID id-ce-cRLDistributionPoints - TYPE CRLDistributionPoints } - -extKeyUsage EXTENSION-CLASS ::= { - ID id-ce-extKeyUsage - TYPE ExtKeyUsageSyntax } - -inhibitAnyPolicy EXTENSION-CLASS ::= { - ID id-ce-inhibitAnyPolicy - TYPE InhibitAnyPolicy } - -freshestCRL EXTENSION-CLASS ::= { - ID id-ce-freshestCRL - TYPE FreshestCRL } - -authorityInfoAccess EXTENSION-CLASS ::= { - ID id-pe-authorityInfoAccess - TYPE AuthorityInfoAccessSyntax } - -subjectInfoAccess EXTENSION-CLASS ::= { - ID id-pe-subjectInfoAccess - TYPE SubjectInfoAccessSyntax } - -cRLNumber EXTENSION-CLASS ::= { - ID id-ce-cRLNumber - TYPE CRLNumber } - -issuingDistributionPoint EXTENSION-CLASS ::= { - ID id-ce-issuingDistributionPoint - TYPE IssuingDistributionPoint } - -deltaCRLIndicator EXTENSION-CLASS ::= { - ID id-ce-deltaCRLIndicator - TYPE BaseCRLNumber } - -cRLReasons EXTENSION-CLASS ::= { - ID id-ce-cRLReasons - TYPE CRLReason } - -certificateIssuer EXTENSION-CLASS ::= { - ID id-ce-certificateIssuer - TYPE CertificateIssuer } - -holdInstructionCode EXTENSION-CLASS ::= { - ID id-ce-holdInstructionCode - TYPE HoldInstructionCode } - -invalidityDate EXTENSION-CLASS ::= { - ID id-ce-invalidityDate - TYPE InvalidityDate } - -END diff --git a/lib/ssl/pkix/mk_ssl_pkix_oid.erl b/lib/ssl/pkix/mk_ssl_pkix_oid.erl deleted file mode 100644 index 06edc5113a..0000000000 --- a/lib/ssl/pkix/mk_ssl_pkix_oid.erl +++ /dev/null @@ -1,94 +0,0 @@ --module(mk_ssl_pkix_oid). - --export([make/0]). - --define(PKIX_MODULES, ['OTP-PKIX']). - -make() -> - {ok, Fd} = file:open("ssl_pkix_oid.erl", [write]), - io:fwrite(Fd, "%%% File: ssl_pkix_oid.erl\n" - "%%% NB This file has been automatically generated by " - "mk_ssl_pkix_oid.\n" - "%%% Do not edit it.\n\n", []), - io:fwrite(Fd, "-module(ssl_pkix_oid).\n", []), - io:fwrite(Fd, "-export([id2atom/1, atom2id/1, all_atoms/0, " - "all_ids/0]).\n\n", []), - - - AIds0 = get_atom_ids(?PKIX_MODULES), - - AIds1 = modify_atoms(AIds0), - gen_id2atom(Fd, AIds1), - gen_atom2id(Fd, AIds1), - gen_all(Fd, AIds1), - file:close(Fd). - -get_atom_ids(Ms) -> - get_atom_ids(Ms, []). - -get_atom_ids([], AIdss) -> - lists:flatten(AIdss); -get_atom_ids([M| Ms], AIdss) -> - {value, {exports, Exports}} = - lists:keysearch(exports, 1, M:module_info()), - As = lists:zf( - fun ({info, 0}) -> false; - ({module_info, 0}) -> false; - ({encoding_rule, 0}) -> false; - ({F, 0}) -> - case atom_to_list(F) of - %% Remove upper-bound (ub-) functions - "ub-" ++ _Rest -> - false; - _ -> - {true, F} - end; - (_) -> false - end, Exports), - AIds = lists:map(fun(F) -> {F, M:F()} end, As), - get_atom_ids(Ms, [AIds| AIdss]). - -modify_atoms(AIds) -> - F = fun({A, I}) -> - NAS = case atom_to_list(A) of - "id-" ++ Rest -> - Rest; - Any -> - Any - end, - {list_to_atom(NAS), I} end, - lists:map(F, AIds). - -gen_id2atom(Fd, AIds0) -> - AIds1 = lists:keysort(2, AIds0), - Txt = join(";\n", - lists:map( - fun({Atom, Id}) -> - io_lib:fwrite("id2atom(~p) ->\n ~p", [Id, Atom]) - end, AIds1)), - io:fwrite(Fd, "~s;\nid2atom(Any)->\n Any.\n\n", [Txt]). - -gen_atom2id(Fd, AIds0) -> - AIds1 = lists:keysort(1, AIds0), - Txt = join(";\n", - lists:map( - fun({Atom, Id}) -> - io_lib:fwrite("atom2id(~p) ->\n ~p", [Atom, Id]) - end, AIds1)), - io:fwrite(Fd, "~s;\natom2id(Any)->\n Any.\n\n", [Txt]). - -gen_all(Fd, AIds) -> - Atoms = lists:sort([A || {A, _} <- AIds]), - Ids = lists:sort([I || {_, I} <- AIds]), - F = fun(X) -> io_lib:fwrite(" ~w", [X]) end, - ATxt = "all_atoms() ->\n" ++ join(",\n", lists:map(F, Atoms)), - io:fwrite(Fd, "~s.\n\n", [ATxt]), - ITxt = "all_ids() ->\n" ++ join(",\n", lists:map(F, Ids)), - io:fwrite(Fd, "~s.\n\n", [ITxt]). - -join(Sep, [H1, H2| T]) -> - [H1, Sep| join(Sep, [H2| T])]; -join(_Sep, [H1]) -> - H1; -join(_, []) -> - []. diff --git a/lib/ssl/pkix/prebuild.skip b/lib/ssl/pkix/prebuild.skip deleted file mode 100644 index ffe82be68b..0000000000 --- a/lib/ssl/pkix/prebuild.skip +++ /dev/null @@ -1,5 +0,0 @@ -PKIX1Algorithms88.asn1db -PKIXAttributeCertificate.asn1db -PKIX1Explicit88.asn1db -SSL-PKIX.asn1db -PKIX1Implicit88.asn1db diff --git a/lib/ssl/src/Makefile b/lib/ssl/src/Makefile index fabf8a4e0d..7514ad2aa2 100644 --- a/lib/ssl/src/Makefile +++ b/lib/ssl/src/Makefile @@ -1,19 +1,19 @@ # # %CopyrightBegin% -# -# Copyright Ericsson AB 1999-2009. All Rights Reserved. -# +# +# Copyright Ericsson AB 1999-2010. All Rights Reserved. +# # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in # compliance with the License. You should have received a copy of the # Erlang Public License along with this software. If not, it can be # retrieved online at http://www.erlang.org/. -# +# # Software distributed under the License is distributed on an "AS IS" # basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See # the License for the specific language governing rights and limitations # under the License. -# +# # %CopyrightEnd% # @@ -46,9 +46,6 @@ MODULES= \ ssl_server \ ssl_sup \ ssl_prim \ - ssl_pkix \ - ssl_pem \ - ssl_base64 \ inet_ssl_dist \ ssl_certificate\ ssl_certificate_db\ @@ -71,8 +68,6 @@ INTERNAL_HRL_FILES = \ ssl_alert.hrl ssl_cipher.hrl ssl_handshake.hrl ssl_internal.hrl \ ssl_record.hrl -PUBLIC_HRL_FILES = ssl_pkix.hrl - ERL_FILES= $(MODULES:%=%.erl) TARGET_FILES= $(MODULES:%=$(EBIN)/%.$(EMULATOR)) @@ -85,15 +80,12 @@ APP_TARGET= $(EBIN)/$(APP_FILE) APPUP_SRC= $(APPUP_FILE).src APPUP_TARGET= $(EBIN)/$(APPUP_FILE) -INCLUDE = ../include - # ---------------------------------------------------- # FLAGS # ---------------------------------------------------- EXTRA_ERLC_FLAGS = +warn_unused_vars ERL_COMPILE_FLAGS += -I$(ERL_TOP)/lib/kernel/src \ -pz $(ERL_TOP)/lib/public_key/ebin \ - -I$(INCLUDE) \ $(EXTRA_ERLC_FLAGS) -DVSN=\"$(VSN)\" @@ -101,7 +93,7 @@ ERL_COMPILE_FLAGS += -I$(ERL_TOP)/lib/kernel/src \ # Targets # ---------------------------------------------------- -debug opt: $(TARGET_FILES) $(APP_TARGET) $(APPUP_TARGET) $(PUBLIC_HRL_FILES) +debug opt: $(TARGET_FILES) $(APP_TARGET) $(APPUP_TARGET) clean: rm -f $(TARGET_FILES) $(APP_TARGET) $(APPUP_TARGET) @@ -113,9 +105,6 @@ $(APP_TARGET): $(APP_SRC) ../vsn.mk $(APPUP_TARGET): $(APPUP_SRC) ../vsn.mk sed -e 's;%VSN%;$(VSN);' $< > $@ -$(PUBLIC_HRL_FILES): - cp -f $(PUBLIC_HRL_FILES) $(INCLUDE) - docs: # ---------------------------------------------------- @@ -126,8 +115,6 @@ include $(ERL_TOP)/make/otp_release_targets.mk release_spec: opt $(INSTALL_DIR) $(RELSYSDIR)/src $(INSTALL_DATA) $(ERL_FILES) $(INTERNAL_HRL_FILES) $(RELSYSDIR)/src - $(INSTALL_DIR) $(RELSYSDIR)/include - $(INSTALL_DATA) $(PUBLIC_HRL_FILES) $(RELSYSDIR)/include $(INSTALL_DIR) $(RELSYSDIR)/ebin $(INSTALL_DATA) $(TARGET_FILES) $(APP_TARGET) \ $(APPUP_TARGET) $(RELSYSDIR)/ebin diff --git a/lib/ssl/src/ssl.app.src b/lib/ssl/src/ssl.app.src index 2a7d451341..b9716786e6 100644 --- a/lib/ssl/src/ssl.app.src +++ b/lib/ssl/src/ssl.app.src @@ -7,10 +7,6 @@ ssl_server, ssl_broker, ssl_broker_sup, - ssl_base64, - ssl_pem, - ssl_pkix, - ssl_pkix_oid, ssl_prim, inet_ssl_dist, ssl_tls1, @@ -28,11 +24,10 @@ ssl_cipher, ssl_certificate_db, ssl_certificate, - ssl_alert, - 'OTP-PKIX' + ssl_alert ]}, {registered, [ssl_sup, ssl_server, ssl_broker_sup]}, - {applications, [kernel, stdlib]}, + {applications, [crypto, public_key, kernel, stdlib]}, {env, []}, {mod, {ssl_app, []}}]}. diff --git a/lib/ssl/src/ssl.appup.src b/lib/ssl/src/ssl.appup.src index 52a41617bb..65f23e2f74 100644 --- a/lib/ssl/src/ssl.appup.src +++ b/lib/ssl/src/ssl.appup.src @@ -1,6 +1,7 @@ %% -*- erlang -*- {"%VSN%", [ + {"3.11.1", [{restart_application, ssl}]}, {"3.11", [{restart_application, ssl}]}, {"3.10", [{restart_application, ssl}]}, {"3.10.1", [{restart_application, ssl}]}, @@ -14,6 +15,7 @@ {"3.10.9", [{restart_application, ssl}]} ], [ + {"3.11.1", [{restart_application, ssl}]}, {"3.11", [{restart_application, ssl}]}, {"3.10", [{restart_application, ssl}]}, {"3.10.1", [{restart_application, ssl}]}, diff --git a/lib/ssl/src/ssl.erl b/lib/ssl/src/ssl.erl index 185a1f755a..df4cd7c84d 100644 --- a/lib/ssl/src/ssl.erl +++ b/lib/ssl/src/ssl.erl @@ -34,10 +34,14 @@ %% Should be deprecated as soon as old ssl is removed %%-deprecated({pid, 1, next_major_release}). +-deprecated({peercert, 2, next_major_release}). -include("ssl_int.hrl"). -include("ssl_internal.hrl"). -include("ssl_record.hrl"). +-include("ssl_cipher.hrl"). + +-include_lib("public_key/include/public_key.hrl"). -record(config, {ssl, %% SSL parameters inet_user, %% User set inet options @@ -47,22 +51,25 @@ }). %%-------------------------------------------------------------------- -%% Function: start([, Type]) -> ok -%% -%% Type = permanent | transient | temporary -%% Vsns = [Vsn] -%% Vsn = ssl3 | tlsv1 | 'tlsv1.1' +-spec start() -> ok. +-spec start(permanent | transient | temporary) -> ok. %% -%% Description: Starts the ssl application. Default type +%% Description: Utility function that starts the ssl, +%% crypto and public_key applications. Default type %% is temporary. see application(3) %%-------------------------------------------------------------------- start() -> + application:start(crypto), + application:start(public_key), application:start(ssl). + start(Type) -> + application:start(crypto, Type), + application:start(public_key, Type), application:start(ssl, Type). %%-------------------------------------------------------------------- -%% Function: stop() -> ok +-spec stop() -> ok. %% %% Description: Stops the ssl application. %%-------------------------------------------------------------------- @@ -70,7 +77,8 @@ stop() -> application:stop(ssl). %%-------------------------------------------------------------------- -%% Function: connect(Address, Port, Options[, Timeout]) -> {ok, Socket} +-spec connect(host() | port(), port_num(), list()) -> {ok, #sslsocket{}}. +-spec connect(host() | port(), port_num(), list(), timeout()) -> {ok, #sslsocket{}}. %% %% Description: Connect to a ssl server. %%-------------------------------------------------------------------- @@ -96,13 +104,13 @@ connect(Socket, SslOptions0, Timeout) when is_port(Socket) -> {error, Reason} end; -connect(Address, Port, Options) -> - connect(Address, Port, Options, infinity). +connect(Host, Port, Options) -> + connect(Host, Port, Options, infinity). -connect(Address, Port, Options0, Timeout) -> - case proplists:get_value(ssl_imp, Options0, old) of +connect(Host, Port, Options0, Timeout) -> + case proplists:get_value(ssl_imp, Options0, new) of new -> - new_connect(Address, Port, Options0, Timeout); + new_connect(Host, Port, Options0, Timeout); old -> %% Allow the option reuseaddr to be present %% so that new and old ssl can be run by the same @@ -110,20 +118,21 @@ connect(Address, Port, Options0, Timeout) -> %% that hardcodes reuseaddr to true in its portprogram. Options1 = proplists:delete(reuseaddr, Options0), Options = proplists:delete(ssl_imp, Options1), - old_connect(Address, Port, Options, Timeout); + old_connect(Host, Port, Options, Timeout); Value -> {error, {eoptions, {ssl_imp, Value}}} end. %%-------------------------------------------------------------------- -%% Function: listen(Port, Options) -> {ok, ListenSock} | {error, Reason} +-spec listen(port_num(), list()) ->{ok, #sslsocket{}} | {error, reason()}. + %% %% Description: Creates a ssl listen socket. %%-------------------------------------------------------------------- listen(_Port, []) -> {error, enooptions}; listen(Port, Options0) -> - case proplists:get_value(ssl_imp, Options0, old) of + case proplists:get_value(ssl_imp, Options0, new) of new -> new_listen(Port, Options0); old -> @@ -139,7 +148,8 @@ listen(Port, Options0) -> end. %%-------------------------------------------------------------------- -%% Function: transport_accept(ListenSocket[, Timeout]) -> {ok, Socket}. +-spec transport_accept(#sslsocket{}) -> {ok, #sslsocket{}}. +-spec transport_accept(#sslsocket{}, timeout()) -> {ok, #sslsocket{}}. %% %% Description: Performs transport accept on a ssl listen socket %%-------------------------------------------------------------------- @@ -177,8 +187,8 @@ transport_accept(#sslsocket{} = ListenSocket, Timeout) -> ssl_broker:transport_accept(Pid, ListenSocket, Timeout). %%-------------------------------------------------------------------- -%% Function: ssl_accept(ListenSocket[, Timeout]) -> {ok, Socket} | -%% {error, Reason} +-spec ssl_accept(#sslsocket{}) -> {ok, #sslsocket{}} | {error, reason()}. +-spec ssl_accept(#sslsocket{}, timeout()) -> {ok, #sslsocket{}} | {error, reason()}. %% %% Description: Performs accept on a ssl listen socket. e.i. performs %% ssl handshake. @@ -212,7 +222,7 @@ ssl_accept(Socket, SslOptions, Timeout) when is_port(Socket) -> end. %%-------------------------------------------------------------------- -%% Function: close() -> ok +-spec close(#sslsocket{}) -> term(). %% %% Description: Close a ssl connection %%-------------------------------------------------------------------- @@ -225,7 +235,7 @@ close(Socket = #sslsocket{}) -> ssl_broker:close(Socket). %%-------------------------------------------------------------------- -%% Function: send(Socket, Data) -> ok | {error, Reason} +-spec send(#sslsocket{}, iolist()) -> ok | {error, reason()}. %% %% Description: Sends data over the ssl connection %%-------------------------------------------------------------------- @@ -237,7 +247,8 @@ send(#sslsocket{} = Socket, Data) -> ssl_broker:send(Socket, Data). %%-------------------------------------------------------------------- -%% Function: recv(Socket, Length [,Timeout]) -> {ok, Data} | {error, reason} +-spec recv(#sslsocket{}, integer()) -> {ok, binary()| list()} | {error, reason()}. +-spec recv(#sslsocket{}, integer(), timeout()) -> {ok, binary()| list()} | {error, reason()}. %% %% Description: Receives data when active = false %%-------------------------------------------------------------------- @@ -251,8 +262,8 @@ recv(Socket = #sslsocket{}, Length, Timeout) -> ssl_broker:recv(Socket, Length, Timeout). %%-------------------------------------------------------------------- -%% Function: controlling_process(Socket, NewOwner) -> ok | {error, Reason} -%% +-spec controlling_process(#sslsocket{}, pid()) -> ok | {error, reason()}. +%% %% Description: Changes process that receives the messages when active = true %% or once. %%-------------------------------------------------------------------- @@ -265,11 +276,8 @@ controlling_process(Socket, NewOwner) when is_pid(NewOwner) -> ssl_broker:controlling_process(Socket, NewOwner). %%-------------------------------------------------------------------- -%% Function: connection_info(Socket) -> {ok, {Protocol, CipherSuite}} | -%% {error, Reason} -%% Protocol = sslv3 | tlsv1 | tlsv1.1 -%% CipherSuite = {KeyExchange, Chipher, Hash, Exportable} -%% +-spec connection_info(#sslsocket{}) -> {ok, {tls_atom_version(), erl_cipher_suite()}} | + {error, reason()}. %% %% Description: Returns ssl protocol and cipher used for the connection %%-------------------------------------------------------------------- @@ -281,9 +289,9 @@ connection_info(#sslsocket{} = Socket) -> ssl_broker:connection_info(Socket). %%-------------------------------------------------------------------- -%% Function: peercert(Socket[, Opts]) -> {ok, Cert} | {error, Reason} +-spec peercert(#sslsocket{}) ->{ok, der_cert()} | {error, reason()}. %% -%% Description: +%% Description: Returns the peercert. %%-------------------------------------------------------------------- peercert(Socket) -> peercert(Socket, []). @@ -293,14 +301,7 @@ peercert(#sslsocket{pid = Pid, fd = new_ssl}, Opts) -> {ok, undefined} -> {error, no_peercert}; {ok, BinCert} -> - PKOpts = [case Opt of ssl -> otp; pkix -> plain end || - Opt <- Opts, Opt =:= ssl orelse Opt =:= pkix], - case PKOpts of - [Opt] -> - public_key:pkix_decode_cert(BinCert, Opt); - [] -> - {ok, BinCert} - end; + decode_peercert(BinCert, Opts); {error, Reason} -> {error, Reason} end; @@ -309,15 +310,44 @@ peercert(#sslsocket{} = Socket, Opts) -> ensure_old_ssl_started(), case ssl_broker:peercert(Socket) of {ok, Bin} -> - ssl_pkix:decode_cert(Bin, Opts); + decode_peercert(Bin, Opts); {error, Reason} -> {error, Reason} end. + +decode_peercert(BinCert, Opts) -> + PKOpts = [case Opt of ssl -> otp; pkix -> plain end || + Opt <- Opts, Opt =:= ssl orelse Opt =:= pkix], + case PKOpts of + [Opt] -> + select_part(Opt, public_key:pkix_decode_cert(BinCert, Opt), Opts); + [] -> + {ok, BinCert} + end. + +select_part(otp, {ok, Cert}, Opts) -> + case lists:member(subject, Opts) of + true -> + TBS = Cert#'OTPCertificate'.tbsCertificate, + {ok, TBS#'OTPTBSCertificate'.subject}; + false -> + {ok, Cert} + end; + +select_part(plain, {ok, Cert}, Opts) -> + case lists:member(subject, Opts) of + true -> + TBS = Cert#'Certificate'.tbsCertificate, + {ok, TBS#'TBSCertificate'.subject}; + false -> + {ok, Cert} + end. + %%-------------------------------------------------------------------- -%% Function: peername(Socket) -> {ok, {Address, Port}} | {error, Reason} +-spec peername(#sslsocket{}) -> {ok, {tuple(), port_num()}} | {error, reason()}. %% -%% Description: +%% Description: same as inet:peername/1. %%-------------------------------------------------------------------- peername(#sslsocket{fd = new_ssl, pid = Pid}) -> ssl_connection:peername(Pid); @@ -327,9 +357,10 @@ peername(#sslsocket{} = Socket) -> ssl_broker:peername(Socket). %%-------------------------------------------------------------------- -%% Function: cipher_suites() -> -%% -%% Description: +-spec cipher_suites() -> [erl_cipher_suite()]. +-spec cipher_suites(erlang | openssl) -> [erl_cipher_suite()] | [string()]. + +%% Description: Returns all supported cipher suites. %%-------------------------------------------------------------------- cipher_suites() -> cipher_suites(erlang). @@ -343,7 +374,7 @@ cipher_suites(openssl) -> [ssl_cipher:openssl_suite_name(S) || S <- ssl_cipher:suites(Version)]. %%-------------------------------------------------------------------- -%% Function: getopts(Socket, OptTags) -> {ok, Options} | {error, Reason} +-spec getopts(#sslsocket{}, [atom()]) -> {ok, [{atom(), term()}]}| {error, reason()}. %% %% Description: %%-------------------------------------------------------------------- @@ -356,7 +387,7 @@ getopts(#sslsocket{} = Socket, Options) -> ssl_broker:getopts(Socket, Options). %%-------------------------------------------------------------------- -%% Function: setopts(Socket, Options) -> ok | {error, Reason} +-spec setopts(#sslsocket{}, [{atom(), term()}]) -> ok | {error, reason()}. %% %% Description: %%-------------------------------------------------------------------- @@ -371,8 +402,8 @@ setopts(#sslsocket{} = Socket, Options) -> ssl_broker:setopts(Socket, Options). %%--------------------------------------------------------------- -%% Function: shutdown(Socket, How) -> ok | {error, Reason} -%% +-spec shutdown(#sslsocket{}, read | write | read_write) -> ok | {error, reason()}. +%% %% Description: Same as gen_tcp:shutdown/2 %%-------------------------------------------------------------------- shutdown(#sslsocket{pid = {ListenSocket, #config{cb={CbMod,_, _, _}}}, fd = new_ssl}, How) -> @@ -381,8 +412,8 @@ shutdown(#sslsocket{pid = Pid, fd = new_ssl}, How) -> ssl_connection:shutdown(Pid, How). %%-------------------------------------------------------------------- -%% Function: sockname(Socket) -> {ok, {Address, Port}} | {error, Reason} -%% +-spec sockname(#sslsocket{}) -> {ok, {tuple(), port_num()}} | {error, reason()}. +%% %% Description: Same as inet:sockname/1 %%-------------------------------------------------------------------- sockname(#sslsocket{fd = new_ssl, pid = {ListenSocket, _}}) -> @@ -396,9 +427,9 @@ sockname(#sslsocket{} = Socket) -> ssl_broker:sockname(Socket). %%--------------------------------------------------------------- -%% Function: seed(Data) -> ok | {error, edata} +-spec seed(term()) ->term(). %% -%% Description: +%% Description: Only used by old ssl. %%-------------------------------------------------------------------- %% TODO: crypto:seed ? seed(Data) -> @@ -406,20 +437,17 @@ seed(Data) -> ssl_server:seed(Data). %%--------------------------------------------------------------- -%% Function: session_id(Socket) -> {ok, PropList} | {error, Reason} +-spec session_info(#sslsocket{}) -> {ok, list()} | {error, reason()}. %% -%% Description: +%% Description: Returns list of session info currently [{session_id, session_id(), +%% {cipher_suite, cipher_suite()}] %%-------------------------------------------------------------------- session_info(#sslsocket{pid = Pid, fd = new_ssl}) -> ssl_connection:session_info(Pid). %%--------------------------------------------------------------- -%% Function: versions() -> [{SslAppVer, SupportedSslVer, AvailableSslVsn}] -%% -%% SslAppVer = string() - t.ex: ssl-4.0 -%% SupportedSslVer = [SslVer] -%% AvailableSslVsn = [SSLVer] -%% SSLVer = sslv3 | tlsv1 | 'tlsv1.1' +-spec versions() -> [{{ssl_app, string()}, {supported, [tls_version()]}, + {available, [tls_version()]}}]. %% %% Description: Returns a list of relevant versions. %%-------------------------------------------------------------------- @@ -429,7 +457,11 @@ versions() -> AvailableVsns = ?DEFAULT_SUPPORTED_VERSIONS, [{ssl_app, ?VSN}, {supported, SupportedVsns}, {available, AvailableVsns}]. - +%%--------------------------------------------------------------- +-spec renegotiate(#sslsocket{}) -> ok | {error, reason()}. +%% +%% Description: +%%-------------------------------------------------------------------- renegotiate(#sslsocket{pid = Pid, fd = new_ssl}) -> ssl_connection:renegotiation(Pid). @@ -633,7 +665,7 @@ validate_option(secure_renegotiate, Value) when Value == true; Value == false -> Value; validate_option(renegotiate_at, Value) when is_integer(Value) -> - min(Value, ?DEFAULT_RENEGOTIATE_AT); + erlang:min(Value, ?DEFAULT_RENEGOTIATE_AT); validate_option(debug, Value) when is_list(Value); Value == true -> Value; @@ -863,10 +895,6 @@ version() -> end, {ok, {SSLVsn, CompVsn, LibVsn}}. -min(N,M) when N < M -> - N; -min(_, M) -> - M. %% Only used to remove exit messages from old ssl %% First is a nonsense clause to provide some diff --git a/lib/ssl/src/ssl_alert.erl b/lib/ssl/src/ssl_alert.erl index db9a883654..eb1228afa4 100644 --- a/lib/ssl/src/ssl_alert.erl +++ b/lib/ssl/src/ssl_alert.erl @@ -32,6 +32,17 @@ -export([alert_txt/1, reason_code/2]). +%%==================================================================== +%% Internal application API +%%==================================================================== +%%-------------------------------------------------------------------- +-spec reason_code(#alert{}, client | server) -> closed | esslconnect | + esslaccept | string(). +%% +%% Description: Returns the error reason that will be returned to the +%% user. +%%-------------------------------------------------------------------- + reason_code(#alert{description = ?CLOSE_NOTIFY}, _) -> closed; reason_code(#alert{description = ?HANDSHAKE_FAILURE}, client) -> @@ -41,10 +52,19 @@ reason_code(#alert{description = ?HANDSHAKE_FAILURE}, server) -> reason_code(#alert{description = Description}, _) -> description_txt(Description). +%%-------------------------------------------------------------------- +-spec alert_txt(#alert{}) -> string(). +%% +%% Description: Returns the error string for given alert. +%%-------------------------------------------------------------------- + alert_txt(#alert{level = Level, description = Description, where = {Mod,Line}}) -> Mod ++ ":" ++ integer_to_list(Line) ++ ":" ++ level_txt(Level) ++" "++ description_txt(Description). +%%-------------------------------------------------------------------- +%%% Internal functions +%%-------------------------------------------------------------------- level_txt(?WARNING) -> "Warning:"; level_txt(?FATAL) -> @@ -96,8 +116,3 @@ description_txt(?USER_CANCELED) -> "user canceled"; description_txt(?NO_RENEGOTIATION) -> "no renegotiation". - - - - - diff --git a/lib/ssl/src/ssl_app.erl b/lib/ssl/src/ssl_app.erl index 6ca1c42631..d9a354086d 100644 --- a/lib/ssl/src/ssl_app.erl +++ b/lib/ssl/src/ssl_app.erl @@ -27,14 +27,16 @@ -export([start/2, stop/1]). -%% start/2(Type, StartArgs) -> {ok, Pid} | {ok, Pid, State} | -%% {error, Reason} -%% +%%-------------------------------------------------------------------- +-spec start(normal | {takeover, node()} | {failover, node()}, list()) -> + {ok, pid()} | {ok, pid(), term()} | {error, term()}. +%%-------------------------------------------------------------------- start(_Type, _StartArgs) -> ssl_sup:start_link(). -%% stop(State) -> void() -%% +%-------------------------------------------------------------------- +-spec stop(term())-> ok. +%%-------------------------------------------------------------------- stop(_State) -> ok. diff --git a/lib/ssl/src/ssl_base64.erl b/lib/ssl/src/ssl_base64.erl deleted file mode 100644 index cfc42407e8..0000000000 --- a/lib/ssl/src/ssl_base64.erl +++ /dev/null @@ -1,129 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% - -%% - -%%% Purpose : Base 64 encoding and decoding. - --module(ssl_base64). - --export([encode/1, encode_split/1, decode/1, join_decode/1]). - --define(st(X,A), ((X-A+256) div 256)). --define(CHARS, 64). - -%% A PEM encoding consists of characters A-Z, a-z, 0-9, +, / and -%% =. Each character encodes a 6 bits value from 0 to 63 (A = 0, / = -%% 63); = is a padding character. -%% - -%% -%% encode(Bytes|Binary) -> Chars -%% -%% Take 3 bytes a time (3 x 8 = 24 bits), and make 4 characters out of -%% them (4 x 6 = 24 bits). -%% -encode(Bs) when is_list(Bs) -> - encode(list_to_binary(Bs)); -encode(<<B:3/binary, Bs/binary>>) -> - <<C1:6, C2:6, C3:6, C4:6>> = B, - [enc(C1), enc(C2), enc(C3), enc(C4)| encode(Bs)]; -encode(<<B:2/binary>>) -> - <<C1:6, C2:6, C3:6, _:6>> = <<B/binary, 0>>, - [enc(C1), enc(C2), enc(C3), $=]; -encode(<<B:1/binary>>) -> - <<C1:6, C2:6, _:12>> = <<B/binary, 0, 0>>, - [enc(C1), enc(C2), $=, $=]; -encode(<<>>) -> - []. - -%% -%% encode_split(Bytes|Binary) -> Lines -%% -%% The encoding is divided into lines separated by <NL>, and each line -%% is precisely 64 characters long (excluding the <NL> characters, -%% except the last line which 64 characters long or shorter. <NL> may -%% follow the last line. -%% -encode_split(Bs) -> - split(encode(Bs)). - -%% -%% decode(Chars) -> Binary -%% -decode(Cs) -> - list_to_binary(decode1(Cs)). - -decode1([C1, C2, $=, $=]) -> - <<B1, _:16>> = <<(dec(C1)):6, (dec(C2)):6, 0:12>>, - [B1]; -decode1([C1, C2, C3, $=]) -> - <<B1, B2, _:8>> = <<(dec(C1)):6, (dec(C2)):6, (dec(C3)):6, (dec(0)):6>>, - [B1, B2]; -decode1([C1, C2, C3, C4| Cs]) -> - Bin = <<(dec(C1)):6, (dec(C2)):6, (dec(C3)):6, (dec(C4)):6>>, - [Bin| decode1(Cs)]; -decode1([]) -> - []. - -%% -%% join_decode(Lines) -> Binary -%% -%% Remove <NL> before decoding. -%% -join_decode(Cs) -> - decode(join(Cs)). - -%% -%% Locals -%% - -%% enc/1 and dec/1 -%% -%% Mapping: 0-25 -> A-Z, 26-51 -> a-z, 52-61 -> 0-9, 62 -> +, 63 -> / -%% -enc(C) -> - 65 + C + 6*?st(C,26) - 75*?st(C,52) -15*?st(C,62) + 3*?st(C,63). - -dec(C) -> - 62*?st(C,43) + ?st(C,47) + (C-59)*?st(C,48) - 69*?st(C,65) - 6*?st(C,97). - -%% split encoding into lines -%% -split(Cs) -> - split(Cs, ?CHARS). - -split([], _N) -> - [$\n]; -split(Cs, 0) -> - [$\n| split(Cs, ?CHARS)]; -split([C| Cs], N) -> - [C| split(Cs, N-1)]. - -%% join lines of encodings -%% -join([$\r, $\n| Cs]) -> - join(Cs); -join([$\n| Cs]) -> - join(Cs); -join([C| Cs]) -> - [C| join(Cs)]; -join([]) -> - []. - diff --git a/lib/ssl/src/ssl_certificate.erl b/lib/ssl/src/ssl_certificate.erl index 9aa31ae8a4..8a79f75725 100644 --- a/lib/ssl/src/ssl_certificate.erl +++ b/lib/ssl/src/ssl_certificate.erl @@ -46,6 +46,14 @@ %% Internal application API %%==================================================================== +%%-------------------------------------------------------------------- +-spec trusted_cert_and_path([der_cert()], certdb_ref(), boolean()) -> + {der_cert(), [der_cert()], list()}. +%% +%% Description: Extracts the root cert (if not presents tries to +%% look it up, if not found {bad_cert, unknown_ca} will be added verification +%% errors. Returns {RootCert, Path, VerifyErrors} +%%-------------------------------------------------------------------- trusted_cert_and_path(CertChain, CertDbRef, Verify) -> [Cert | RestPath] = lists:reverse(CertChain), {ok, OtpCert} = public_key:pkix_decode_cert(Cert, otp), @@ -84,19 +92,31 @@ trusted_cert_and_path(CertChain, CertDbRef, Verify) -> end end. - +%%-------------------------------------------------------------------- +-spec certificate_chain(undefined | binary(), certdb_ref()) -> + {error, no_cert} | [der_cert()]. +%% +%% Description: Return the certificate chain to send to peer. +%%-------------------------------------------------------------------- certificate_chain(undefined, _CertsDbRef) -> {error, no_cert}; certificate_chain(OwnCert, CertsDbRef) -> {ok, ErlCert} = public_key:pkix_decode_cert(OwnCert, otp), certificate_chain(ErlCert, OwnCert, CertsDbRef, [OwnCert]). - +%%-------------------------------------------------------------------- +-spec file_to_certificats(string()) -> [der_cert()]. +%% +%% Description: Return list of DER encoded certificates. +%%-------------------------------------------------------------------- file_to_certificats(File) -> {ok, List} = ssl_manager:cache_pem_file(File), [Bin || {cert, Bin, not_encrypted} <- List]. - - -%% Validates ssl/tls specific extensions +%%-------------------------------------------------------------------- +-spec validate_extensions([#'Extension'{}], term(), [#'Extension'{}], + boolean(), list(), client | server) -> {[#'Extension'{}], term(), list()}. +%% +%% Description: Validates ssl/tls specific extensions +%%-------------------------------------------------------------------- validate_extensions([], ValidationState, UnknownExtensions, _, AccErr, _) -> {UnknownExtensions, ValidationState, AccErr}; @@ -119,21 +139,42 @@ validate_extensions([Extension | Rest], ValidationState, UnknownExtensions, validate_extensions(Rest, ValidationState, [Extension | UnknownExtensions], Verify, AccErr, Role). +%%-------------------------------------------------------------------- +-spec is_valid_key_usage(list(), term()) -> boolean(). +%% +%% Description: Checks if Use is a valid key usage. +%%-------------------------------------------------------------------- is_valid_key_usage(KeyUse, Use) -> lists:member(Use, KeyUse). - select_extension(_, []) -> +%%-------------------------------------------------------------------- +-spec select_extension(term(), list()) -> undefined | #'Extension'{}. +%% +%% Description: Selects the extension identified by Id if present in +%% a list of extensions. +%%-------------------------------------------------------------------- +select_extension(_, []) -> undefined; select_extension(Id, [#'Extension'{extnID = Id} = Extension | _]) -> Extension; select_extension(Id, [_ | Extensions]) -> select_extension(Id, Extensions). +%%-------------------------------------------------------------------- +-spec extensions_list(asn1_NOVALUE | list()) -> list(). +%% +%% Description: Handles that +%%-------------------------------------------------------------------- extensions_list(asn1_NOVALUE) -> []; extensions_list(Extensions) -> Extensions. +%%-------------------------------------------------------------------- +-spec signature_type(term()) -> rsa | dsa . +%% +%% Description: +%%-------------------------------------------------------------------- signature_type(RSA) when RSA == ?sha1WithRSAEncryption; RSA == ?md5WithRSAEncryption -> rsa; diff --git a/lib/ssl/src/ssl_certificate_db.erl b/lib/ssl/src/ssl_certificate_db.erl index b8c3c6f6b7..e953821057 100644 --- a/lib/ssl/src/ssl_certificate_db.erl +++ b/lib/ssl/src/ssl_certificate_db.erl @@ -22,7 +22,7 @@ %%---------------------------------------------------------------------- -module(ssl_certificate_db). - +-include("ssl_internal.hrl"). -include_lib("public_key/include/public_key.hrl"). -export([create/0, remove/1, add_trusted_certs/3, @@ -34,8 +34,7 @@ %%==================================================================== %%-------------------------------------------------------------------- -%% Function: create() -> Db -%% Db = term() - Reference to the crated database +-spec create() -> certdb_ref(). %% %% Description: Creates a new certificate db. %% Note: lookup_trusted_cert/3 may be called from any process but only @@ -47,8 +46,7 @@ create() -> ets:new(ssl_pid_to_file, [bag, private])]. %%-------------------------------------------------------------------- -%% Function: delete(Db) -> _ -%% Db = Database refererence as returned by create/0 +-spec remove(certdb_ref()) -> term(). %% %% Description: Removes database db %%-------------------------------------------------------------------- @@ -56,11 +54,10 @@ remove(Dbs) -> lists:foreach(fun(Db) -> true = ets:delete(Db) end, Dbs). %%-------------------------------------------------------------------- -%% Function: lookup_trusted_cert(Ref, SerialNumber, Issuer) -> {BinCert,DecodedCert} -%% Ref = ref() +-spec lookup_trusted_cert(reference(), serialnumber(), issuer()) -> {der_cert(), #'OTPCertificate'{}}. + %% SerialNumber = integer() %% Issuer = {rdnSequence, IssuerAttrs} -%% BinCert = binary() %% %% Description: Retrives the trusted certificate identified by %% <SerialNumber, Issuer>. Ref is used as it is specified @@ -78,11 +75,7 @@ lookup_cached_certs(File) -> ets:lookup(certificate_db_name(), {file, File}). %%-------------------------------------------------------------------- -%% Function: add_trusted_certs(Pid, File, Db) -> {ok, Ref} -%% Pid = pid() -%% File = string() -%% Db = Database refererence as returned by create/0 -%% Ref = ref() +-spec add_trusted_certs(pid(), string(), certdb_ref()) -> {ok, certdb_ref()}. %% %% Description: Adds the trusted certificates from file <File> to the %% runtime database. Returns Ref that should be handed to lookup_trusted_cert @@ -103,7 +96,7 @@ add_trusted_certs(Pid, File, [CertsDb, FileToRefDb, PidToFileDb]) -> {ok, Ref}. %%-------------------------------------------------------------------- -%% Function: cache_pem_file(Pid, File, Db) -> FileContent +-spec cache_pem_file(pid(), string(), certdb_ref()) -> term(). %% %% Description: Cache file as binary in DB %%-------------------------------------------------------------------- @@ -114,7 +107,8 @@ cache_pem_file(Pid, File, [CertsDb, _FileToRefDb, PidToFileDb]) -> Res. %%-------------------------------------------------------------------- -%% Function: remove_trusted_certs(Pid, Db) -> _ +-spec remove_trusted_certs(pid(), certdb_ref()) -> term(). + %% %% Description: Removes trusted certs originating from %% the file associated to Pid from the runtime database. @@ -144,11 +138,9 @@ remove_trusted_certs(Pid, [CertsDb, FileToRefDb, PidToFileDb]) -> end. %%-------------------------------------------------------------------- -%% Function: issuer_candidate() -> {Key, Candidate} | no_more_candidates +-spec issuer_candidate(no_candidate | cert_key()) -> + {cert_key(), der_cert()} | no_more_candidates. %% -%% Candidate -%% -%% %% Description: If a certificat does not define its issuer through %% the extension 'ce-authorityKeyIdentifier' we can %% try to find the issuer in the database over known diff --git a/lib/ssl/src/ssl_cipher.erl b/lib/ssl/src/ssl_cipher.erl index 2a71df8ee1..a6e80047c2 100644 --- a/lib/ssl/src/ssl_cipher.erl +++ b/lib/ssl/src/ssl_cipher.erl @@ -40,11 +40,8 @@ -compile(inline). %%-------------------------------------------------------------------- -%% Function: security_parameters(CipherSuite, SecParams) -> -%% #security_parameters{} -%% -%% CipherSuite - as defined in ssl_cipher.hrl -%% SecParams - #security_parameters{} +-spec security_parameters(erl_cipher_suite(), #security_parameters{}) -> + #security_parameters{}. %% %% Description: Returns a security parameters record where the %% cipher values has been updated according to <CipherSuite> @@ -63,15 +60,11 @@ security_parameters(CipherSuite, SecParams) -> hash_size = hash_size(Hash)}. %%-------------------------------------------------------------------- -%% Function: cipher(Method, CipherState, Mac, Data) -> -%% {Encrypted, UpdateCipherState} -%% -%% Method - integer() (as defined in ssl_cipher.hrl) -%% CipherState, UpdatedCipherState - #cipher_state{} -%% Data, Encrypted - binary() +-spec cipher(cipher_enum(), #cipher_state{}, binary(), binary()) -> + {binary(), #cipher_state{}}. %% -%% Description: Encrypts the data and the mac using method, updating -%% the cipher state +%% Description: Encrypts the data and the MAC using chipher described +%% by cipher_enum() and updating the cipher state %%------------------------------------------------------------------- cipher(?NULL, CipherState, <<>>, Fragment) -> GenStreamCipherList = [Fragment, <<>>], @@ -125,15 +118,11 @@ block_cipher(Fun, BlockSz, #cipher_state{key=Key, iv=IV} = CS0, {T, CS0#cipher_state{iv=NextIV}}. %%-------------------------------------------------------------------- -%% Function: decipher(Method, CipherState, Mac, Data, Version) -> -%% {Decrypted, UpdateCipherState} -%% -%% Method - integer() (as defined in ssl_cipher.hrl) -%% CipherState, UpdatedCipherState - #cipher_state{} -%% Data, Encrypted - binary() +-spec decipher(cipher_enum(), integer(), #cipher_state{}, binary(), tls_version()) -> + {binary(), #cipher_state{}}. %% -%% Description: Decrypts the data and the mac using method, updating -%% the cipher state +%% Description: Decrypts the data and the MAC using cipher described +%% by cipher_enum() and updating the cipher state. %%------------------------------------------------------------------- decipher(?NULL, _HashSz, CipherState, Fragment, _) -> {Fragment, <<>>, CipherState}; @@ -192,10 +181,7 @@ block_decipher(Fun, #cipher_state{key=Key, iv=IV} = CipherState0, end. %%-------------------------------------------------------------------- -%% Function: suites(Version) -> [Suite] -%% -%% Version = version() -%% Suite = binary() from ssl_cipher.hrl +-spec suites(tls_version()) -> [cipher_suite()]. %% %% Description: Returns a list of supported cipher suites. %%-------------------------------------------------------------------- @@ -205,19 +191,9 @@ suites({3, N}) when N == 1; N == 2 -> ssl_tls1:suites(). %%-------------------------------------------------------------------- -%% Function: suite_definition(CipherSuite) -> -%% {KeyExchange, Cipher, Hash} -%% -%% -%% CipherSuite - as defined in ssl_cipher.hrl -%% KeyExchange - rsa | dh_anon | dhe_dss | dhe_rsa | kerb5 -%% -%% Cipher - null | rc4_128 | idea_cbc | des_cbc | '3des_ede_cbc' -%% des40_cbc | aes_128_cbc | aes_256_cbc -%% Hash - null | md5 | sha +-spec suite_definition(cipher_suite()) -> erl_cipher_suite(). %% -%% Description: Returns a security parameters tuple where the -%% cipher values has been updated according to <CipherSuite> +%% Description: Return erlang cipher suite definition. %% Note: Currently not supported suites are commented away. %% They should be supported or removed in the future. %%------------------------------------------------------------------- @@ -261,6 +237,12 @@ suite_definition(?TLS_DHE_DSS_WITH_AES_256_CBC_SHA) -> suite_definition(?TLS_DHE_RSA_WITH_AES_256_CBC_SHA) -> {dhe_rsa, aes_256_cbc, sha}. +%%-------------------------------------------------------------------- +-spec suite(erl_cipher_suite()) -> cipher_suite(). +%% +%% Description: Return TLS cipher suite definition. +%%-------------------------------------------------------------------- + %% TLS v1.1 suites %%suite({rsa, null, md5}) -> %% ?TLS_RSA_WITH_NULL_MD5; @@ -309,7 +291,11 @@ suite({dhe_rsa, aes_256_cbc, sha}) -> %% suite({dh_anon, aes_256_cbc, sha}) -> %% ?TLS_DH_anon_WITH_AES_256_CBC_SHA. - +%%-------------------------------------------------------------------- +-spec openssl_suite(openssl_cipher_suite()) -> cipher_suite(). +%% +%% Description: Return TLS cipher suite definition. +%%-------------------------------------------------------------------- %% translate constants <-> openssl-strings openssl_suite("DHE-RSA-AES256-SHA") -> ?TLS_DHE_RSA_WITH_AES_256_CBC_SHA; @@ -339,7 +325,11 @@ openssl_suite("EDH-RSA-DES-CBC-SHA") -> ?TLS_DHE_RSA_WITH_DES_CBC_SHA; openssl_suite("DES-CBC-SHA") -> ?TLS_RSA_WITH_DES_CBC_SHA. - +%%-------------------------------------------------------------------- +-spec openssl_suite_name(cipher_suite()) -> openssl_cipher_suite(). +%% +%% Description: Return openssl cipher suite name. +%%------------------------------------------------------------------- openssl_suite_name(?TLS_DHE_RSA_WITH_AES_256_CBC_SHA) -> "DHE-RSA-AES256-SHA"; openssl_suite_name(?TLS_DHE_DSS_WITH_AES_256_CBC_SHA) -> @@ -372,6 +362,11 @@ openssl_suite_name(?TLS_RSA_WITH_DES_CBC_SHA) -> openssl_suite_name(Cipher) -> suite_definition(Cipher). +%%-------------------------------------------------------------------- +-spec filter(undefined | binary(), [cipher_suite()]) -> [cipher_suite()]. +%% +%% Description: . +%%------------------------------------------------------------------- filter(undefined, Ciphers) -> Ciphers; filter(DerCert, Ciphers) -> diff --git a/lib/ssl/src/ssl_cipher.hrl b/lib/ssl/src/ssl_cipher.hrl index 80fe527f45..19de709d9c 100644 --- a/lib/ssl/src/ssl_cipher.hrl +++ b/lib/ssl/src/ssl_cipher.hrl @@ -26,6 +26,14 @@ -ifndef(ssl_cipher). -define(ssl_cipher, true). +-type cipher() :: null |rc4_128 | idea_cbc | des40_cbc | des_cbc | '3des_ede_cbc' + | aes_128_cbc | aes_256_cbc. +-type hash() :: sha | md5. +-type erl_cipher_suite() :: {key_algo(), cipher(), hash()}. +-type cipher_suite() :: binary(). +-type cipher_enum() :: integer(). +-type openssl_cipher_suite() :: string(). + %%% SSL cipher protocol %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -define(CHANGE_CIPHER_SPEC_PROTO, 1). % _PROTO to not clash with % SSL record protocol diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index abd1b59011..5b4b129e30 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -98,12 +98,17 @@ #'DHParameter'{prime = ?DEFAULT_DIFFIE_HELLMAN_PRIME, base = ?DEFAULT_DIFFIE_HELLMAN_GENERATOR}). +-type state_name() :: hello | abbreviated | certify | cipher | connection. +-type gen_fsm_state_return() :: {next_state, state_name(), #state{}} | + {next_state, state_name(), #state{}, timeout()} | + {stop, term(), #state{}}. + %%==================================================================== %% Internal application API %%==================================================================== %%-------------------------------------------------------------------- -%% Function: send(Pid, Data) -> ok | {error, Reason} +-spec send(pid(), iolist()) -> ok | {error, reason()}. %% %% Description: Sends data over the ssl connection %%-------------------------------------------------------------------- @@ -112,34 +117,43 @@ send(Pid, Data) -> erlang:iolist_to_binary(Data)}, infinity). %%-------------------------------------------------------------------- -%% Function: recv(Socket, Length Timeout) -> {ok, Data} | {error, reason} +-spec recv(pid(), integer(), timeout()) -> + {ok, binary() | list()} | {error, reason()}. %% %% Description: Receives data when active = false %%-------------------------------------------------------------------- recv(Pid, Length, Timeout) -> sync_send_all_state_event(Pid, {recv, Length}, Timeout). %%-------------------------------------------------------------------- -%% Function: : connect(Host, Port, Socket, Options, -%% User, CbInfo, Timeout) -> {ok, Socket} +-spec connect(host(), port_num(), port(), list(), pid(), tuple(), timeout()) -> + {ok, #sslsocket{}} | {error, reason()}. %% %% Description: Connect to a ssl server. %%-------------------------------------------------------------------- connect(Host, Port, Socket, Options, User, CbInfo, Timeout) -> - start_fsm(client, Host, Port, Socket, Options, User, CbInfo, - Timeout). + try start_fsm(client, Host, Port, Socket, Options, User, CbInfo, + Timeout) + catch + exit:{noproc, _} -> + {error, ssl_not_started} + end. %%-------------------------------------------------------------------- -%% Function: accept(Port, Socket, Opts, User, -%% CbInfo, Timeout) -> {ok, Socket} | {error, Reason} +-spec ssl_accept(port_num(), port(), list(), pid(), tuple(), timeout()) -> + {ok, #sslsocket{}} | {error, reason()}. %% %% Description: Performs accept on a ssl listen socket. e.i. performs %% ssl handshake. %%-------------------------------------------------------------------- ssl_accept(Port, Socket, Opts, User, CbInfo, Timeout) -> - start_fsm(server, "localhost", Port, Socket, Opts, User, - CbInfo, Timeout). + try start_fsm(server, "localhost", Port, Socket, Opts, User, + CbInfo, Timeout) + catch + exit:{noproc, _} -> + {error, ssl_not_started} + end. %%-------------------------------------------------------------------- -%% Function: handshake(SslSocket, Timeout) -> ok | {error, Reason} +-spec handshake(#sslsocket{}, timeout()) -> ok | {error, reason()}. %% %% Description: Starts ssl handshake. %%-------------------------------------------------------------------- @@ -151,7 +165,8 @@ handshake(#sslsocket{pid = Pid}, Timeout) -> Error end. %-------------------------------------------------------------------- -%% Function: socket_control(Pid) -> {ok, SslSocket} | {error, Reason} +-spec socket_control(port(), pid(), atom()) -> + {ok, #sslsocket{}} | {error, reason()}. %% %% Description: Set the ssl process to own the accept socket %%-------------------------------------------------------------------- @@ -164,7 +179,7 @@ socket_control(Socket, Pid, CbModule) -> end. %%-------------------------------------------------------------------- -%% Function: close() -> ok +-spec close(pid()) -> ok | {error, reason()}. %% %% Description: Close a ssl connection %%-------------------------------------------------------------------- @@ -177,7 +192,7 @@ close(ConnectionPid) -> end. %%-------------------------------------------------------------------- -%% Function: shutdown(Socket, How) -> ok | {error, Reason} +-spec shutdown(pid(), atom()) -> ok | {error, reason()}. %% %% Description: Same as gen_tcp:shutdown/2 %%-------------------------------------------------------------------- @@ -185,7 +200,7 @@ shutdown(ConnectionPid, How) -> sync_send_all_state_event(ConnectionPid, {shutdown, How}). %%-------------------------------------------------------------------- -%% Function: new_user(ConnectionPid, User) -> ok | {error, Reason} +-spec new_user(pid(), pid()) -> ok | {error, reason()}. %% %% Description: Changes process that receives the messages when active = true %% or once. @@ -193,28 +208,28 @@ shutdown(ConnectionPid, How) -> new_user(ConnectionPid, User) -> sync_send_all_state_event(ConnectionPid, {new_user, User}). %%-------------------------------------------------------------------- -%% Function: sockname(ConnectionPid) -> {ok, {Address, Port}} | {error, Reason} +-spec sockname(pid()) -> {ok, {tuple(), port_num()}} | {error, reason()}. %% %% Description: Same as inet:sockname/1 %%-------------------------------------------------------------------- sockname(ConnectionPid) -> sync_send_all_state_event(ConnectionPid, sockname). %%-------------------------------------------------------------------- -%% Function: peername(ConnectionPid) -> {ok, {Address, Port}} | {error, Reason} +-spec peername(pid()) -> {ok, {tuple(), port_num()}} | {error, reason()}. %% %% Description: Same as inet:peername/1 %%-------------------------------------------------------------------- peername(ConnectionPid) -> sync_send_all_state_event(ConnectionPid, peername). %%-------------------------------------------------------------------- -%% Function: get_opts(ConnectionPid, OptTags) -> {ok, Options} | {error, Reason} +-spec get_opts(pid(), list()) -> {ok, list()} | {error, reason()}. %% %% Description: Same as inet:getopts/2 %%-------------------------------------------------------------------- get_opts(ConnectionPid, OptTags) -> sync_send_all_state_event(ConnectionPid, {get_opts, OptTags}). %%-------------------------------------------------------------------- -%% Function: setopts(Socket, Options) -> ok | {error, Reason} +-spec set_opts(pid(), list()) -> ok | {error, reason()}. %% %% Description: Same as inet:setopts/2 %%-------------------------------------------------------------------- @@ -222,8 +237,7 @@ set_opts(ConnectionPid, Options) -> sync_send_all_state_event(ConnectionPid, {set_opts, Options}). %%-------------------------------------------------------------------- -%% Function: info(ConnectionPid) -> {ok, {Protocol, CipherSuite}} | -%% {error, Reason} +-spec info(pid()) -> {ok, {atom(), tuple()}} | {error, reason()}. %% %% Description: Returns ssl protocol and cipher used for the connection %%-------------------------------------------------------------------- @@ -231,7 +245,7 @@ info(ConnectionPid) -> sync_send_all_state_event(ConnectionPid, info). %%-------------------------------------------------------------------- -%% Function: session_info(ConnectionPid) -> {ok, PropList} | {error, Reason} +-spec session_info(pid()) -> {ok, list()} | {error, reason()}. %% %% Description: Returns info about the ssl session %%-------------------------------------------------------------------- @@ -239,7 +253,7 @@ session_info(ConnectionPid) -> sync_send_all_state_event(ConnectionPid, session_info). %%-------------------------------------------------------------------- -%% Function: peercert(ConnectionPid) -> {ok, Cert} | {error, Reason} +-spec peer_certificate(pid()) -> {ok, binary()} | {error, reason()}. %% %% Description: Returns the peer cert %%-------------------------------------------------------------------- @@ -247,7 +261,7 @@ peer_certificate(ConnectionPid) -> sync_send_all_state_event(ConnectionPid, peer_certificate). %%-------------------------------------------------------------------- -%% Function: renegotiation(ConnectionPid) -> ok | {error, Reason} +-spec renegotiation(pid()) -> ok | {error, reason()}. %% %% Description: Starts a renegotiation of the ssl session. %%-------------------------------------------------------------------- @@ -259,7 +273,8 @@ renegotiation(ConnectionPid) -> %%==================================================================== %%-------------------------------------------------------------------- -%% Function: start_link() -> {ok,Pid} | ignore | {error,Error} +-spec start_link(atom(), host(), port_num(), port(), list(), pid(), tuple()) -> + {ok, pid()} | ignore | {error, reason()}. %% %% Description: Creates a gen_fsm process which calls Module:init/1 to %% initialize. To ensure a synchronized start-up procedure, this function @@ -273,10 +288,9 @@ start_link(Role, Host, Port, Socket, Options, User, CbInfo) -> %% gen_fsm callbacks %%==================================================================== %%-------------------------------------------------------------------- -%% Function: init(Args) -> {ok, StateName, State} | -%% {ok, StateName, State, Timeout} | -%% ignore | -%% {stop, StopReason} +-spec init(list()) -> {ok, state_name(), #state{}} + | {ok, state_name(), #state{}, timeout()} | + ignore | {stop, term()}. %% Description:Whenever a gen_fsm is started using gen_fsm:start/[3,4] or %% gen_fsm:start_link/3,4, this function is called by the new process to %% initialize. @@ -301,11 +315,7 @@ init([Role, Host, Port, Socket, {SSLOpts0, _} = Options, end. %%-------------------------------------------------------------------- -%% Function: -%% state_name(Event, State) -> {next_state, NextStateName, NextState}| -%% {next_state, NextStateName, -%% NextState, Timeout} | -%% {stop, Reason, NewState} +%% -spec state_name(event(), #state{}) -> gen_fsm_state_return() %% %% Description:There should be one instance of this function for each %% possible state name. Whenever a gen_fsm receives an event sent @@ -314,6 +324,9 @@ init([Role, Host, Port, Socket, {SSLOpts0, _} = Options, %% the event. It is also called if a timeout occurs. %% %%-------------------------------------------------------------------- +-spec hello(start | #hello_request{} | #client_hello{} | #server_hello{} | term(), + #state{}) -> gen_fsm_state_return(). +%%-------------------------------------------------------------------- hello(start, #state{host = Host, port = Port, role = client, ssl_options = SslOpts, transport_cb = Transport, socket = Socket, @@ -351,49 +364,30 @@ hello(#hello_request{}, #state{role = client} = State0) -> hello(#server_hello{cipher_suite = CipherSuite, compression_method = Compression} = Hello, - #state{session = Session0 = #session{session_id = OldId}, + #state{session = #session{session_id = OldId}, connection_states = ConnectionStates0, role = client, negotiated_version = ReqVersion, - host = Host, port = Port, renegotiation = {Renegotiation, _}, - ssl_options = SslOptions, - session_cache = Cache, - session_cache_cb = CacheCb} = State0) -> + ssl_options = SslOptions} = State0) -> case ssl_handshake:hello(Hello, SslOptions, ConnectionStates0, Renegotiation) of - {Version, NewId, ConnectionStates1} -> + {Version, NewId, ConnectionStates} -> {KeyAlgorithm, _, _} = ssl_cipher:suite_definition(CipherSuite), PremasterSecret = make_premaster_secret(ReqVersion, KeyAlgorithm), - State1 = State0#state{key_algorithm = KeyAlgorithm, - negotiated_version = Version, - connection_states = ConnectionStates1, - premaster_secret = PremasterSecret}, + State = State0#state{key_algorithm = KeyAlgorithm, + negotiated_version = Version, + connection_states = ConnectionStates, + premaster_secret = PremasterSecret}, case ssl_session:is_new(OldId, NewId) of true -> - Session = Session0#session{session_id = NewId, - cipher_suite = CipherSuite, - compression_method = Compression}, - {Record, State} = next_record(State1#state{session = Session}), - next_state(certify, Record, State); + handle_new_session(NewId, CipherSuite, Compression, State); false -> - Session = CacheCb:lookup(Cache, {{Host, Port}, NewId}), - case ssl_handshake:master_secret(Version, Session, - ConnectionStates1, client) of - {_, ConnectionStates2} -> - {Record, State} = - next_record(State1#state{ - connection_states = ConnectionStates2, - session = Session}), - next_state(abbreviated, Record, State); - #alert{} = Alert -> - handle_own_alert(Alert, Version, hello, State1), - {stop, normal, State1} - end + handle_resumed_session(NewId, State#state{connection_states = ConnectionStates}) end; #alert{} = Alert -> handle_own_alert(Alert, ReqVersion, hello, State0), @@ -423,7 +417,10 @@ hello(Hello = #client_hello{client_version = ClientVersion}, hello(Msg, State) -> handle_unexpected_message(Msg, hello, State). - +%%-------------------------------------------------------------------- +-spec abbreviated(#hello_request{} | #finished{} | term(), + #state{}) -> gen_fsm_state_return(). +%%-------------------------------------------------------------------- abbreviated(#hello_request{}, State0) -> {Record, State} = next_record(State0), next_state(hello, Record, State); @@ -469,6 +466,11 @@ abbreviated(#finished{verify_data = Data} = Finished, abbreviated(Msg, State) -> handle_unexpected_message(Msg, abbreviated, State). +%%-------------------------------------------------------------------- +-spec certify(#hello_request{} | #certificate{} | #server_key_exchange{} | + #certificate_request{} | #server_hello_done{} | #client_key_exchange{} | term(), + #state{}) -> gen_fsm_state_return(). +%%-------------------------------------------------------------------- certify(#hello_request{}, State0) -> {Record, State} = next_record(State0), next_state(hello, Record, State); @@ -638,6 +640,10 @@ certify(#client_key_exchange{exchange_keys = #client_diffie_hellman_public{ certify(Msg, State) -> handle_unexpected_message(Msg, certify, State). +%%-------------------------------------------------------------------- +-spec cipher(#hello_request{} | #certificate_verify{} | #finished{} | term(), + #state{}) -> gen_fsm_state_return(). +%%-------------------------------------------------------------------- cipher(#hello_request{}, State0) -> {Record, State} = next_record(State0), next_state(hello, Record, State); @@ -668,31 +674,14 @@ cipher(#finished{verify_data = Data} = Finished, role = Role, session = #session{master_secret = MasterSecret} = Session0, - tls_handshake_hashes = Hashes0, - connection_states = ConnectionStates0} = State) -> + tls_handshake_hashes = Hashes0} = State) -> case ssl_handshake:verify_connection(Version, Finished, opposite_role(Role), MasterSecret, Hashes0) of verified -> Session = register_session(Role, Host, Port, Session0), - case Role of - client -> - ConnectionStates = ssl_record:set_server_verify_data(current_both, Data, ConnectionStates0), - next_state_connection(cipher, ack_connection(State#state{session = Session, - connection_states = ConnectionStates})); - server -> - ConnectionStates1 = ssl_record:set_client_verify_data(current_read, Data, ConnectionStates0), - {ConnectionStates, Hashes} = - finalize_handshake(State#state{ - connection_states = ConnectionStates1, - session = Session}, cipher), - next_state_connection(cipher, ack_connection(State#state{connection_states = - ConnectionStates, - session = Session, - tls_handshake_hashes = - Hashes})) - end; + cipher_role(Role, Data, Session, State); #alert{} = Alert -> handle_own_alert(Alert, Version, cipher, State), {stop, normal, State} @@ -701,7 +690,10 @@ cipher(#finished{verify_data = Data} = Finished, cipher(Msg, State) -> handle_unexpected_message(Msg, cipher, State). - +%%-------------------------------------------------------------------- +-spec connection(#hello_request{} | #client_hello{} | term(), + #state{}) -> gen_fsm_state_return(). +%%-------------------------------------------------------------------- connection(#hello_request{}, #state{host = Host, port = Port, socket = Socket, ssl_options = SslOpts, @@ -728,30 +720,22 @@ connection(#client_hello{} = Hello, #state{role = server} = State) -> connection(Msg, State) -> handle_unexpected_message(Msg, connection, State). %%-------------------------------------------------------------------- -%% Function: -%% handle_event(Event, StateName, State) -> {next_state, NextStateName, -%% NextState} | -%% {next_state, NextStateName, -%% NextState, Timeout} | -%% {stop, Reason, NewState} +-spec handle_event(term(), state_name(), #state{}) -> gen_fsm_state_return(). +%% %% Description: Whenever a gen_fsm receives an event sent using %% gen_fsm:send_all_state_event/2, this function is called to handle -%% the event. +%% the event. Not currently used! %%-------------------------------------------------------------------- handle_event(_Event, StateName, State) -> {next_state, StateName, State}. %%-------------------------------------------------------------------- -%% Function: -%% handle_sync_event(Event, From, StateName, -%% State) -> {next_state, NextStateName, NextState} | -%% {next_state, NextStateName, NextState, -%% Timeout} | -%% {reply, Reply, NextStateName, NextState}| -%% {reply, Reply, NextStateName, NextState, -%% Timeout} | -%% {stop, Reason, NewState} | -%% {stop, Reason, Reply, NewState} +-spec handle_sync_event(term(), from(), state_name(), #state{}) -> + gen_fsm_state_return() | + {reply, reply(), state_name(), #state{}} | + {reply, reply(), state_name(), #state{}, timeout()} | + {stop, reason(), reply(), #state{}}. +%% %% Description: Whenever a gen_fsm receives an event sent using %% gen_fsm:sync_send_all_state_event/2,3, this function is called to handle %% the event. @@ -928,11 +912,11 @@ handle_sync_event(peer_certificate, _, StateName, {reply, {ok, Cert}, StateName, State}. %%-------------------------------------------------------------------- -%% Function: -%% handle_info(Info,StateName,State)-> {next_state, NextStateName, NextState}| -%% {next_state, NextStateName, NextState, -%% Timeout} | -%% {stop, Reason, NewState} +-spec handle_info(msg(),state_name(), #state{}) -> + {next_state, state_name(), #state{}}| + {next_state, state_name(), #state{}, timeout()} | + {stop, reason(), #state{}}. +%% %% Description: This function is called by a gen_fsm when it receives any %% other message than a synchronous or asynchronous event %% (or a system message). @@ -992,7 +976,8 @@ handle_info(Msg, StateName, State) -> {next_state, StateName, State}. %%-------------------------------------------------------------------- -%% Function: terminate(Reason, StateName, State) -> void() +-spec terminate(reason(), state_name(), #state{}) -> term(). +%% %% Description:This function is called by a gen_fsm when it is about %% to terminate. It should be the opposite of Module:init/1 and do any %% necessary cleaning up. When it returns, the gen_fsm terminates with @@ -1019,7 +1004,8 @@ terminate(_Reason, _StateName, #state{transport_cb = Transport, Transport:close(Socket). %%-------------------------------------------------------------------- -%% Function: +-spec code_change(term(), state_name(), #state{}, list()) -> {ok, state_name(), #state{}}. +%% %% code_change(OldVsn, StateName, State, Extra) -> {ok, StateName, NewState} %% Description: Convert process state when code is changed %%-------------------------------------------------------------------- @@ -1031,24 +1017,17 @@ code_change(_OldVsn, StateName, State, _Extra) -> %%-------------------------------------------------------------------- start_fsm(Role, Host, Port, Socket, Opts, User, {CbModule, _,_, _} = CbInfo, Timeout) -> - case ssl_connection_sup:start_child([Role, Host, Port, Socket, - Opts, User, CbInfo]) of - {ok, Pid} -> - case socket_control(Socket, Pid, CbModule) of - {ok, SslSocket} -> - case handshake(SslSocket, Timeout) of - ok -> - {ok, SslSocket}; - {error, Reason} -> - {error, Reason} - end; - {error, Reason} -> - {error, Reason} - end; - {error, Reason} -> - {error, Reason} + try + {ok, Pid} = ssl_connection_sup:start_child([Role, Host, Port, Socket, + Opts, User, CbInfo]), + {ok, SslSocket} = socket_control(Socket, Pid, CbModule), + ok = handshake(SslSocket, Timeout), + {ok, SslSocket} + catch + error:{badmatch, {error, _} = Error} -> + Error end. - + ssl_init(SslOpts, Role) -> {ok, CertDbRef, CacheRef, OwnCert} = init_certificates(SslOpts, Role), PrivateKey = @@ -1259,6 +1238,33 @@ do_server_hello(#server_hello{cipher_suite = CipherSuite, {stop, normal, State0} end. +handle_new_session(NewId, CipherSuite, Compression, #state{session = Session0} = State0) -> + Session = Session0#session{session_id = NewId, + cipher_suite = CipherSuite, + compression_method = Compression}, + {Record, State} = next_record(State0#state{session = Session}), + next_state(certify, Record, State). + +handle_resumed_session(SessId, #state{connection_states = ConnectionStates0, + negotiated_version = Version, + host = Host, port = Port, + session_cache = Cache, + session_cache_cb = CacheCb} = State0) -> + Session = CacheCb:lookup(Cache, {{Host, Port}, SessId}), + case ssl_handshake:master_secret(Version, Session, + ConnectionStates0, client) of + {_, ConnectionStates1} -> + {Record, State} = + next_record(State0#state{ + connection_states = ConnectionStates1, + session = Session}), + next_state(abbreviated, Record, State); + #alert{} = Alert -> + handle_own_alert(Alert, Version, hello, State0), + {stop, normal, State0} + end. + + client_certify_and_key_exchange(#state{negotiated_version = Version} = State0) -> try do_client_certify_and_key_exchange(State0) of @@ -1346,9 +1352,7 @@ key_exchange(#state{role = server, key_algorithm = Algo, transport_cb = Transport } = State) when Algo == dhe_dss; - Algo == dhe_dss_export; - Algo == dhe_rsa; - Algo == dhe_rsa_export -> + Algo == dhe_rsa -> Keys = public_key:gen_key(Params), ConnectionState = @@ -1394,9 +1398,7 @@ key_exchange(#state{role = client, socket = Socket, transport_cb = Transport, tls_handshake_hashes = Hashes0} = State) when Algorithm == dhe_dss; - Algorithm == dhe_dss_export; - Algorithm == dhe_rsa; - Algorithm == dhe_rsa_export -> + Algorithm == dhe_rsa -> Msg = ssl_handshake:key_exchange(client, {dh, DhPubKey}), {BinMsg, ConnectionStates1, Hashes1} = encode_handshake(Msg, Version, ConnectionStates0, Hashes0), @@ -1492,15 +1494,15 @@ handle_server_key( SecParams = ConnectionState#connection_state.security_parameters, #security_parameters{client_random = ClientRandom, server_random = ServerRandom} = SecParams, - Plain = ssl_handshake:server_key_exchange_plain(KeyAlgo, - <<ClientRandom/binary, + Hash = ssl_handshake:server_key_exchange_hash(KeyAlgo, + <<ClientRandom/binary, ServerRandom/binary, - ?UINT16(PLen), P/binary, - ?UINT16(GLen), G/binary, - ?UINT16(YLen), + ?UINT16(PLen), P/binary, + ?UINT16(GLen), G/binary, + ?UINT16(YLen), ServerPublicDhKey/binary>>), - - case verify_dh_params(Signed, Plain, PubKeyInfo) of + + case verify_dh_params(Signed, Hash, PubKeyInfo) of true -> PMpint = mpint_binary(P), GMpint = mpint_binary(G), @@ -1533,10 +1535,25 @@ verify_dh_params(Signed, Hashes, {?rsaEncryption, PubKey, _PubKeyParams}) -> _ -> false end; -verify_dh_params(Signed, Plain, {?'id-dsa', PublicKey, PublicKeyParams}) -> - public_key:verify_signature(Plain, sha, Signed, PublicKey, PublicKeyParams). +verify_dh_params(Signed, Hash, {?'id-dsa', PublicKey, PublicKeyParams}) -> + public_key:verify_signature(Hash, none, Signed, PublicKey, PublicKeyParams). +cipher_role(client, Data, Session, #state{connection_states = ConnectionStates0} = State) -> + ConnectionStates = ssl_record:set_server_verify_data(current_both, Data, ConnectionStates0), + next_state_connection(cipher, ack_connection(State#state{session = Session, + connection_states = ConnectionStates})); + +cipher_role(server, Data, Session, #state{connection_states = ConnectionStates0} = State) -> + ConnectionStates1 = ssl_record:set_client_verify_data(current_read, Data, ConnectionStates0), + {ConnectionStates, Hashes} = + finalize_handshake(State#state{connection_states = ConnectionStates1, + session = Session}, cipher), + next_state_connection(cipher, ack_connection(State#state{connection_states = + ConnectionStates, + session = Session, + tls_handshake_hashes = + Hashes})). encode_alert(#alert{} = Alert, Version, ConnectionStates) -> ?DBG_TERM(Alert), ssl_record:encode_alert_record(Alert, Version, ConnectionStates). diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index c8245e2fb4..fcc30f6137 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -35,23 +35,22 @@ hello_request/0, certify/7, certificate/3, client_certificate_verify/6, certificate_verify/6, certificate_request/2, - key_exchange/2, server_key_exchange_plain/2, finished/4, + key_exchange/2, server_key_exchange_hash/2, finished/4, verify_connection/5, get_tls_handshake/4, server_hello_done/0, sig_alg/1, encode_handshake/3, init_hashes/0, update_hashes/2, decrypt_premaster_secret/2]). +-type tls_handshake() :: #client_hello{} | #server_hello{} | #server_hello_done{} | +#certificate{} | #client_key_exchange{} | #finished{} | #certificate_verify{}. + %%==================================================================== %% Internal application API %%==================================================================== %%-------------------------------------------------------------------- -%% Function: client_hello(Host, Port, ConnectionStates, SslOpts, Cert, Renegotiation) -> -%% #client_hello{} -%% Host -%% Port -%% ConnectionStates = #connection_states{} -%% SslOpts = #ssl_options{} +-spec client_hello(host(), port_num(), #connection_states{}, + #ssl_options{}, binary(), boolean()) -> #client_hello{}. %% %% Description: Creates a client hello message. %%-------------------------------------------------------------------- @@ -79,13 +78,8 @@ client_hello(Host, Port, ConnectionStates, #ssl_options{versions = Versions, }. %%-------------------------------------------------------------------- -%% Function: server_hello(SessionId, Version, -%% ConnectionStates, Renegotiation) -> #server_hello{} -%% SessionId -%% Version -%% ConnectionStates -%% Renegotiation -%% +-spec server_hello(session_id(), tls_version(), #connection_states{}, + boolean()) -> #server_hello{}. %% %% Description: Creates a server hello message. %%-------------------------------------------------------------------- @@ -103,7 +97,7 @@ server_hello(SessionId, Version, ConnectionStates, Renegotiation) -> }. %%-------------------------------------------------------------------- -%% Function: hello_request() -> #hello_request{} +-spec hello_request() -> #hello_request{}. %% %% Description: Creates a hello request message sent by server to %% trigger renegotiation. @@ -112,15 +106,12 @@ hello_request() -> #hello_request{}. %%-------------------------------------------------------------------- -%% Function: hello(Hello, Info, Renegotiation) -> -%% {Version, Id, NewConnectionStates} | -%% #alert{} -%% -%% Hello = #client_hello{} | #server_hello{} -%% Info = ConnectionStates | {Port, #ssl_options{}, Session, -%% Cahce, CahceCb, ConnectionStates} -%% ConnectionStates = #connection_states{} -%% Renegotiation = boolean() +-spec hello(#server_hello{} | #client_hello{}, #ssl_options{}, + #connection_states{} | {port_num(), #session{}, cache_ref(), + atom(), #connection_states{}, binary()}, + boolean()) -> {tls_version(), session_id(), #connection_states{}}| + {tls_version(), {resumed | new, session_id()}, + #connection_states{}} | #alert{}. %% %% Description: Handles a recieved hello message %%-------------------------------------------------------------------- @@ -183,12 +174,9 @@ hello(#client_hello{client_version = ClientVersion, random = Random, end. %%-------------------------------------------------------------------- -%% Function: certify(Certs, CertDbRef, MaxPathLen) -> -%% {PeerCert, PublicKeyInfo} | #alert{} -%% -%% Certs = #certificate{} -%% CertDbRef = reference() -%% MaxPathLen = integer() | nolimit +-spec certify(#certificate{}, term(), integer() | nolimit, + verify_peer | verify_none, fun(), fun(), + client | server) -> {der_cert(), public_key_info()} | #alert{}. %% %% Description: Handles a certificate handshake message %%-------------------------------------------------------------------- @@ -244,10 +232,7 @@ certify(#certificate{asn1_certificates = ASN1Certs}, CertDbRef, end. %%-------------------------------------------------------------------- -%% Function: certificate(OwnCert, CertDbRef, Role) -> #certificate{} -%% -%% OwnCert = binary() -%% CertDbRef = term() as returned by ssl_certificate_db:create() +-spec certificate(der_cert(), term(), client | server) -> #certificate{}. %% %% Description: Creates a certificate message. %%-------------------------------------------------------------------- @@ -273,10 +258,10 @@ certificate(OwnCert, CertDbRef, server) -> end. %%-------------------------------------------------------------------- -%% Function: client_certificate_verify(Cert, ConnectionStates) -> -%% #certificate_verify{} | ignore -%% Cert = #'OTPcertificate'{} -%% ConnectionStates = #connection_states{} +-spec client_certificate_verify(undefined | der_cert(), binary(), + tls_version(), key_algo(), private_key(), + {binary(), binary()}) -> + #certificate_verify{} | ignore. %% %% Description: Creates a certificate_verify message, called by the client. %%-------------------------------------------------------------------- @@ -298,10 +283,9 @@ client_certificate_verify(OwnCert, MasterSecret, Version, Algorithm, end. %%-------------------------------------------------------------------- -%% Function: certificate_verify(Signature, PublicKeyInfo) -> valid | #alert{} -%% -%% Signature = binary() -%% PublicKeyInfo = {Algorithm, PublicKey, PublicKeyParams} +-spec certificate_verify(binary(), public_key_info(), tls_version(), + binary(), key_algo(), + {binary(), binary()}) -> valid | #alert{}. %% %% Description: Checks that the certificate_verify message is valid. %%-------------------------------------------------------------------- @@ -320,13 +304,19 @@ certificate_verify(Signature, {_, PublicKey, _}, Version, end; certificate_verify(Signature, {_, PublicKey, PublicKeyParams}, Version, MasterSecret, dhe_dss = Algorithm, {_, Hashes0}) -> - Hashes = calc_certificate_verify(Version, MasterSecret, - Algorithm, Hashes0), - public_key:verify_signature(Hashes, sha, Signature, PublicKey, PublicKeyParams). + Hashes = calc_certificate_verify(Version, MasterSecret, + Algorithm, Hashes0), + case public_key:verify_signature(Hashes, none, Signature, PublicKey, PublicKeyParams) of + true -> + valid; + false -> + ?ALERT_REC(?FATAL, ?BAD_CERTIFICATE) + end. + %%-------------------------------------------------------------------- -%% Function: certificate_request(ConnectionStates, CertDbRef) -> -%% #certificate_request{} +-spec certificate_request(#connection_states{}, certdb_ref()) -> + #certificate_request{}. %% %% Description: Creates a certificate_request message, called by the server. %%-------------------------------------------------------------------- @@ -342,11 +332,12 @@ certificate_request(ConnectionStates, CertDbRef) -> }. %%-------------------------------------------------------------------- -%% Function: key_exchange(Role, Secret, Params) -> -%% #client_key_exchange{} | #server_key_exchange{} -%% -%% Secret - -%% Params - +-spec key_exchange(client | server, + {premaster_secret, binary(), public_key_info()} | + {dh, binary()} | + {dh, binary(), #'DHParameter'{}, key_algo(), + binary(), binary(), private_key()}) -> + #client_key_exchange{} | #server_key_exchange{}. %% %% Description: Creates a keyexchange message. %%-------------------------------------------------------------------- @@ -371,26 +362,20 @@ key_exchange(server, {dh, {<<?UINT32(Len), PublicKey:Len/binary>>, _}, YLen = byte_size(PublicKey), ServerDHParams = #server_dh_params{dh_p = PBin, dh_g = GBin, dh_y = PublicKey}, - Plain = - server_key_exchange_plain(KeyAlgo, <<ClientRandom/binary, + Hash = + server_key_exchange_hash(KeyAlgo, <<ClientRandom/binary, ServerRandom/binary, ?UINT16(PLen), PBin/binary, ?UINT16(GLen), GBin/binary, ?UINT16(YLen), PublicKey/binary>>), - Signed = digitally_signed(Plain, PrivateKey), + Signed = digitally_signed(Hash, PrivateKey), #server_key_exchange{params = ServerDHParams, signed_params = Signed}. %%-------------------------------------------------------------------- -%% Function: master_secret(Version, Session/PremasterSecret, -%% ConnectionStates, Role) -> -%% {MasterSecret, NewConnectionStates} | #alert{} -%% Version = #protocol_version{} -%% Session = #session{} (session contains master secret) -%% PremasterSecret = binary() -%% ConnectionStates = #connection_states{} -%% Role = client | server -%% +-spec master_secret(tls_version(), #session{} | binary(), #connection_states{}, + client | server) -> {binary(), #connection_states{}} | #alert{}. +%% %% Description: Sets or calculates the master secret and calculate keys, %% updating the pending connection states. The Mastersecret and the update %% connection states are returned or an alert if the calculation fails. @@ -427,9 +412,8 @@ master_secret(Version, PremasterSecret, ConnectionStates, Role) -> end. %%-------------------------------------------------------------------- -%% Function: finished(Version, Role, MacSecret, Hashes) -> #finished{} -%% -%% ConnectionStates = #connection_states{} +-spec finished(tls_version(), client | server, binary(), {binary(), binary()}) -> + #finished{}. %% %% Description: Creates a handshake finished message %%------------------------------------------------------------------- @@ -438,15 +422,8 @@ finished(Version, Role, MasterSecret, {Hashes, _}) -> % use the current hashes calc_finished(Version, Role, MasterSecret, Hashes)}. %%-------------------------------------------------------------------- -%% Function: verify_connection(Finished, Role, -%% MasterSecret, Hashes) -> verified | #alert{} -%% -%% Finished = #finished{} -%% Role = client | server - the role of the process that sent the finished -%% message. -%% MasterSecret = binary() -%% Hashes = binary() - {md5_hash, sha_hash} -%% +-spec verify_connection(tls_version(), #finished{}, client | server, binary(), + {binary(), binary()}) -> verified | #alert{}. %% %% Description: Checks the ssl handshake finished message to verify %% the connection. @@ -462,17 +439,18 @@ verify_connection(Version, #finished{verify_data = Data}, _E -> ?ALERT_REC(?FATAL, ?HANDSHAKE_FAILURE) end. - +%%-------------------------------------------------------------------- +-spec server_hello_done() -> #server_hello_done{}. +%% +%% Description: Creates a server hello done message. +%%-------------------------------------------------------------------- server_hello_done() -> #server_hello_done{}. %%-------------------------------------------------------------------- -%% Function: encode_handshake(HandshakeRec) -> BinHandshake -%% HandshakeRec = #client_hello | #server_hello{} | server_hello_done | -%% #certificate{} | #client_key_exchange{} | #finished{} | -%% #client_certify_request{} +-spec encode_handshake(tls_handshake(), tls_version(), key_algo()) -> binary(). %% -%% encode a handshake packet to binary +%% Description: Encode a handshake packet to binary %%-------------------------------------------------------------------- encode_handshake(Package, Version, KeyAlg) -> SigAlg = sig_alg(KeyAlg), @@ -481,12 +459,11 @@ encode_handshake(Package, Version, KeyAlg) -> [MsgType, ?uint24(Len), Bin]. %%-------------------------------------------------------------------- -%% Function: get_tls_handshake(Data, Buffer) -> Result -%% Result = {[#handshake{}], [Raw], NewBuffer} -%% Data = Buffer = NewBuffer = Raw = binary() +-spec get_tls_handshake(binary(), binary(), key_algo(), tls_version()) -> + {[tls_handshake()], [binary()], binary()}. %% %% Description: Given buffered and new data from ssl_record, collects -%% and returns it as a list of #handshake, also returns leftover +%% and returns it as a list of handshake messages, also returns leftover %% data. %%-------------------------------------------------------------------- get_tls_handshake(Data, <<>>, KeyAlg, Version) -> @@ -495,6 +472,9 @@ get_tls_handshake(Data, Buffer, KeyAlg, Version) -> get_tls_handshake_aux(list_to_binary([Buffer, Data]), KeyAlg, Version, []). +%%-------------------------------------------------------------------- +%%% Internal functions +%%-------------------------------------------------------------------- get_tls_handshake_aux(<<?BYTE(Type), ?UINT24(Length), Body:Length/binary,Rest/binary>>, KeyAlg, Version, Acc) -> @@ -504,9 +484,6 @@ get_tls_handshake_aux(<<?BYTE(Type), ?UINT24(Length), get_tls_handshake_aux(Data, _KeyAlg, _Version, Acc) -> {lists:reverse(Acc), Data}. -%%-------------------------------------------------------------------- -%%% Internal functions -%%-------------------------------------------------------------------- verify_bool(verify_peer) -> true; verify_bool(verify_none) -> @@ -1087,12 +1064,12 @@ certificate_authorities_from_db(CertDbRef, PrevKey, Acc) -> certificate_authorities_from_db(CertDbRef, Key, Acc) end. -digitally_signed(Hashes, #'RSAPrivateKey'{} = Key) -> - public_key:encrypt_private(Hashes, Key, +digitally_signed(Hash, #'RSAPrivateKey'{} = Key) -> + public_key:encrypt_private(Hash, Key, [{rsa_pad, rsa_pkcs1_padding}]); -digitally_signed(Plain, #'DSAPrivateKey'{} = Key) -> - public_key:sign(Plain, Key). - +digitally_signed(Hash, #'DSAPrivateKey'{} = Key) -> + public_key:sign(none, Hash, Key). + calc_master_secret({3,0}, PremasterSecret, ClientRandom, ServerRandom) -> ssl_ssl3:master_secret(PremasterSecret, ClientRandom, ServerRandom); @@ -1122,15 +1099,14 @@ calc_certificate_verify({3, N}, _, Algorithm, Hashes) when N == 1; N == 2 -> ssl_tls1:certificate_verify(Algorithm, Hashes). -server_key_exchange_plain(Algorithm, Value) when Algorithm == rsa; +server_key_exchange_hash(Algorithm, Value) when Algorithm == rsa; Algorithm == dhe_rsa -> MD5 = crypto:md5(Value), SHA = crypto:sha(Value), <<MD5/binary, SHA/binary>>; -server_key_exchange_plain(dhe_dss, Value) -> - %% Hash will be done by crypto. - Value. +server_key_exchange_hash(dhe_dss, Value) -> + crypto:sha(Value). sig_alg(dh_anon) -> ?SIGNATURE_ANONYMOUS; diff --git a/lib/ssl/src/ssl_internal.hrl b/lib/ssl/src/ssl_internal.hrl index fdc0c33750..ddace02dea 100644 --- a/lib/ssl/src/ssl_internal.hrl +++ b/lib/ssl/src/ssl_internal.hrl @@ -23,6 +23,8 @@ -ifndef(ssl_internal). -define(ssl_internal, true). +-include_lib("public_key/include/public_key.hrl"). + %% basic binary constructors -define(BOOLEAN(X), X:8/unsigned-big-integer). -define(BYTE(X), X:8/unsigned-big-integer). @@ -88,6 +90,28 @@ active = true }). +-type reason() :: term(). +-type reply() :: term(). +-type msg() :: term(). +-type from() :: term(). +-type host() :: string() | tuple(). +-type port_num() :: integer(). +-type session_id() :: binary(). +-type tls_version() :: {integer(), integer()}. +-type tls_atom_version() :: sslv3 | tlsv1. +-type cache_ref() :: term(). +-type certdb_ref() :: term(). +-type key_algo() :: rsa | dhe_rsa | dhe_dss. +-type enum_algo() :: integer(). +-type public_key() :: #'RSAPublicKey'{} | integer(). +-type public_key_params() :: #'Dss-Parms'{} | term(). +-type public_key_info() :: {enum_algo(), public_key(), public_key_params()}. +-type der_cert() :: binary(). +-type private_key() :: #'RSAPrivateKey'{} | #'DSAPrivateKey'{}. +-type issuer() :: tuple(). +-type serialnumber() :: integer(). +-type cert_key() :: {reference(), integer(), issuer()}. + -endif. % -ifdef(ssl_internal). diff --git a/lib/ssl/src/ssl_manager.erl b/lib/ssl/src/ssl_manager.erl index 19bdcfa1f5..af30f78dbf 100644 --- a/lib/ssl/src/ssl_manager.erl +++ b/lib/ssl/src/ssl_manager.erl @@ -24,8 +24,10 @@ -module(ssl_manager). -behaviour(gen_server). +-include("ssl_internal.hrl"). + %% Internal application API --export([start_link/0, start_link/1, +-export([start_link/1, connection_init/2, cache_pem_file/1, lookup_trusted_cert/3, issuer_candidate/1, client_session_id/3, server_session_id/3, register_session/2, register_session/3, invalidate_session/2, @@ -58,21 +60,25 @@ %% API %%==================================================================== %%-------------------------------------------------------------------- -%% Function: start_link() -> {ok,Pid} | ignore | {error,Error} +-spec start_link(list()) -> {ok, pid()} | ignore | {error, term()}. +%% %% Description: Starts the server %%-------------------------------------------------------------------- -start_link() -> - gen_server:start_link({local, ?MODULE}, ?MODULE, [], []). start_link(Opts) -> gen_server:start_link({local, ?MODULE}, ?MODULE, [Opts], []). %%-------------------------------------------------------------------- -%% Function: -%% Description: +-spec connection_init(string(), client | server) -> {ok, reference(), cache_ref()}. +%% +%% Description: Do necessary initializations for a new connection. %%-------------------------------------------------------------------- connection_init(TrustedcertsFile, Role) -> call({connection_init, TrustedcertsFile, Role}). - +%%-------------------------------------------------------------------- +-spec cache_pem_file(string()) -> {ok, term()}. +%% +%% Description: Cach a pem file and +%%-------------------------------------------------------------------- cache_pem_file(File) -> case ssl_certificate_db:lookup_cached_certs(File) of [{_,Content}] -> @@ -80,48 +86,51 @@ cache_pem_file(File) -> [] -> call({cache_pem, File}) end. - %%-------------------------------------------------------------------- -%% Function: -%% Description: +-spec lookup_trusted_cert(reference(), serialnumber(), issuer()) -> + {der_cert(), #'OTPCertificate'{}}. +%% +%% Description: Lookup the trusted cert with Key = {reference(), serialnumber(), issuer()}. %%-------------------------------------------------------------------- lookup_trusted_cert(Ref, SerialNumber, Issuer) -> ssl_certificate_db:lookup_trusted_cert(Ref, SerialNumber, Issuer). - %%-------------------------------------------------------------------- -%% Function: -%% Description: +-spec issuer_candidate(cert_key()) -> {cert_key(), der_cert()} | no_more_candidates. +%% +%% Description: Return next issuer candidate. %%-------------------------------------------------------------------- issuer_candidate(PrevCandidateKey) -> ssl_certificate_db:issuer_candidate(PrevCandidateKey). - %%-------------------------------------------------------------------- -%% Function: -%% Description: +-spec client_session_id(host(), port_num(), #ssl_options{}) -> session_id(). +%% +%% Description: Select a session id for the client. %%-------------------------------------------------------------------- client_session_id(Host, Port, SslOpts) -> call({client_session_id, Host, Port, SslOpts}). - + %%-------------------------------------------------------------------- -%% Function: -%% Description: +-spec server_session_id(host(), port_num(), #ssl_options{}) -> session_id(). +%% +%% Description: Select a session id for the server. %%-------------------------------------------------------------------- server_session_id(Port, SuggestedSessionId, SslOpts) -> call({server_session_id, Port, SuggestedSessionId, SslOpts}). %%-------------------------------------------------------------------- -%% Function: -%% Description: +-spec register_session(host(), port_num(), #session{}) -> ok. +%% +%% Description: Make the session available for reuse. %%-------------------------------------------------------------------- register_session(Host, Port, Session) -> cast({register_session, Host, Port, Session}). register_session(Port, Session) -> cast({register_session, Port, Session}). - %%-------------------------------------------------------------------- -%% Function: -%% Description: +-spec invalidate_session(host(), port_num(), #session{}) -> ok. +%% +%% Description: Make the session unavilable for reuse. %%-------------------------------------------------------------------- invalidate_session(Host, Port, Session) -> cast({invalidate_session, Host, Port, Session}). @@ -134,10 +143,9 @@ invalidate_session(Port, Session) -> %%==================================================================== %%-------------------------------------------------------------------- -%% Function: init(Args) -> {ok, State} | -%% {ok, State, Timeout} | -%% ignore | -%% {stop, Reason} +-spec init(list()) -> {ok, #state{}} | {ok, #state{}, timeout()} | + ignore | {stop, term()}. +%% %% Description: Initiates the server %%-------------------------------------------------------------------- init([Opts]) -> @@ -156,12 +164,13 @@ init([Opts]) -> session_validation_timer = Timer}}. %%-------------------------------------------------------------------- -%% Function: %% handle_call(Request, From, State) -> {reply, Reply, State} | -%% {reply, Reply, State, Timeout} | -%% {noreply, State} | -%% {noreply, State, Timeout} | -%% {stop, Reason, Reply, State} | -%% {stop, Reason, State} +-spec handle_call(msg(), from(), #state{}) -> {reply, reply(), #state{}} | + {reply, reply(), #state{}, timeout()} | + {noreply, #state{}} | + {noreply, #state{}, timeout()} | + {stop, reason(), reply(), #state{}} | + {stop, reason(), #state{}}. +%% %% Description: Handling call messages %%-------------------------------------------------------------------- handle_call({{connection_init, "", _Role}, Pid}, _From, @@ -207,9 +216,10 @@ handle_call({{cache_pem, File},Pid}, _, State = #state{certificate_db = Db}) -> {reply, {error, Reason}, State} end. %%-------------------------------------------------------------------- -%% Function: handle_cast(Msg, State) -> {noreply, State} | -%% {noreply, State, Timeout} | -%% {stop, Reason, State} +-spec handle_cast(msg(), #state{}) -> {noreply, #state{}} | + {noreply, #state{}, timeout()} | + {stop, reason(), #state{}}. +%% %% Description: Handling cast messages %%-------------------------------------------------------------------- handle_cast({register_session, Host, Port, Session}, @@ -243,9 +253,10 @@ handle_cast({invalidate_session, Port, #session{session_id = ID}}, {noreply, State}. %%-------------------------------------------------------------------- -%% Function: handle_info(Info, State) -> {noreply, State} | -%% {noreply, State, Timeout} | -%% {stop, Reason, State} +-spec handle_info(msg(), #state{}) -> {noreply, #state{}} | + {noreply, #state{}, timeout()} | + {stop, reason(), #state{}}. +%% %% Description: Handling all non call/cast messages %%-------------------------------------------------------------------- handle_info(validate_sessions, #state{session_cache_cb = CacheCb, @@ -278,7 +289,8 @@ handle_info(_Info, State) -> {noreply, State}. %%-------------------------------------------------------------------- -%% Function: terminate(Reason, State) -> void() +-spec terminate(reason(), #state{}) -> term(). +%% %% Description: This function is called by a gen_server when it is about to %% terminate. It should be the opposite of Module:init/1 and do any necessary %% cleaning up. When it returns, the gen_server terminates with Reason. @@ -294,7 +306,8 @@ terminate(_Reason, #state{certificate_db = Db, ok. %%-------------------------------------------------------------------- -%% Func: code_change(OldVsn, State, Extra) -> {ok, NewState} +-spec code_change(term(), #state{}, list()) -> {ok, #state{}}. +%% %% Description: Convert process state when code is changed %%-------------------------------------------------------------------- code_change(_OldVsn, State, _Extra) -> @@ -339,4 +352,3 @@ session_validation({{{Host, Port}, _}, Session}, LifeTime) -> session_validation({{Port, _}, Session}, LifeTime) -> validate_session(Port, Session, LifeTime), LifeTime. - diff --git a/lib/ssl/src/ssl_pem.erl b/lib/ssl/src/ssl_pem.erl deleted file mode 100644 index 0a1bf0f32a..0000000000 --- a/lib/ssl/src/ssl_pem.erl +++ /dev/null @@ -1,147 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% - -%% - --module(ssl_pem). - -%%% Purpose: Reading and writing of PEM type encoded files for SSL. - -%% NB write_file/2 is only preliminary. - -%% PEM encoded files have the following structure: -%% -%% <text> -%% -----BEGIN SOMETHING-----<CR><LF> -%% <Base64 encoding line><CR><LF> -%% <Base64 encoding line><CR><LF> -%% ... -%% -----END SOMETHING-----<CR><LF> -%% <text> -%% -%% A file can contain several BEGIN/END blocks. Text lines between -%% blocks are ignored. - --export([read_file/1, read_file/2, write_file/2]). - -%% Read a PEM file and return each decoding as a binary. - -read_file(File) -> - read_file(File, no_passwd). - -read_file(File, Passwd) -> - {ok, Fd} = file:open(File, [read]), - Result = decode_file(Fd, Passwd), - file:close(Fd), - Result. - -decode_file(Fd, Passwd) -> - decode_file(Fd, [], [], notag, [Passwd]). - -decode_file(Fd, _RLs, Ens, notag, Info) -> - case io:get_line(Fd, "") of - "-----BEGIN CERTIFICATE REQUEST-----" ++ _ -> - decode_file(Fd, [], Ens, cert_req, Info); - "-----BEGIN CERTIFICATE-----" ++ _ -> - decode_file(Fd, [], Ens, cert, Info); - "-----BEGIN RSA PRIVATE KEY-----" ++ _ -> - decode_file(Fd, [], Ens, rsa_private_key, Info); - eof -> - {ok, lists:reverse(Ens)}; - _ -> - decode_file(Fd, [], Ens, notag, Info) - end; -decode_file(Fd, RLs, Ens, Tag, Info0) -> - case io:get_line(Fd, "") of - "Proc-Type: 4,ENCRYPTED"++_ -> - Info = dek_info(Fd, Info0), - decode_file(Fd, RLs, Ens, Tag, Info); - "-----END" ++ _ -> % XXX sloppy - Cs = lists:flatten(lists:reverse(RLs)), - Bin = ssl_base64:join_decode(Cs), - case Info0 of - [Password, Cipher, SaltHex | Info1] -> - Decoded = decode_key(Bin, Password, Cipher, unhex(SaltHex)), - decode_file(Fd, [], [{Tag, Decoded}| Ens], notag, Info1); - _ -> - decode_file(Fd, [], [{Tag, Bin}| Ens], notag, Info0) - end; - eof -> - {ok, lists:reverse(Ens)}; - L -> - decode_file(Fd, [L|RLs], Ens, Tag, Info0) - end. - -dek_info(Fd, Info) -> - Line = io:get_line(Fd, ""), - [_, DekInfo0] = string:tokens(Line, ": "), - DekInfo1 = string:tokens(DekInfo0, ",\n"), - Info ++ DekInfo1. - -unhex(S) -> - unhex(S, []). - -unhex("", Acc) -> - lists:reverse(Acc); -unhex([D1, D2 | Rest], Acc) -> - unhex(Rest, [erlang:list_to_integer([D1, D2], 16) | Acc]). - -decode_key(Data, Password, "DES-CBC", Salt) -> - Key = password_to_key(Password, Salt, 8), - IV = Salt, - crypto:des_cbc_decrypt(Key, IV, Data); -decode_key(Data, Password, "DES-EDE3-CBC", Salt) -> - Key = password_to_key(Password, Salt, 24), - IV = Salt, - <<Key1:8/binary, Key2:8/binary, Key3:8/binary>> = Key, - crypto:des_ede3_cbc_decrypt(Key1, Key2, Key3, IV, Data). - -write_file(File, Ds) -> - file:write_file(File, encode_file(Ds)). - -encode_file(Ds) -> - [encode_file_1(D) || D <- Ds]. - -encode_file_1({cert, Bin}) -> - %% PKIX (X.509) - ["-----BEGIN CERTIFICATE-----\n", - ssl_base64:encode_split(Bin), - "-----END CERTIFICATE-----\n\n"]; -encode_file_1({cert_req, Bin}) -> - %% PKCS#10 - ["-----BEGIN CERTIFICATE REQUEST-----\n", - ssl_base64:encode_split(Bin), - "-----END CERTIFICATE REQUEST-----\n\n"]; -encode_file_1({rsa_private_key, Bin}) -> - %% PKCS#? - ["XXX Following key assumed not encrypted\n", - "-----BEGIN RSA PRIVATE KEY-----\n", - ssl_base64:encode_split(Bin), - "-----END RSA PRIVATE KEY-----\n\n"]. - -password_to_key(Data, Salt, KeyLen) -> - <<Key:KeyLen/binary, _/binary>> = - password_to_key(<<>>, Data, Salt, KeyLen, <<>>), - Key. - -password_to_key(_, _, _, Len, Acc) when Len =< 0 -> - Acc; -password_to_key(Prev, Data, Salt, Len, Acc) -> - M = crypto:md5([Prev, Data, Salt]), - password_to_key(M, Data, Salt, Len - byte_size(M), <<Acc/binary, M/binary>>). diff --git a/lib/ssl/src/ssl_pkix.erl b/lib/ssl/src/ssl_pkix.erl deleted file mode 100644 index 8f540f74ad..0000000000 --- a/lib/ssl/src/ssl_pkix.erl +++ /dev/null @@ -1,307 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% - -%%% Purpose : API module for decoding of certificates. - --module(ssl_pkix). - --include("ssl_pkix.hrl"). - --export([decode_cert_file/1, decode_cert_file/2, - decode_cert/1, decode_cert/2, encode_cert/1, encoded_tbs_cert/1, - signature_digest/1, decode_rsa_keyfile/2]). - -%% The public API is dprecated by public_key and -%% the internal application API is no longer used ssl. -%% So this file can be compleatly removed in R14. --deprecated({decode_cert_file, 1, next_major_release}). --deprecated({decode_cert_file, 2, next_major_release}). --deprecated({decode_cert, 1, next_major_release}). --deprecated({decode_cert, 2, next_major_release}). - -%%==================================================================== -%% API -%%==================================================================== - -%%-------------------------------------------------------------------- -%% Function: decode_cert_file(File, <Opts>) -> {ok, Cert} | {ok, [Cert]} -%% -%% File = string() -%% Opts = [Opt] -%% Opt = pem | ssl | pkix - ssl and pkix are mutual exclusive -%% Cert = term() -%% -%% Description: Decodes certificats found in file <File>. -%% If the options list is empty the certificate is -%% returned as a DER encoded binary, i.e. {ok, Bin} is returned, where -%% Bin> is the provided input. The options pkix and ssl imply that the -%% certificate is returned as a parsed ASN.1 structure in the form of -%% an Erlang term. The ssl option gives a more elaborate return -%% structure, with more explicit information. In particular object -%% identifiers are replaced by atoms. The option subject implies that -%% only the subject's distinguished name part of the certificate is -%% returned. It can only be used together with the option pkix or the -%% option ssl. -%%-------------------------------------------------------------------- -decode_cert_file(File) -> - decode_cert_file(File, []). - -decode_cert_file(File, Opts) -> - case lists:member(pem, Opts) of - true -> - {ok, List} = ssl_pem:read_file(File), - Certs = [Bin || {cert, Bin} <- List], - NewOpts = lists:delete(pem, Opts), - Fun = fun(Cert) -> - {ok, Decoded} = decode_cert(Cert, NewOpts), - Decoded - end, - case lists:map(Fun, Certs) of - [DecodedCert] -> - {ok, DecodedCert}; - DecodedCerts -> - {ok, DecodedCerts} - end; - false -> - {ok, Bin} = file:read_file(File), - decode_cert(Bin, Opts) - end. -%%-------------------------------------------------------------------- -%% Function: decode_cert(Bin, <Opts>) -> {ok, Cert} -%% Bin - binary() -%% Opts = [Opt] -%% Opt = ssl | pkix | subject - ssl and pkix are mutual exclusive -%% Cert = term() -%% -%% Description: If the options list is empty the certificate is -%% returned as a DER encoded binary, i.e. {ok, Bin} is returned, where -%% Bin> is the provided input. The options pkix and ssl imply that the -%% certificate is returned as a parsed ASN.1 structure in the form of -%% an Erlang term. The ssl option gives a more elaborate return -%% structure, with more explicit information. In particular object -%% identifiers are replaced by atoms. The option subject implies that -%% only the subject's distinguished name part of the certificate is -%% returned. It can only be used together with the option pkix or the -%% option ssl. -%%-------------------------------------------------------------------- -decode_cert(Bin) -> - decode_cert(Bin, []). - -decode_cert(Bin, []) when is_binary(Bin) -> - {ok, Bin}; -decode_cert(Bin, Opts) when is_binary(Bin) -> - - {ok, Cert} = 'OTP-PKIX':decode('Certificate', Bin), - - case {lists:member(ssl, Opts), lists:member(pkix, Opts)} of - {true, false} -> - cert_return(transform(Cert, ssl), Opts); - {false, true} -> - cert_return(transform(Cert, pkix), Opts); - _ -> - {error, eoptions} - end. - -encode_cert(#'Certificate'{} = Cert) -> - {ok, List} = 'OTP-PKIX':encode('Certificate', Cert), - list_to_binary(List). - -decode_rsa_keyfile(KeyFile, Password) -> - {ok, List} = ssl_pem:read_file(KeyFile, Password), - [PrivatKey] = [Bin || {rsa_private_key, Bin} <- List], - 'OTP-PKIX':decode('RSAPrivateKey', PrivatKey). - -%%==================================================================== -%% Application internal API -%%==================================================================== - -%%-------------------------------------------------------------------- -%% Function: encoded_tbs_cert(Cert) -> PKXCert -%% -%% Cert = binary() - Der encoded -%% PKXCert = binary() - Der encoded -%% -%% Description: Extracts the binary TBSCert from the binary Certificate. -%%-------------------------------------------------------------------- -encoded_tbs_cert(Cert) -> - {ok, PKIXCert} = - 'OTP-PKIX':decode_TBSCert_exclusive(Cert), - {'Certificate', - {'Certificate_tbsCertificate', EncodedTBSCert}, _, _} = PKIXCert, - EncodedTBSCert. - -%%-------------------------------------------------------------------- -%%% Internal functions -%%-------------------------------------------------------------------- - -cert_return(Cert, Opts) -> - case lists:member(subject, Opts) of - true -> - {ok, get_subj(Cert)}; - false -> - {ok, Cert} - end. - - -%% Transfrom from PKIX1-Explicit88 to SSL-PKIX. - -transform(#'Certificate'{signature = Signature, - signatureAlgorithm = SignatureAlgorithm, - tbsCertificate = TbsCertificate} = Cert, Type) -> - Cert#'Certificate'{tbsCertificate = transform(TbsCertificate, Type), - signatureAlgorithm = transform(SignatureAlgorithm, Type), - signature = transform(Signature, Type)}; - -%% -record('TBSCertificate',{ -%% version = asn1_DEFAULT, serialNumber, signature, issuer, validity, subject, -%% subjectPublicKeyInfo, issuerUniqueID = asn1_NOVALUE, -%% subjectUniqueID = asn1_NOVALUE, extensions = asn1_NOVALUE}). - -transform(#'TBSCertificate'{signature = Signature, issuer = Issuer, - subject = Subject, extensions = Extensions, - subjectPublicKeyInfo = SPKInfo} = TBSCert, Type) -> - TBSCert#'TBSCertificate'{signature = transform(Signature, Type), - issuer = transform(Issuer, Type), - subject = transform(Subject, Type), - subjectPublicKeyInfo = transform(SPKInfo, Type), - extensions = transform_extensions(Extensions, Type) - }; - -transform(#'AlgorithmIdentifier'{algorithm = Algorithm, - parameters = Params}, ssl) -> - SignAlgAny = - #'SignatureAlgorithm-Any'{algorithm = Algorithm, parameters = Params}, - {ok, AnyEnc} = 'OTP-PKIX':encode('SignatureAlgorithm-Any', SignAlgAny), - {ok, SignAlgCd} = 'OTP-PKIX':decode('SignatureAlgorithm', - list_to_binary(AnyEnc)), - NAlgo = ssl_pkix_oid:id2atom(SignAlgCd#'SignatureAlgorithm'.algorithm), - SignAlgCd#'SignatureAlgorithm'{algorithm = NAlgo}; - -transform({rdnSequence, Lss}, Type) when is_list(Lss) -> - {rdnSequence, [[transform(L, Type) || L <- Ls] || Ls <- Lss]}; -transform({rdnSequence, Lss}, _) -> - {rdnSequence, Lss}; - -transform(#'AttributeTypeAndValue'{} = ATAV, ssl) -> - {ok, ATAVEnc} = - 'OTP-PKIX':encode('AttributeTypeAndValue', ATAV), - {ok, ATAVDec} = 'OTP-PKIX':decode('SSLAttributeTypeAndValue', - list_to_binary(ATAVEnc)), - AttrType = ATAVDec#'SSLAttributeTypeAndValue'.type, - #'AttributeTypeAndValue'{type = ssl_pkix_oid:id2atom(AttrType), - value = - ATAVDec#'SSLAttributeTypeAndValue'.value}; - -transform(#'AttributeTypeAndValue'{} = Att, pkix) -> - Att; - -%% -record('SubjectPublicKeyInfo',{ -%% algorithm, subjectPublicKey}). -%% -%% -record('SubjectPublicKeyInfo_algorithm',{ -%% algo, parameters = asn1_NOVALUE}). -%% -%% -record('SubjectPublicKeyInfo-Any',{ -%% algorithm, subjectPublicKey}). -%% -%% -record('PublicKeyAlgorithm',{ -%% algorithm, parameters = asn1_NOVALUE}). - -transform(#'SubjectPublicKeyInfo'{subjectPublicKey = SubjectPublicKey, - algorithm = Algorithm}, ssl) -> - %% Transform from SubjectPublicKeyInfo (PKIX1Explicit88) - %% to SubjectPublicKeyInfo-Any (SSL-PKIX). - Algo = Algorithm#'AlgorithmIdentifier'.algorithm, - Parameters = Algorithm#'AlgorithmIdentifier'.parameters, - AlgorithmAny = #'PublicKeyAlgorithm'{algorithm = Algo, - parameters = Parameters}, - {0, Bin} = SubjectPublicKey, - SInfoAny = #'SSLSubjectPublicKeyInfo-Any'{algorithm = AlgorithmAny, - subjectPublicKey = Bin}, - - %% Encode according to SubjectPublicKeyInfo-Any, and decode according - %% to SubjectPublicKeyInfo. - {ok, AnyEnc} = - 'OTP-PKIX':encode('SSLSubjectPublicKeyInfo-Any', SInfoAny), - {ok, SInfoCd} = 'OTP-PKIX':decode('SSLSubjectPublicKeyInfo', - list_to_binary(AnyEnc)), - %% Replace object identifier by atom - AlgorithmCd = SInfoCd#'SSLSubjectPublicKeyInfo'.algorithm, - AlgoCd = AlgorithmCd#'SSLSubjectPublicKeyInfo_algorithm'.algo, - Params = AlgorithmCd#'SSLSubjectPublicKeyInfo_algorithm'.parameters, - Key = SInfoCd#'SSLSubjectPublicKeyInfo'.subjectPublicKey, - NAlgoCd = ssl_pkix_oid:id2atom(AlgoCd), - NAlgorithmCd = - #'SubjectPublicKeyInfo_algorithm'{algorithm = NAlgoCd, - parameters = Params}, - #'SubjectPublicKeyInfo'{algorithm = NAlgorithmCd, - subjectPublicKey = Key - }; -transform(#'SubjectPublicKeyInfo'{} = SInfo, pkix) -> - SInfo; - -transform(#'Extension'{extnID = ExtnID} = Ext, ssl) -> - NewExtID = ssl_pkix_oid:id2atom(ExtnID), - ExtAny = setelement(1, Ext, 'Extension-Any'), - {ok, AnyEnc} = 'OTP-PKIX':encode('Extension-Any', ExtAny), - {ok, ExtCd} = 'OTP-PKIX':decode('SSLExtension', list_to_binary(AnyEnc)), - - ExtValue = transform_extension_value(NewExtID, - ExtCd#'SSLExtension'.extnValue, - ssl), - #'Extension'{extnID = NewExtID, - critical = ExtCd#'SSLExtension'.critical, - extnValue = ExtValue}; - -transform(#'Extension'{extnID = ExtnID, extnValue = ExtnValue} = Ext, pkix) -> - NewExtID = ssl_pkix_oid:id2atom(ExtnID), - ExtValue = transform_extension_value(NewExtID, ExtnValue, pkix), - Ext#'Extension'{extnValue = ExtValue}; - -transform(#'AuthorityKeyIdentifier'{authorityCertIssuer = CertIssuer} = Ext, - Type) -> - Ext#'AuthorityKeyIdentifier'{authorityCertIssuer = - transform(CertIssuer, Type)}; - -transform([{directoryName, Value}], Type) -> - [{directoryName, transform(Value, Type)}]; - -transform(X, _) -> - X. - -transform_extension_value('ce-authorityKeyIdentifier', Value, Type) -> - transform(Value, Type); -transform_extension_value(_, Value, _) -> - Value. - -transform_extensions(Exts, Type) when is_list(Exts) -> - [transform(Ext, Type) || Ext <- Exts]; -transform_extensions(Exts, _) -> - Exts. - -get_subj(Cert) -> - (Cert#'Certificate'.tbsCertificate)#'TBSCertificate'.subject. - -signature_digest(BinSignature) -> - case (catch 'OTP-PKIX':decode('DigestInfo', BinSignature)) of - {ok, DigestInfo} -> - list_to_binary(DigestInfo#'DigestInfo'.digest); - _ -> - {error, decode_error} - end. diff --git a/lib/ssl/src/ssl_pkix.hrl b/lib/ssl/src/ssl_pkix.hrl deleted file mode 100644 index a8463369f6..0000000000 --- a/lib/ssl/src/ssl_pkix.hrl +++ /dev/null @@ -1,81 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% - -%% - --ifndef(ssl_pkix). --define(ssl_pkix, true). - --include("OTP-PKIX.hrl"). - -%% The following commented out records are currently defined in OTP-PKIX.hrl -%% and are considered a public interface through ssl_pkix.hrl. -%% NOTE do not include OTP-PKIX.hrl it is an generated file -%% and may change but the following records will still be -%% availanble from this file. - -% -record('Certificate', { -% tbsCertificate, -% signatureAlgorithm, -% signature}). - -% -record('TBSCertificate', { -% version = asn1_DEFAULT, -% serialNumber, -% signature, -% issuer, -% validity, -% subject, -% subjectPublicKeyInfo, -% issuerUniqueID = asn1_NOVALUE, -% subjectUniqueID = asn1_NOVALUE, -% extensions = asn1_NOVALUE}). - -% -record('AttributeTypeAndValue', { -% type, -% value}). - -% -record('SubjectPublicKeyInfo', { -% algorithm, -% subjectPublicKey}). - --record('SubjectPublicKeyInfo_algorithm', { - algorithm, - parameters = asn1_NOVALUE}). - -% -record('FieldID', { -% fieldType, -% parameters}). - -% -record('Characteristic-two', { -% m, -% basis, -% parameters}). - -% -record('ExtensionAttribute', { -% extensionAttributeType, -% extensionAttributeValue}). - -% -record('Extension', { -% extnID, -% critical = asn1_DEFAULT, -% extnValue}). - --endif. % -ifdef(ssl_pkix). - diff --git a/lib/ssl/src/ssl_record.erl b/lib/ssl/src/ssl_record.erl index 6b7cffaa7d..90615c22a1 100644 --- a/lib/ssl/src/ssl_record.erl +++ b/lib/ssl/src/ssl_record.erl @@ -66,10 +66,9 @@ %%==================================================================== %% Internal application API %%==================================================================== + %%-------------------------------------------------------------------- -%% Function: init_connection_states(Role) -> #connection_states{} -%% Role = client | server -%% Random = binary() +-spec init_connection_states(client | server) -> #connection_states{}. %% %% Description: Creates a connection_states record with appropriate %% values for the initial SSL connection setup. @@ -85,9 +84,8 @@ init_connection_states(Role) -> }. %%-------------------------------------------------------------------- -%% Function: current_connection_state(States, Type) -> #connection_state{} -%% States = #connection_states{} -%% Type = read | write +-spec current_connection_state(#connection_states{}, read | write) -> + #connection_state{}. %% %% Description: Returns the instance of the connection_state record %% that is currently defined as the current conection state. @@ -100,9 +98,8 @@ current_connection_state(#connection_states{current_write = Current}, Current. %%-------------------------------------------------------------------- -%% Function: pending_connection_state(States, Type) -> #connection_state{} -%% States = #connection_states{} -%% Type = read | write +-spec pending_connection_state(#connection_states{}, read | write) -> + #connection_state{}. %% %% Description: Returns the instance of the connection_state record %% that is currently defined as the pending conection state. @@ -115,14 +112,11 @@ pending_connection_state(#connection_states{pending_write = Pending}, Pending. %%-------------------------------------------------------------------- -%% Function: update_security_params(Params, States) -> -%% #connection_states{} -%% Params = #security_parameters{} -%% States = #connection_states{} +-spec update_security_params(#security_parameters{}, #security_parameters{}, + #connection_states{}) -> #connection_states{}. %% %% Description: Creates a new instance of the connection_states record -%% where the pending states gets its security parameters -%% updated to <Params>. +%% where the pending states gets its security parameters updated. %%-------------------------------------------------------------------- update_security_params(ReadParams, WriteParams, States = #connection_states{pending_read = Read, @@ -135,14 +129,10 @@ update_security_params(ReadParams, WriteParams, States = WriteParams} }. %%-------------------------------------------------------------------- -%% Function: set_mac_secret(ClientWriteMacSecret, -%% ServerWriteMacSecret, Role, States) -> -%% #connection_states{} -%% MacSecret = binary() -%% States = #connection_states{} -%% Role = server | client +-spec set_mac_secret(binary(), binary(), client | server, + #connection_states{}) -> #connection_states{}. %% -%% update the mac_secret field in pending connection states +%% Description: update the mac_secret field in pending connection states %%-------------------------------------------------------------------- set_mac_secret(ClientWriteMacSecret, ServerWriteMacSecret, client, States) -> set_mac_secret(ServerWriteMacSecret, ClientWriteMacSecret, States); @@ -159,12 +149,9 @@ set_mac_secret(ReadMacSecret, WriteMacSecret, %%-------------------------------------------------------------------- -%% Function: set_master_secret(MasterSecret, States) -> -%% #connection_states{} -%% MacSecret = -%% States = #connection_states{} +-spec set_master_secret(binary(), #connection_state{}) -> #connection_states{}. %% -%% Set master_secret in pending connection states +%% Description: Set master_secret in pending connection states %%-------------------------------------------------------------------- set_master_secret(MasterSecret, States = #connection_states{pending_read = Read, @@ -180,12 +167,9 @@ set_master_secret(MasterSecret, States#connection_states{pending_read = Read1, pending_write = Write1}. %%-------------------------------------------------------------------- -%% Function: set_renegotiation_flag(Flag, States) -> -%% #connection_states{} -%% Flag = boolean() -%% States = #connection_states{} +-spec set_renegotiation_flag(boolean(), #connection_states{}) -> #connection_states{}. %% -%% Set master_secret in pending connection states +%% Description: Set secure_renegotiation in pending connection states %%-------------------------------------------------------------------- set_renegotiation_flag(Flag, #connection_states{ current_read = CurrentRead0, @@ -203,13 +187,11 @@ set_renegotiation_flag(Flag, #connection_states{ pending_write = PendingWrite}. %%-------------------------------------------------------------------- -%% Function: set_client_verify_data(State, Data, States) -> -%% #connection_states{} -%% State = atom() -%% Data = binary() -%% States = #connection_states{} +-spec set_client_verify_data(current_read | current_write | current_both, + binary(), #connection_states{})-> + #connection_states{}. %% -%% Set verify data in connection states. +%% Description: Set verify data in connection states. %%-------------------------------------------------------------------- set_client_verify_data(current_read, Data, #connection_states{current_read = CurrentRead0, @@ -235,15 +217,12 @@ set_client_verify_data(current_both, Data, CurrentWrite = CurrentWrite0#connection_state{client_verify_data = Data}, ConnectionStates#connection_states{current_read = CurrentRead, current_write = CurrentWrite}. - %%-------------------------------------------------------------------- -%% Function: set_server_verify_data(State, Data, States) -> -%% #connection_states{} -%% State = atom() -%% Data = binary() -%% States = #connection_states{} +-spec set_server_verify_data(current_read | current_write | current_both, + binary(), #connection_states{})-> + #connection_states{}. %% -%% Set verify data in pending connection states. +%% Description: Set verify data in pending connection states. %%-------------------------------------------------------------------- set_server_verify_data(current_write, Data, #connection_states{pending_read = PendingRead0, @@ -273,10 +252,8 @@ set_server_verify_data(current_both, Data, current_write = CurrentWrite}. %%-------------------------------------------------------------------- -%% Function: activate_pending_connection_state(States, Type) -> -%% #connection_states{} -%% States = #connection_states{} -%% Type = read | write +-spec activate_pending_connection_state(#connection_states{}, read | write) -> + #connection_states{}. %% %% Description: Creates a new instance of the connection_states record %% where the pending state of <Type> has been activated. @@ -308,11 +285,9 @@ activate_pending_connection_state(States = }. %%-------------------------------------------------------------------- -%% Function: set_pending_cipher_state(States, ClientState, -%% ServerState, Role) -> -%% #connection_states{} -%% ClientState = ServerState = #cipher_state{} -%% States = #connection_states{} +-spec set_pending_cipher_state(#connection_states{}, #cipher_state{}, + #cipher_state{}, client | server) -> + #connection_states{}. %% %% Description: Set the cipher state in the specified pending connection state. %%-------------------------------------------------------------------- @@ -331,12 +306,10 @@ set_pending_cipher_state(#connection_states{pending_read = Read, pending_write = Write#connection_state{cipher_state = ClientState}}. %%-------------------------------------------------------------------- -%% Function: get_tls_record(Data, Buffer) -> Result -%% Result = {[#tls_compressed{}], NewBuffer} -%% Data = Buffer = NewBuffer = binary() -%% -%% Description: given old buffer and new data from TCP, packs up a records -%% and returns it as a list of #tls_compressed, also returns leftover +-spec get_tls_records(binary(), binary()) -> {[binary()], binary()}. +%% +%% Description: Given old buffer and new data from TCP, packs up a records +%% and returns it as a list of tls_compressed binaries also returns leftover %% data %%-------------------------------------------------------------------- get_tls_records(Data, <<>>) -> @@ -399,8 +372,7 @@ get_tls_records_aux(Data, Acc) -> {lists:reverse(Acc), Data}. %%-------------------------------------------------------------------- -%% Function: protocol_version(Version) -> #protocol_version{} -%% Version = atom() +-spec protocol_version(tls_atom_version()) -> tls_version(). %% %% Description: Creates a protocol version record from a version atom %% or vice versa. @@ -420,8 +392,7 @@ protocol_version({3, 1}) -> protocol_version({3, 0}) -> sslv3. %%-------------------------------------------------------------------- -%% Function: protocol_version(Version1, Version2) -> #protocol_version{} -%% Version1 = Version2 = #protocol_version{} +-spec lowest_protocol_version(tls_version(), tls_version()) -> tls_version(). %% %% Description: Lowes protocol version of two given versions %%-------------------------------------------------------------------- @@ -436,8 +407,7 @@ lowest_protocol_version(Version = {M,_}, lowest_protocol_version(_,Version) -> Version. %%-------------------------------------------------------------------- -%% Function: protocol_version(Versions) -> #protocol_version{} -%% Versions = [#protocol_version{}] +-spec highest_protocol_version([tls_version()]) -> tls_version(). %% %% Description: Highest protocol version present in a list %%-------------------------------------------------------------------- @@ -459,9 +429,8 @@ highest_protocol_version(_, [Version | Rest]) -> highest_protocol_version(Version, Rest). %%-------------------------------------------------------------------- -%% Function: supported_protocol_versions() -> Versions -%% Versions = [#protocol_version{}] -%% +-spec supported_protocol_versions() -> [tls_version()]. +%% %% Description: Protocol versions supported %%-------------------------------------------------------------------- supported_protocol_versions() -> @@ -487,8 +456,7 @@ supported_protocol_versions([_|_] = Vsns) -> Vsns. %%-------------------------------------------------------------------- -%% Function: is_acceptable_version(Version) -> true | false -%% Version = #protocol_version{} +-spec is_acceptable_version(tls_version()) -> boolean(). %% %% Description: ssl version 2 is not acceptable security risks are too big. %%-------------------------------------------------------------------- @@ -499,7 +467,7 @@ is_acceptable_version(_) -> false. %%-------------------------------------------------------------------- -%% Function: compressions() -> binary() +-spec compressions() -> binary(). %% %% Description: return a list of compressions supported (currently none) %%-------------------------------------------------------------------- @@ -507,8 +475,8 @@ compressions() -> [?byte(?NULL)]. %%-------------------------------------------------------------------- -%% Function: decode_cipher_text(CipherText, ConnectionStates0) -> -%% {Plain, ConnectionStates} +-spec decode_cipher_text(#ssl_tls{}, #connection_states{}) -> + {#ssl_tls{}, #connection_states{}}. %% %% Description: Decode cipher text %%-------------------------------------------------------------------- diff --git a/lib/ssl/src/ssl_session.erl b/lib/ssl/src/ssl_session.erl index bcb10daf69..e9755cb0e1 100644 --- a/lib/ssl/src/ssl_session.erl +++ b/lib/ssl/src/ssl_session.erl @@ -32,11 +32,10 @@ -define(GEN_UNIQUE_ID_MAX_TRIES, 10). +-type seconds() :: integer(). + %%-------------------------------------------------------------------- -%% Function: is_new(ClientSuggestedId, ServerDecidedId) -> true | false -%% -%% ClientSuggestedId = binary() -%% ServerDecidedId = binary() +-spec is_new(binary(), binary()) -> boolean(). %% %% Description: Checks if the session id decided by the server is a %% new or resumed sesion id. @@ -45,17 +44,11 @@ is_new(<<>>, _) -> true; is_new(SessionId, SessionId) -> false; -is_new(_, _) -> +is_new(_ClientSuggestion, _ServerDecision) -> true. %%-------------------------------------------------------------------- -%% Function: id(ClientInfo, Cache, CacheCb) -> SessionId -%% -%% ClientInfo = {HostIP, Port, SslOpts} -%% HostIP = ipadress() -%% Port = integer() -%% CacheCb = atom() -%% SessionId = binary() +-spec id({host(), port_num(), #ssl_options{}}, cache_ref(), atom()) -> binary(). %% %% Description: Should be called by the client side to get an id %% for the client hello message. @@ -69,14 +62,8 @@ id(ClientInfo, Cache, CacheCb) -> end. %%-------------------------------------------------------------------- -%% Function: id(Port, SuggestedSessionId, ReuseFun, CacheCb, -%% SecondLifeTime) -> SessionId -%% -%% Port = integer() -%% SuggestedSessionId = SessionId = binary() -%% ReuseFun = fun(SessionId, PeerCert, Compression, CipherSuite) -> -%% true | false -%% CacheCb = atom() +-spec id(port_num(), binary(), #ssl_options{}, cache_ref(), + atom(), seconds()) -> binary(). %% %% Description: Should be called by the server side to get an id %% for the server hello message. @@ -95,10 +82,7 @@ id(Port, SuggestedSessionId, #ssl_options{reuse_sessions = ReuseEnabled, new_id(Port, ?GEN_UNIQUE_ID_MAX_TRIES, Cache, CacheCb) end. %%-------------------------------------------------------------------- -%% Function: valid_session(Session, LifeTime) -> true | false -%% -%% Session = #session{} -%% LifeTime = integer() - seconds +-spec valid_session(#session{}, seconds()) -> boolean(). %% %% Description: Check that the session has not expired %%-------------------------------------------------------------------- diff --git a/lib/ssl/src/ssl_session_cache.erl b/lib/ssl/src/ssl_session_cache.erl index 1f2d1fc7d3..823bf7acfa 100644 --- a/lib/ssl/src/ssl_session_cache.erl +++ b/lib/ssl/src/ssl_session_cache.erl @@ -22,13 +22,16 @@ -behaviour(ssl_session_cache_api). +-include("ssl_handshake.hrl"). +-include("ssl_internal.hrl"). + -export([init/1, terminate/1, lookup/2, update/3, delete/2, foldl/3, select_session/2]). +-type key() :: {{host(), port_num()}, session_id()} | {port_num(), session_id()}. + %%-------------------------------------------------------------------- -%% Function: init() -> Cache -%% -%% Cache - Reference to the cash (opaque) +-spec init(list()) -> cache_ref(). %% Returns reference to the cache (opaque) %% %% Description: Return table reference. Called by ssl_manager process. %%-------------------------------------------------------------------- @@ -36,9 +39,7 @@ init(_) -> ets:new(cache_name(), [set, protected]). %%-------------------------------------------------------------------- -%% Function: terminate(Cache) -> -%% -%% Cache - as returned by create/0 +-spec terminate(cache_ref()) -> any(). %% %% %% Description: Handles cache table at termination of ssl manager. %%-------------------------------------------------------------------- @@ -46,9 +47,7 @@ terminate(Cache) -> ets:delete(Cache). %%-------------------------------------------------------------------- -%% Function: lookup(Cache, Key) -> Session | undefined -%% Cache - as returned by create/0 -%% Session = #session{} +-spec lookup(cache_ref(), key()) -> #session{} | undefined. %% %% Description: Looks up a cach entry. Should be callable from any %% process. @@ -62,9 +61,7 @@ lookup(Cache, Key) -> end. %%-------------------------------------------------------------------- -%% Function: update(Cache, Key, Session) -> _ -%% Cache - as returned by create/0 -%% Session = #session{} +-spec update(cache_ref(), key(), #session{}) -> any(). %% %% Description: Caches a new session or updates a already cached one. %% Will only be called from the ssl_manager process. @@ -73,11 +70,7 @@ update(Cache, Key, Session) -> ets:insert(Cache, {Key, Session}). %%-------------------------------------------------------------------- -%% Function: delete(Cache, HostIP, Port, Id) -> _ -%% Cache - as returned by create/0 -%% HostIP = Host = string() | ipadress() -%% Port = integer() -%% Id = +-spec delete(cache_ref(), key()) -> any(). %% %% Description: Delets a cache entry. %% Will only be called from the ssl_manager process. @@ -86,28 +79,19 @@ delete(Cache, Key) -> ets:delete(Cache, Key). %%-------------------------------------------------------------------- -%% Function: foldl(Fun, Acc0, Cache) -> Acc -%% -%% Fun - fun() -%% Acc0 - term() -%% Cache - cache_ref() -%% +-spec foldl(fun(), term(), cache_ref()) -> term(). %% %% Description: Calls Fun(Elem, AccIn) on successive elements of the %% cache, starting with AccIn == Acc0. Fun/2 must return a new %% accumulator which is passed to the next call. The function returns -%% the final value of the accumulator. Acc0 is returned if the cache is -%% empty. -%% Should be callable from any process +%% the final value of the accumulator. Acc0 is returned if the cache +%% is empty.Should be callable from any process %%-------------------------------------------------------------------- foldl(Fun, Acc0, Cache) -> ets:foldl(Fun, Acc0, Cache). %%-------------------------------------------------------------------- -%% Function: select_session(Cache, PartialKey) -> [Sessions] -%% -%% Cache - as returned by create/0 -%% PartialKey - opaque Key = {PartialKey, SessionId} +-spec select_session(cache_ref(), {host(), port_num()} | port_num()) -> [#session{}]. %% %% Description: Selects a session that could be reused. Should be callable %% from any process. @@ -119,6 +103,5 @@ select_session(Cache, PartialKey) -> %%-------------------------------------------------------------------- %%% Internal functions %%-------------------------------------------------------------------- - cache_name() -> ssl_otp_session_cache. diff --git a/lib/ssl/src/ssl_ssl3.erl b/lib/ssl/src/ssl_ssl3.erl index 400298a322..375adf263a 100644 --- a/lib/ssl/src/ssl_ssl3.erl +++ b/lib/ssl/src/ssl_ssl3.erl @@ -38,6 +38,8 @@ %% Internal application API %%==================================================================== +-spec master_secret(binary(), binary(), binary()) -> binary(). + master_secret(PremasterSecret, ClientRandom, ServerRandom) -> ?DBG_HEX(PremasterSecret), ?DBG_HEX(ClientRandom), @@ -57,6 +59,8 @@ master_secret(PremasterSecret, ClientRandom, ServerRandom) -> ?DBG_HEX(B), B. +-spec finished(client | server, binary(), {binary(), binary()}) -> binary(). + finished(Role, MasterSecret, {MD5Hash, SHAHash}) -> %% draft-ietf-tls-ssl-version3-00 - 5.6.9 Finished %% struct { @@ -75,6 +79,8 @@ finished(Role, MasterSecret, {MD5Hash, SHAHash}) -> SHA = handshake_hash(?SHA, MasterSecret, Sender, SHAHash), <<MD5/binary, SHA/binary>>. +-spec certificate_verify(key_algo(), binary(), {binary(), binary()}) -> binary(). + certificate_verify(Algorithm, MasterSecret, {MD5Hash, SHAHash}) when Algorithm == rsa; Algorithm == dhe_rsa -> %% md5_hash @@ -94,6 +100,8 @@ certificate_verify(dhe_dss, MasterSecret, {_, SHAHash}) -> %% SHA(handshake_messages + master_secret + pad_1)); handshake_hash(?SHA, MasterSecret, undefined, SHAHash). +-spec mac_hash(integer(), binary(), integer(), integer(), integer(), binary()) -> binary(). + mac_hash(Method, Mac_write_secret, Seq_num, Type, Length, Fragment) -> %% draft-ietf-tls-ssl-version3-00 - 5.2.3.1 %% hash(MAC_write_secret + pad_2 + @@ -113,6 +121,10 @@ mac_hash(Method, Mac_write_secret, Seq_num, Type, Length, Fragment) -> ?DBG_HEX(Mac), Mac. +-spec setup_keys(binary(), binary(), binary(), binary(), + integer(), integer(), binary()) -> {binary(), binary(), binary(), + binary(), binary(), binary()}. + setup_keys(MasterSecret, ServerRandom, ClientRandom, HS, KML, _EKML, IVS) -> KeyBlock = generate_keyblock(MasterSecret, ServerRandom, ClientRandom, 2*(HS+KML+IVS)), @@ -136,6 +148,8 @@ setup_keys(MasterSecret, ServerRandom, ClientRandom, HS, KML, _EKML, IVS) -> {ClientWriteMacSecret, ServerWriteMacSecret, ClientWriteKey, ServerWriteKey, ClientIV, ServerIV}. +-spec suites() -> [cipher_suite()]. + suites() -> [ ?TLS_DHE_RSA_WITH_AES_256_CBC_SHA, @@ -147,7 +161,7 @@ suites() -> ?TLS_DHE_RSA_WITH_AES_128_CBC_SHA, ?TLS_DHE_DSS_WITH_AES_128_CBC_SHA, ?TLS_RSA_WITH_AES_128_CBC_SHA, - ?TLS_RSA_WITH_IDEA_CBC_SHA, + %%?TLS_RSA_WITH_IDEA_CBC_SHA, ?TLS_RSA_WITH_RC4_128_SHA, ?TLS_RSA_WITH_RC4_128_MD5, ?TLS_RSA_WITH_DES_CBC_SHA diff --git a/lib/ssl/src/ssl_tls1.erl b/lib/ssl/src/ssl_tls1.erl index 70db632835..d1bc0730ba 100644 --- a/lib/ssl/src/ssl_tls1.erl +++ b/lib/ssl/src/ssl_tls1.erl @@ -36,6 +36,8 @@ %% Internal application API %%==================================================================== +-spec master_secret(binary(), binary(), binary()) -> binary(). + master_secret(PreMasterSecret, ClientRandom, ServerRandom) -> %% RFC 2246 & 4346 - 8.1 %% master_secret = PRF(pre_master_secret, %% "master secret", ClientHello.random + @@ -43,6 +45,8 @@ master_secret(PreMasterSecret, ClientRandom, ServerRandom) -> prf(PreMasterSecret, <<"master secret">>, [ClientRandom, ServerRandom], 48). +-spec finished(client | server, binary(), {binary(), binary()}) -> binary(). + finished(Role, MasterSecret, {MD5Hash, SHAHash}) -> %% RFC 2246 & 4346 - 7.4.9. Finished %% struct { @@ -56,6 +60,7 @@ finished(Role, MasterSecret, {MD5Hash, SHAHash}) -> SHA = hash_final(?SHA, SHAHash), prf(MasterSecret, finished_label(Role), [MD5, SHA], 12). +-spec certificate_verify(key_algo(), {binary(), binary()}) -> binary(). certificate_verify(Algorithm, {MD5Hash, SHAHash}) when Algorithm == rsa; Algorithm == dhe_rsa -> @@ -65,7 +70,11 @@ certificate_verify(Algorithm, {MD5Hash, SHAHash}) when Algorithm == rsa; certificate_verify(dhe_dss, {_, SHAHash}) -> hash_final(?SHA, SHAHash). - + +-spec setup_keys(binary(), binary(), binary(), integer(), + integer(), integer()) -> {binary(), binary(), binary(), + binary(), binary(), binary()}. + setup_keys(MasterSecret, ServerRandom, ClientRandom, HashSize, KeyMatLen, IVSize) -> %% RFC 2246 - 6.3. Key calculation @@ -112,6 +121,9 @@ setup_keys(MasterSecret, ServerRandom, ClientRandom, HashSize, %% {ClientWriteMacSecret, ServerWriteMacSecret, ClientWriteKey, %% ServerWriteKey, undefined, undefined}. +-spec mac_hash(integer(), binary(), integer(), integer(), tls_version(), + integer(), binary()) -> binary(). + mac_hash(Method, Mac_write_secret, Seq_num, Type, {Major, Minor}, Length, Fragment) -> %% RFC 2246 & 4346 - 6.2.3.1. @@ -132,6 +144,8 @@ mac_hash(Method, Mac_write_secret, Seq_num, Type, {Major, Minor}, ?DBG_HEX(Mac), Mac. +-spec suites() -> [cipher_suite()]. + suites() -> [ ?TLS_DHE_RSA_WITH_AES_256_CBC_SHA, diff --git a/lib/ssl/test/Makefile b/lib/ssl/test/Makefile index d35cafc47b..9e4aecac45 100644 --- a/lib/ssl/test/Makefile +++ b/lib/ssl/test/Makefile @@ -59,12 +59,10 @@ ERL_FILES = $(MODULES:%=%.erl) HRL_FILES = ssl_test_MACHINE.hrl HRL_FILES_SRC = \ - ssl_pkix.hrl \ ssl_alert.hrl \ ssl_handshake.hrl -HRL_FILES_INC = \ - OTP-PKIX.hrl +HRL_FILES_INC = HRL_FILES_NEEDED_IN_TEST = \ $(HRL_FILES_SRC:%=../src/%) \ diff --git a/lib/ssl/test/old_ssl_active_SUITE.erl b/lib/ssl/test/old_ssl_active_SUITE.erl index 010596f351..d1cec26827 100644 --- a/lib/ssl/test/old_ssl_active_SUITE.erl +++ b/lib/ssl/test/old_ssl_active_SUITE.erl @@ -87,6 +87,8 @@ config(Config) -> %% operating system, version of OTP, Erts, kernel and stdlib. %% Check if SSL exists. If this case fails, all other cases are skipped + crypto:start(), + application:start(public_key), case ssl:start() of ok -> ssl:stop(); {error, {already_started, _}} -> ssl:stop(); diff --git a/lib/ssl/test/old_ssl_active_once_SUITE.erl b/lib/ssl/test/old_ssl_active_once_SUITE.erl index 6224b17aa7..63eaa730e9 100644 --- a/lib/ssl/test/old_ssl_active_once_SUITE.erl +++ b/lib/ssl/test/old_ssl_active_once_SUITE.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2002-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2002-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -79,6 +79,8 @@ config(Config) -> io:format("Config: ~p~n", [Config]), %% Check if SSL exists. If this case fails, all other cases are skipped + crypto:start(), + application:start(public_key), case ssl:start() of ok -> ssl:stop(); {error, {already_started, _}} -> ssl:stop(); diff --git a/lib/ssl/test/old_ssl_dist_SUITE.erl b/lib/ssl/test/old_ssl_dist_SUITE.erl index 56209c3530..97090c1409 100644 --- a/lib/ssl/test/old_ssl_dist_SUITE.erl +++ b/lib/ssl/test/old_ssl_dist_SUITE.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2007-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2007-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -254,7 +254,8 @@ mk_node_cmdline(ListenPort, Name, Args) -> Prog ++ " " ++ Static ++ " " ++ NameSw ++ " " ++ Name ++ " " - ++ "-pa " ++ Pa ++ " " + ++ "-pa " ++ Pa ++ " " + ++ "-run application start crypto -run application start public_key " ++ "-run " ++ atom_to_list(?MODULE) ++ " cnct2tstsrvr " ++ host_name() ++ " " ++ integer_to_list(ListenPort) ++ " " @@ -524,23 +525,10 @@ add_ssl_opts_config(Config) -> KrnlDir = filename:join([LibDir, "kernel-" ++ KRNL_VSN]), {ok, _} = file:read_file_info(StdlDir), {ok, _} = file:read_file_info(KrnlDir), - SSL_VSN = case lists:keysearch(ssl, 1, Apps) of - {value, {ssl, _, VSN}} -> - VSN; - _ -> - application:start(ssl), - try - {value, - {ssl, - _, - VSN}} = lists:keysearch(ssl, - 1, - application:which_applications()), - VSN - after - application:stop(ssl) - end - end, + SSL_VSN = vsn(ssl), + VSN_CRYPTO = vsn(crypto), + VSN_PKEY = vsn(public_key), + SslDir = filename:join([LibDir, "ssl-" ++ SSL_VSN]), {ok, _} = file:read_file_info(SslDir), %% We are using an installed otp system, create the boot script. @@ -552,6 +540,8 @@ add_ssl_opts_config(Config) -> " {erts, \"~s\"},~n" " [{kernel, \"~s\"},~n" " {stdlib, \"~s\"},~n" + " {crypto, \"~s\"},~n" + " {public_key, \"~s\"},~n" " {ssl, \"~s\"}]}.~n", [case catch erlang:system_info(otp_release) of {'EXIT', _} -> "R11B"; @@ -560,6 +550,8 @@ add_ssl_opts_config(Config) -> erlang:system_info(version), KRNL_VSN, STDL_VSN, + VSN_CRYPTO, + VSN_PKEY, SSL_VSN]), ok = file:close(RelFile), ok = systools:make_script(Script, []), @@ -593,3 +585,17 @@ success(Config) -> {value, {comment, _} = Res} -> Res; _ -> ok end. + +vsn(App) -> + application:start(App), + try + {value, + {ssl, + _, + VSN}} = lists:keysearch(App, + 1, + application:which_applications()), + VSN + after + application:stop(ssl) + end. diff --git a/lib/ssl/test/old_ssl_misc_SUITE.erl b/lib/ssl/test/old_ssl_misc_SUITE.erl index 55d1b71025..2767123a12 100644 --- a/lib/ssl/test/old_ssl_misc_SUITE.erl +++ b/lib/ssl/test/old_ssl_misc_SUITE.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2003-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -61,6 +61,8 @@ config(Config) -> io:format("Config: ~p~n", [Config]), %% Check if SSL exists. If this case fails, all other cases are skipped + crypto:start(), + application:start(public_key), case ssl:start() of ok -> ssl:stop(); {error, {already_started, _}} -> ssl:stop(); diff --git a/lib/ssl/test/old_ssl_passive_SUITE.erl b/lib/ssl/test/old_ssl_passive_SUITE.erl index 4cb8c1f0cd..96a7938583 100644 --- a/lib/ssl/test/old_ssl_passive_SUITE.erl +++ b/lib/ssl/test/old_ssl_passive_SUITE.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 1999-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 1999-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -78,6 +78,8 @@ config(Config) -> io:format("Config: ~p~n", [Config]), %% Check if SSL exists. If this case fails, all other cases are skipped + crypto:start(), + application:start(public_key), case ssl:start() of ok -> ssl:stop(); {error, {already_started, _}} -> ssl:stop(); diff --git a/lib/ssl/test/old_ssl_peer_cert_SUITE.erl b/lib/ssl/test/old_ssl_peer_cert_SUITE.erl index f0b8db2607..e5b3975d41 100644 --- a/lib/ssl/test/old_ssl_peer_cert_SUITE.erl +++ b/lib/ssl/test/old_ssl_peer_cert_SUITE.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2003-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -62,6 +62,8 @@ config(Config) -> io:format("Config: ~p~n", [Config]), %% Check if SSL exists. If this case fails, all other cases are skipped + crypto:start(), + application:start(public_key), case ssl:start() of ok -> ssl:stop(); {error, {already_started, _}} -> ssl:stop(); diff --git a/lib/ssl/test/old_ssl_protocol_SUITE.erl b/lib/ssl/test/old_ssl_protocol_SUITE.erl index 7bde5d6749..efdbf45a3d 100644 --- a/lib/ssl/test/old_ssl_protocol_SUITE.erl +++ b/lib/ssl/test/old_ssl_protocol_SUITE.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2005-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2005-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -55,6 +55,8 @@ config(Config) -> io:format("Config: ~p~n", [Config]), %% Check if SSL exists. If this case fails, all other cases are skipped + crypto:start(), + application:start(public_key), case ssl:start() of ok -> ssl:stop(); {error, {already_started, _}} -> ssl:stop(); diff --git a/lib/ssl/test/old_ssl_verify_SUITE.erl b/lib/ssl/test/old_ssl_verify_SUITE.erl index 5db964526f..7a8cd1578a 100644 --- a/lib/ssl/test/old_ssl_verify_SUITE.erl +++ b/lib/ssl/test/old_ssl_verify_SUITE.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 1999-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 1999-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -60,6 +60,8 @@ config(Config) -> io:format("Config: ~p~n", [Config]), %% Check if SSL exists. If this case fails, all other cases are skipped + crypto:start(), + application:start(public_key), case ssl:start() of ok -> ssl:stop(); {error, {already_started, _}} -> ssl:stop(); diff --git a/lib/ssl/test/old_transport_accept_SUITE.erl b/lib/ssl/test/old_transport_accept_SUITE.erl index 4bb09cee19..71c1d9e181 100644 --- a/lib/ssl/test/old_transport_accept_SUITE.erl +++ b/lib/ssl/test/old_transport_accept_SUITE.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2007-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2007-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -224,12 +224,9 @@ tolerant_server_loop(Client, LSock, Msg, N) -> tolerant_server_loop(Client, LSock, Msg, N-1). app() -> - case application:get_application(ssl) of - undefined -> - application:start(ssl); - _ -> - ok - end. + crypto:start(), + application:start(public_key), + ssl:start(). start_node(Kind, Params) -> S = atom_to_list(?MODULE)++"_" ++ atom_to_list(Kind), diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index 0d9a912e30..8a1b90ed98 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -50,18 +50,21 @@ %% Note: This function is free to add any key/value pairs to the Config %% variable, but should NOT alter/remove any existing entries. %%-------------------------------------------------------------------- -init_per_suite(Config) -> +init_per_suite(Config0) -> + Dog = ssl_test_lib:timetrap(?TIMEOUT *2), crypto:start(), + application:start(public_key), ssl:start(), %% make rsa certs using oppenssl Result = - (catch make_certs:all(?config(data_dir, Config), - ?config(priv_dir, Config))), + (catch make_certs:all(?config(data_dir, Config0), + ?config(priv_dir, Config0))), test_server:format("Make certs ~p~n", [Result]), - NewConfig = ssl_test_lib:make_dsa_cert(Config), - ssl_test_lib:cert_options(NewConfig). + Config1 = ssl_test_lib:make_dsa_cert(Config0), + Config = ssl_test_lib:cert_options(Config1), + [{watchdog, Dog} | Config]. %%-------------------------------------------------------------------- %% Function: end_per_suite(Config) -> _ diff --git a/lib/ssl/test/ssl_packet_SUITE.erl b/lib/ssl/test/ssl_packet_SUITE.erl index a019e660e9..1b8754afe9 100644 --- a/lib/ssl/test/ssl_packet_SUITE.erl +++ b/lib/ssl/test/ssl_packet_SUITE.erl @@ -54,6 +54,7 @@ %%-------------------------------------------------------------------- init_per_suite(Config) -> crypto:start(), + application:start(public_key), ssl:start(), Result = (catch make_certs:all(?config(data_dir, Config), diff --git a/lib/ssl/test/ssl_payload_SUITE.erl b/lib/ssl/test/ssl_payload_SUITE.erl index a0aa92bdf2..d80df0bfbd 100644 --- a/lib/ssl/test/ssl_payload_SUITE.erl +++ b/lib/ssl/test/ssl_payload_SUITE.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2008-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2008-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -38,6 +38,7 @@ %%-------------------------------------------------------------------- init_per_suite(Config) -> crypto:start(), + application:start(public_key), ssl:start(), make_certs:all(?config(data_dir, Config), ?config(priv_dir, Config)), ssl_test_lib:cert_options(Config). diff --git a/lib/ssl/test/ssl_test_MACHINE.erl b/lib/ssl/test/ssl_test_MACHINE.erl index e75f7079ed..e0ffa15d80 100644 --- a/lib/ssl/test/ssl_test_MACHINE.erl +++ b/lib/ssl/test/ssl_test_MACHINE.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2003-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% @@ -60,10 +60,12 @@ many_conns_1() -> %% mk_ssl_cert_opts(_Config) -> Dir = filename:join([code:lib_dir(ssl), "examples", "certs", "etc"]), - COpts = [{cacertfile, filename:join([Dir, "client", "cacerts.pem"])}, + COpts = [{ssl_imp, old}, + {cacertfile, filename:join([Dir, "client", "cacerts.pem"])}, {certfile, filename:join([Dir, "client", "cert.pem"])}, {keyfile, filename:join([Dir, "client", "key.pem"])}], - SOpts = [{cacertfile, filename:join([Dir, "server", "cacerts.pem"])}, + SOpts = [{ssl_imp, old}, + {cacertfile, filename:join([Dir, "server", "cacerts.pem"])}, {certfile, filename:join([Dir, "server", "cert.pem"])}, {keyfile, filename:join([Dir, "server", "key.pem"])}], {ok, {COpts, SOpts}}. @@ -225,11 +227,13 @@ start_ssl(Nodes, Config) -> ok. do_start(Env) -> + application:start(crypto), + application:start(public_key), application:load(ssl), lists:foreach( fun({Par, Val}) -> application:set_env(ssl, Par, Val) end, Env), - application:start(ssl), - application:start(crypto). + application:start(ssl). + %% %% start_node(Name) -> {ok, Node} @@ -542,7 +546,7 @@ get_active(St) -> listen(St, LPort) -> case St#st.protomod of ssl -> - ssl:listen(LPort, St#st.sockopts ++ St#st.sslopts); + ssl:listen(LPort, [{ssl_imp, old} | St#st.sockopts ++ St#st.sslopts]); gen_tcp -> gen_tcp:listen(LPort, St#st.sockopts) end. @@ -584,7 +588,8 @@ connect(St, Host, Port) -> case St#st.protomod of ssl -> - case ssl:connect(Host, Port, St#st.sockopts ++ St#st.sslopts, + case ssl:connect(Host, Port, + [{ssl_imp, old} | St#st.sockopts ++ St#st.sslopts], St#st.timeout) of {ok, Sock} -> {ok, LPort} = ssl:sockname(Sock), diff --git a/lib/ssl/test/ssl_test_lib.erl b/lib/ssl/test/ssl_test_lib.erl index 40715dbf30..dd0818827a 100644 --- a/lib/ssl/test/ssl_test_lib.erl +++ b/lib/ssl/test/ssl_test_lib.erl @@ -325,6 +325,10 @@ make_dsa_cert(Config) -> [{server_dsa_opts, [{ssl_imp, new},{reuseaddr, true}, {cacertfile, ServerCaCertFile}, {certfile, ServerCertFile}, {keyfile, ServerKeyFile}]}, + {server_dsa_verify_opts, [{ssl_imp, new},{reuseaddr, true}, + {cacertfile, ServerCaCertFile}, + {certfile, ServerCertFile}, {keyfile, ServerKeyFile}, + {verify, verify_peer}]}, {client_dsa_opts, [{ssl_imp, new},{reuseaddr, true}, {cacertfile, ClientCaCertFile}, {certfile, ClientCertFile}, {keyfile, ClientKeyFile}]} diff --git a/lib/ssl/test/ssl_to_openssl_SUITE.erl b/lib/ssl/test/ssl_to_openssl_SUITE.erl index 4981ac0424..75cfce0052 100644 --- a/lib/ssl/test/ssl_to_openssl_SUITE.erl +++ b/lib/ssl/test/ssl_to_openssl_SUITE.erl @@ -25,8 +25,6 @@ -compile(export_all). -include("test_server.hrl"). --include("test_server_line.hrl"). --include("ssl_pkix.hrl"). -define(TIMEOUT, 120000). -define(SLEEP, 1000). @@ -45,19 +43,22 @@ %% Note: This function is free to add any key/value pairs to the Config %% variable, but should NOT alter/remove any existing entries. %%-------------------------------------------------------------------- -init_per_suite(Config) -> +init_per_suite(Config0) -> + Dog = ssl_test_lib:timetrap(?TIMEOUT *2), case os:find_executable("openssl") of false -> {skip, "Openssl not found"}; _ -> crypto:start(), + application:start(public_key), ssl:start(), Result = - (catch make_certs:all(?config(data_dir, Config), - ?config(priv_dir, Config))), + (catch make_certs:all(?config(data_dir, Config0), + ?config(priv_dir, Config0))), test_server:format("Make certs ~p~n", [Result]), - NewConfig = ssl_test_lib:make_dsa_cert(Config), - ssl_test_lib:cert_options(NewConfig) + Config1 = ssl_test_lib:make_dsa_cert(Config0), + Config = ssl_test_lib:cert_options(Config1), + [{watchdog, Dog} | Config] end. %%-------------------------------------------------------------------- @@ -143,9 +144,10 @@ all(doc) -> all(suite) -> [erlang_client_openssl_server, erlang_server_openssl_client, - %% Comment out when new crypto sign functions is available - %%erlang_client_openssl_server_dsa_cert, - %%erlang_server_openssl_client_dsa_cert, + tls1_erlang_client_openssl_server_dsa_cert, + tls1_erlang_server_openssl_client_dsa_cert, + ssl3_erlang_client_openssl_server_dsa_cert, + ssl3_erlang_server_openssl_client_dsa_cert, erlang_server_openssl_client_reuse_session, erlang_client_openssl_server_renegotiate, erlang_client_openssl_server_no_wrap_sequence_number, @@ -252,11 +254,11 @@ erlang_server_openssl_client(Config) when is_list(Config) -> %%-------------------------------------------------------------------- -erlang_client_openssl_server_dsa_cert(doc) -> +tls1_erlang_client_openssl_server_dsa_cert(doc) -> ["Test erlang server with openssl client"]; -erlang_client_openssl_server_dsa_cert(suite) -> +tls1_erlang_client_openssl_server_dsa_cert(suite) -> []; -erlang_client_openssl_server_dsa_cert(Config) when is_list(Config) -> +tls1_erlang_client_openssl_server_dsa_cert(Config) when is_list(Config) -> process_flag(trap_exit, true), ClientOpts = ?config(client_dsa_opts, Config), ServerOpts = ?config(server_dsa_opts, Config), @@ -300,14 +302,14 @@ erlang_client_openssl_server_dsa_cert(Config) when is_list(Config) -> %%-------------------------------------------------------------------- -erlang_server_openssl_client_dsa_cert(doc) -> +tls1_erlang_server_openssl_client_dsa_cert(doc) -> ["Test erlang server with openssl client"]; -erlang_server_openssl_client_dsa_cert(suite) -> +tls1_erlang_server_openssl_client_dsa_cert(suite) -> []; -erlang_server_openssl_client_dsa_cert(Config) when is_list(Config) -> +tls1_erlang_server_openssl_client_dsa_cert(Config) when is_list(Config) -> process_flag(trap_exit, true), ClientOpts = ?config(client_dsa_opts, Config), - ServerOpts = ?config(server_dsa_opts, Config), + ServerOpts = ?config(server_dsa_verify_opts, Config), {_, ServerNode, _} = ssl_test_lib:run_where(Config), @@ -338,6 +340,97 @@ erlang_server_openssl_client_dsa_cert(Config) when is_list(Config) -> close_port(OpenSslPort), process_flag(trap_exit, false), ok. + +%%-------------------------------------------------------------------- + +ssl3_erlang_client_openssl_server_dsa_cert(doc) -> + ["Test erlang server with openssl client"]; +ssl3_erlang_client_openssl_server_dsa_cert(suite) -> + []; +ssl3_erlang_client_openssl_server_dsa_cert(Config) when is_list(Config) -> + process_flag(trap_exit, true), + ClientOpts = ?config(client_dsa_opts, Config), + ServerOpts = ?config(server_dsa_opts, Config), + + {ClientNode, _, Hostname} = ssl_test_lib:run_where(Config), + + Data = "From openssl to erlang", + + Port = ssl_test_lib:inet_port(node()), + CaCertFile = proplists:get_value(cacertfile, ServerOpts), + CertFile = proplists:get_value(certfile, ServerOpts), + KeyFile = proplists:get_value(keyfile, ServerOpts), + + Cmd = "openssl s_server -accept " ++ integer_to_list(Port) ++ + " -cert " ++ CertFile ++ " -CAfile " ++ CaCertFile + ++ " -key " ++ KeyFile ++ " -Verify 2 -ssl3 -msg", + + test_server:format("openssl cmd: ~p~n", [Cmd]), + + OpensslPort = open_port({spawn, Cmd}, [stderr_to_stdout]), + + wait_for_openssl_server(), + + Client = ssl_test_lib:start_client([{node, ClientNode}, {port, Port}, + {host, Hostname}, + {from, self()}, + {mfa, {?MODULE, + erlang_ssl_receive, [Data]}}, + {options, ClientOpts}]), + + port_command(OpensslPort, Data), + + ssl_test_lib:check_result(Client, ok), + + %% Clean close down! Server needs to be closed first !! + close_port(OpensslPort), + + ssl_test_lib:close(Client), + process_flag(trap_exit, false), + ok. + +%%-------------------------------------------------------------------- + +ssl3_erlang_server_openssl_client_dsa_cert(doc) -> + ["Test erlang server with openssl client"]; +ssl3_erlang_server_openssl_client_dsa_cert(suite) -> + []; +ssl3_erlang_server_openssl_client_dsa_cert(Config) when is_list(Config) -> + process_flag(trap_exit, true), + ClientOpts = ?config(client_dsa_opts, Config), + ServerOpts = ?config(server_dsa_verify_opts, Config), + + {_, ServerNode, _} = ssl_test_lib:run_where(Config), + + Data = "From openssl to erlang", + CaCertFile = proplists:get_value(cacertfile, ClientOpts), + CertFile = proplists:get_value(certfile, ClientOpts), + KeyFile = proplists:get_value(keyfile, ClientOpts), + + Server = ssl_test_lib:start_server([{node, ServerNode}, {port, 0}, + {from, self()}, + {mfa, {?MODULE, erlang_ssl_receive, [Data]}}, + {options, ServerOpts}]), + Port = ssl_test_lib:inet_port(Server), + + Cmd = "openssl s_client -port " ++ integer_to_list(Port) ++ + " -host localhost " ++ " -cert " ++ CertFile ++ " -CAfile " ++ CaCertFile + ++ " -key " ++ KeyFile ++ " -ssl3 -msg", + + test_server:format("openssl cmd: ~p~n", [Cmd]), + + OpenSslPort = open_port({spawn, Cmd}, [stderr_to_stdout]), + port_command(OpenSslPort, Data), + + ssl_test_lib:check_result(Server, ok), + + ssl_test_lib:close(Server), + + close_port(OpenSslPort), + process_flag(trap_exit, false), + ok. + + %%-------------------------------------------------------------------- erlang_server_openssl_client_reuse_session(doc) -> @@ -1314,7 +1407,9 @@ wait_for_openssl_server() -> check_sane_openssl_renegotaite(Config) -> case os:cmd("openssl version") of - "OpenSSL 0.9.8l" ++ _ -> + "OpenSSL 0.9.8" ++ _ -> + {skip, "Known renegotiation bug in OppenSSL"}; + "OpenSSL 0.9.7" ++ _ -> {skip, "Known renegotiation bug in OppenSSL"}; _ -> Config diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk index 813ce91e32..254ee8b986 100644 --- a/lib/ssl/vsn.mk +++ b/lib/ssl/vsn.mk @@ -17,13 +17,18 @@ # %CopyrightEnd% # -SSL_VSN = 3.11.1 +SSL_VSN = 4.0.1 -TICKETS = OTP-8679 \ - OTP-7047 \ - OTP-7049 \ - OTP-8568 \ - OTP-8588 +TICKETS = OTP-8721 + +#TICKETS_4.0 = OTP-8587\ +# OTP-8695 + +#TICKETS_3.11.1 = OTP-8679 \ +# OTP-7047 \ +# OTP-7049 \ +# OTP-8568 \ +# OTP-8588 #TICKETS_3.11 = OTP-8517 \ # OTP-7046 \ diff --git a/lib/stdlib/doc/src/beam_lib.xml b/lib/stdlib/doc/src/beam_lib.xml index 27308e02f3..adc411e272 100644 --- a/lib/stdlib/doc/src/beam_lib.xml +++ b/lib/stdlib/doc/src/beam_lib.xml @@ -341,6 +341,7 @@ chunkref() = chunkname() | chunkid()</code> <v>Beam1 = Beam2 = beam()</v> <v>Reason = {modules_different, Module1, Module2}</v> <v> | {chunks_different, ChunkId}</v> + <v> | different_chunks</v> <v> | Reason1 -- see info/1</v> <v> Module1 = Module2 = atom()</v> <v> ChunkId = chunkid()</v> diff --git a/lib/stdlib/doc/src/filename.xml b/lib/stdlib/doc/src/filename.xml index 0cf82fa48b..fe6c6f898e 100644 --- a/lib/stdlib/doc/src/filename.xml +++ b/lib/stdlib/doc/src/filename.xml @@ -49,7 +49,7 @@ <title>DATA TYPES</title> <code type="none"> name() = string() | atom() | DeepList - DeepList = [char() | atom() | DeepList]</code> +DeepList = [char() | atom() | DeepList]</code> </section> <funcs> <func> diff --git a/lib/stdlib/doc/src/lists.xml b/lib/stdlib/doc/src/lists.xml index a273a2301f..b3ad7aaf46 100644 --- a/lib/stdlib/doc/src/lists.xml +++ b/lib/stdlib/doc/src/lists.xml @@ -48,7 +48,7 @@ <item><p>if x <c>F</c> y and y <c>F</c> x then x = y (<c>F</c> is antisymmetric);</p> </item> - <item><p>if x <c>F</c> y and and y <c>F</c> z then x <c>F</c> z + <item><p>if x <c>F</c> y and y <c>F</c> z then x <c>F</c> z (<c>F</c> is transitive);</p> </item> <item><p>x <c>F</c> y or y <c>F</c> x (<c>F</c> is total).</p> diff --git a/lib/stdlib/doc/src/notes.xml b/lib/stdlib/doc/src/notes.xml index 28a3719d38..23d1e8b7de 100644 --- a/lib/stdlib/doc/src/notes.xml +++ b/lib/stdlib/doc/src/notes.xml @@ -30,6 +30,283 @@ </header> <p>This document describes the changes made to the STDLIB application.</p> +<section><title>STDLIB 1.17</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>The Erlang code preprocessor (<c>epp</c>) sent extra + messages on the form <c>{eof,Location}</c> to the client + when parsing the <c>file</c> attribute. This bug, + introduced in R11B, has been fixed.</p> + <p> + Own Id: OTP-8470</p> + </item> + <item> + <p>The abstract type 'fun' could not be printed by the + Erlang pretty printer (<c>erl_pp</c>). This bug has been + fixed.</p> + <p> + Own Id: OTP-8473</p> + </item> + <item> + <p>The function <c>erl_scan:reserved_word/1</c> no longer + returns <c>true</c> when given the word <c>spec</c>. This + bug was introduced in STDLIB-1.15.3 (R12B-3).</p> + <p> + Own Id: OTP-8567</p> + </item> + <item> + <p>The documentation of <c>lists:keysort/2</c> states + that the sort is stable.</p> + <p> + Own Id: OTP-8628 Aux Id: seq11576 </p> + </item> + <item> + <p> + The shell's line editing has been improved to more + resemble the behaviour of readline and other shells. + (Thanks to Dave Peticolas)</p> + <p> + Own Id: OTP-8635</p> + </item> + <item> + <p>The Erlang code preprocessor (<c>epp</c>) did not + correctly handle premature end-of-input when defining + macros. This bug, introduced in STDLIB 1.16, has been + fixed.</p> + <p> + Own Id: OTP-8665 Aux Id: OTP-7810 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The module binary from EEP31 (and EEP9) is implemented.</p> + <p> + Own Id: OTP-8217</p> + </item> + <item> + <p>The erlang pretty printer (<c>erl_pp</c>) no longer + quotes atoms in types.</p> + <p> + Own Id: OTP-8501</p> + </item> + <item> + <p>The Erlang code preprocessor (<c>epp</c>) now + considers records with no fields as typed.</p> + <p> + Own Id: OTP-8503</p> + </item> + <item> + <p> + Added function <c>zip:foldl/3</c> to iterate over zip + archives.</p> + <p> + Added functions to create and extract escripts. See + <c>escript:create/2</c> and <c>escript:extract/2</c>.</p> + <p> + The undocumented function <c>escript:foldl/3</c> has been + removed. The same functionality can be achieved with the + more flexible functions <c>escript:extract/2</c> and + <c>zip:foldl/3</c>.</p> + <p> + Record fields has been annotated with type info. Source + files as been adapted to fit within 80 chars and trailing + whitespace has been removed.</p> + <p> + Own Id: OTP-8521</p> + </item> + <item> + <p>The Erlang parser no longer duplicates the singleton + type <c>undefined</c> in the type of record fields + without initial value.</p> + <p> + Own Id: OTP-8522</p> + </item> + <item> + <p>A regular expression with many levels of parenthesis + could cause a buffer overflow. That has been corrected. + (Thanks to Michael Santos.)</p> + <p> + Own Id: OTP-8539</p> + </item> + <item> + <p>When defining macros the closing right parenthesis + before the dot is now mandatory.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8562</p> + </item> + <item> + <p> + Some properties of a compiled re pattern are defined to + allow for guard tests.</p> + <p> + Own Id: OTP-8577</p> + </item> + <item> + <p>Local and imported functions now override the + auto-imported BIFs when the names clash. The pre R14 + behaviour was that auto-imported BIFs would override + local functions. To avoid that old programs change + behaviour, the following will generate an error:</p> + <list><item><p>Doing a call without explicit module name + to a local function having a name clashing with the name + of an auto-imported BIF that was present (and + auto-imported) before OTP R14A</p></item> + <item><p>Explicitly importing a function having a name + clashing with the name of an autoimported BIF that was + present (and autoimported) before OTP R14A</p></item> + <item><p>Using any form of the old compiler directive + <c>nowarn_bif_clash</c></p></item> </list> <p>If the BIF + was added or auto-imported in OTP R14A or later, + overriding it with an import or a local function will + only result in a warning,</p> <p>To resolve clashes, you + can either use the explicit module name <c>erlang</c> to + call the BIF, or you can remove the auto-import of that + specific BIF by using the new compiler directive + <c>-compile({no_auto_import,[F/A]}).</c>, which makes all + calls to the local or imported function without explicit + module name pass without warnings or errors.</p> <p>The + change makes it possible to add auto-imported BIFs + without breaking or silently changing old code in the + future. However some current code ingeniously utilizing + the old behaviour or the <c>nowarn_bif_clash</c> compiler + directive, might need changing to be accepted by the + compiler.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8579</p> + </item> + <item> + <p>The undocumented, unsupport, and deprecated function + <c>lists:flat_length/1</c> has been removed.</p> + <p> + Own Id: OTP-8584</p> + </item> + <item> + <p> + A bug in re that could cause certain regular expression + matches never to terminate is corrected. (Thanks to + Michael Santos and Gordon Guthrie.)</p> + <p> + Own Id: OTP-8589</p> + </item> + <item> + <p>Nested records can now be accessed without + parenthesis. See the Reference Manual for examples. + (Thanks to YAMASHINA Hio and Tuncer Ayaz.)</p> + <p> + Own Id: OTP-8597</p> + </item> + <item> + <p><c>receive</c> statements that can only read out a + newly created reference are now specially optimized so + that it will execute in constant time regardless of the + number of messages in the receive queue for the process. + That optimization will benefit calls to + <c>gen_server:call()</c>. (See <c>gen:do_call/4</c> for + an example of a receive statement that will be + optimized.)</p> + <p> + Own Id: OTP-8623</p> + </item> + <item> + <p>The beam_lib:cmp/2 function now compares BEAM files in + stricter way. The BEAM files will be considered different + if there are any changes except in the compilation + information ("CInf") chunk. beam_lib:cmp/2 used to ignore + differences in the debug information (significant for + Dialyzer) and other chunks that did not directly change + the run-time behavior.</p> + <p> + Own Id: OTP-8625</p> + </item> + <item> + <p> + When a gen_server, gen_fsm process, or gen_event + terminates abnormally, sometimes the text representation + of the process state can occupy many lines of the error + log, depending on the definition of the state term. A + mechanism to trim out parts of the state from the log has + been added (using a format_status/2 callback). See the + documentation.</p> + <p> + Own Id: OTP-8630</p> + </item> + <item> + <p> + Calling <c>sys:get_status()</c> for processes that have + globally registered names that were not atoms would cause + a crash. Corrected. (Thanks to Steve Vinoski.)</p> + <p> + Own Id: OTP-8656</p> + </item> + <item> + <p>The Erlang scanner has been augmented with two new + tokens: <c>..</c> and <c>...</c>.</p> + <p> + Own Id: OTP-8657</p> + </item> + <item> + <p>Expressions evaluating to integers can now be used in + types and function specifications where hitherto only + integers were allowed ("Erlang_Integer").</p> + <p> + Own Id: OTP-8664</p> + </item> + <item> + <p>The compiler optimizes record operations better.</p> + <p> + Own Id: OTP-8668</p> + </item> + <item> + <p> + The recently added BIFs erlang:min/2, erlang:max/2 and + erlang:port_command/3 are now auto-imported (as they were + originally intended to be). Due to the recent compiler + change (OTP-8579), the only impact on old code defining + it's own min/2, max/2 or port_command/3 functions will be + a warning, the local functions will still be used. The + warning can be removed by using + -compile({no_auto_import,[min/2,max/2,port_command/3]}). + in the source file.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8669 Aux Id: OTP-8579 </p> + </item> + <item> + <p> + Now, binary_to_term/2 is auto-imported. This will cause a + compile warning if and only if a module has got a local + function with that name.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8671</p> + </item> + <item> + <p> + The predefined builtin type tid() has been removed. + Instead, ets:tid() should be used.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8687</p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 1.16.5</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/stdlib/doc/src/sofs.xml b/lib/stdlib/doc/src/sofs.xml index 8c8ae51262..729df1e678 100644 --- a/lib/stdlib/doc/src/sofs.xml +++ b/lib/stdlib/doc/src/sofs.xml @@ -210,7 +210,7 @@ X[i] to Y[i] and S a subset of X[1] × ... × X[n]. The <marker id="multiple_relative_product"></marker><em>multiple - relative product</em> of TR and and S is defined to be the + relative product</em> of TR and S is defined to be the set {z : z = ((x[1], ..., x[n]), (y[1],...,y[n])) for some (x[1], ..., x[n]) in S and for some (x[i], y[i]) in R[i], diff --git a/lib/stdlib/doc/src/timer.xml b/lib/stdlib/doc/src/timer.xml index 0b6807dd6c..1b34e71490 100644 --- a/lib/stdlib/doc/src/timer.xml +++ b/lib/stdlib/doc/src/timer.xml @@ -202,18 +202,33 @@ </func> <func> <name>tc(Module, Function, Arguments) -> {Time, Value}</name> - <fsummary>Measure the real time it takes to evaluate <c>apply(Module, Function, Arguments)</c></fsummary> + <name>tc(Fun, Arguments) -> {Time, Value}</name> + <fsummary>Measure the real time it takes to evaluate <c>apply(Module, + Function, Arguments)</c> or <c>apply(Fun, Arguments)</c></fsummary> <type> <v>Module = Function = atom()</v> + <v>Fun = fun()</v> <v>Arguments = [term()]</v> <v>Time = integer() in microseconds</v> <v>Value = term()</v> </type> <desc> - <p>Evaluates <c>apply(Module, Function, Arguments)</c> and measures - the elapsed real time. Returns <c>{Time, Value}</c>, where - <c>Time</c> is the elapsed real time in <em>microseconds</em>, - and <c>Value</c> is what is returned from the apply.</p> + <p></p> + <taglist> + <tag><c>tc/3</c></tag> + <item> + <p>Evaluates <c>apply(Module, Function, Arguments)</c> and measures + the elapsed real time as reported by <c>now/0</c>. + Returns <c>{Time, Value}</c>, where + <c>Time</c> is the elapsed real time in <em>microseconds</em>, + and <c>Value</c> is what is returned from the apply.</p> + </item> + <tag><c>tc/2</c></tag> + <item> + <p>Evaluates <c>apply(Fun, Arguments)</c>. Otherwise works + like <c>tc/3</c>.</p> + </item> + </taglist> </desc> </func> <func> diff --git a/lib/stdlib/src/beam_lib.erl b/lib/stdlib/src/beam_lib.erl index 91ff2438c6..e612bf71e7 100644 --- a/lib/stdlib/src/beam_lib.erl +++ b/lib/stdlib/src/beam_lib.erl @@ -105,6 +105,7 @@ | info_rsn(). -type cmp_rsn() :: {'modules_different', module(), module()} | {'chunks_different', chunkid()} + | 'different_chunks' | info_rsn(). %%------------------------------------------------------------------------- diff --git a/lib/stdlib/src/escript.erl b/lib/stdlib/src/escript.erl index d26443f277..99e454f593 100644 --- a/lib/stdlib/src/escript.erl +++ b/lib/stdlib/src/escript.erl @@ -24,35 +24,41 @@ %% Internal API. -export([start/0, start/1]). --include_lib("kernel/include/file.hrl"). +%%----------------------------------------------------------------------- -define(SHEBANG, "/usr/bin/env escript"). -define(COMMENT, "This is an -*- erlang -*- file"). --record(state, {file, - module, +%%----------------------------------------------------------------------- + +-type mode() :: 'compile' | 'debug' | 'interpret' | 'run'. +-type source() :: 'archive' | 'beam' | 'text'. + +-record(state, {file :: file:filename(), + module :: module(), forms_or_bin, - source, - n_errors, - mode, - exports_main, - has_records}). --record(sections, {type, - shebang, - comment, - emu_args, - body}). --record(extract_options, {compile_source}). + source :: source(), + n_errors :: non_neg_integer(), + mode :: mode(), + exports_main :: boolean(), + has_records :: boolean()}). -type shebang() :: string(). -type comment() :: string(). -type emu_args() :: string(). --type escript_filename() :: string(). --type filename() :: string(). + +-record(sections, {type, + shebang :: shebang(), + comment :: comment(), + emu_args :: emu_args(), + body}). + +-record(extract_options, {compile_source}). + -type zip_file() :: - filename() - | {filename(), binary()} - | {filename(), binary(), #file_info{}}. + file:filename() + | {file:filename(), binary()} + | {file:filename(), binary(), file:file_info()}. -type zip_create_option() :: term(). -type section() :: shebang @@ -60,13 +66,15 @@ | comment | {comment, comment()} | {emu_args, emu_args()} - | {source, filename() | binary()} - | {beam, filename() | binary()} - | {archive, filename() | binary()} + | {source, file:filename() | binary()} + | {beam, file:filename() | binary()} + | {archive, file:filename() | binary()} | {archive, [zip_file()], [zip_create_option()]}. +%%----------------------------------------------------------------------- + %% Create a complete escript file with both header and body --spec create(escript_filename() | binary, [section()]) -> +-spec create(file:filename() | binary, [section()]) -> ok | {ok, binary()} | {error, term()}. create(File, Options) when is_list(Options) -> @@ -149,7 +157,9 @@ prepare(BadOptions, _) -> -type section_name() :: shebang | comment | emu_args | body . -type extract_option() :: compile_source | {section, [section_name()]}. --spec extract(filename(), [extract_option()]) -> {ok, [section()]} | {error, term()}. +-spec extract(file:filename(), [extract_option()]) -> + {ok, [section()]} | {error, term()}. + extract(File, Options) when is_list(File), is_list(Options) -> try EO = parse_extract_options(Options, @@ -239,6 +249,7 @@ normalize_section(Name, Chars) -> {Name, Chars}. -spec script_name() -> string(). + script_name() -> [ScriptName|_] = init:get_plain_arguments(), ScriptName. @@ -248,10 +259,12 @@ script_name() -> %% -spec start() -> no_return(). + start() -> start([]). -spec start([string()]) -> no_return(). + start(EscriptOptions) -> try %% Commands run using -run or -s are run in a process @@ -484,18 +497,12 @@ find_first_body_line(Fd, HeaderSz0, LineNo, KeepFirst, Sections) -> classify_line(Line) -> case Line of - [$\#, $\! | _] -> - shebang; - [$P, $K | _] -> - archive; - [$F, $O, $R, $1 | _] -> - beam; - [$%, $%, $\! | _] -> - emu_args; - [$% | _] -> - comment; - _ -> - undefined + "#!" ++ _ -> shebang; + "PK" ++ _ -> archive; + "FOR1" ++ _ -> beam; + "%%!" ++ _ -> emu_args; + "%" ++ _ -> comment; + _ -> undefined end. guess_type(Line) -> @@ -531,7 +538,6 @@ parse_archive(S, File, HeaderSz) -> end, list_to_atom(lists:reverse(RevBase2)) end, - S#state{source = archive, mode = run, module = Mod, @@ -587,8 +593,8 @@ parse_source(S, File, Fd, StartLine, HeaderSz, CheckOnly) -> epp_parse_file2(Epp, S2, [ModForm, FileForm], OptModRes); {error, _} -> epp_parse_file2(Epp, S2, [FileForm], OptModRes); - {eof,LastLine} -> - S#state{forms_or_bin = [FileForm, {eof,LastLine}]} + {eof, _LastLine} = Eof -> + S#state{forms_or_bin = [FileForm, Eof]} end, ok = epp:close(Epp), ok = file:close(Fd), @@ -683,8 +689,8 @@ epp_parse_file2(Epp, S, Forms, Parsed) -> io:format("~s:~w: ~s\n", [S#state.file,Ln,Mod:format_error(Args)]), epp_parse_file(Epp, S#state{n_errors = S#state.n_errors + 1}, [Form | Forms]); - {eof,LastLine} -> - S#state{forms_or_bin = lists:reverse([{eof, LastLine} | Forms])} + {eof, _LastLine} = Eof -> + S#state{forms_or_bin = lists:reverse([Eof | Forms])} end. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% diff --git a/lib/stdlib/src/otp_internal.erl b/lib/stdlib/src/otp_internal.erl index f3cfd78d54..1514414e48 100644 --- a/lib/stdlib/src/otp_internal.erl +++ b/lib/stdlib/src/otp_internal.erl @@ -349,14 +349,18 @@ obsolete_1(ssh_sftp, stop, 1) -> %% Added in R13B01. obsolete_1(ssl_pkix, decode_cert_file, A) when A =:= 1; A =:= 2 -> - {deprecated,"deprecated (will be removed in R14B); use public_key:pem_to_der/1 and public_key:pkix_decode_cert/2 instead"}; + {removed,"removed in R14A; use public_key:pem_to_der/1 and public_key:pkix_decode_cert/2 instead"}; obsolete_1(ssl_pkix, decode_cert, A) when A =:= 1; A =:= 2 -> - {deprecated,{public_key,pkix_decode_cert,2},"R14B"}; + {removed,{public_key,pkix_decode_cert,2},"R14A"}; %% Added in R13B04. obsolete_1(erlang, concat_binary, 1) -> {deprecated,{erlang,list_to_binary,1},"R15B"}; - + +%% Added in R14A. +obsolete_1(ssl, peercert, 2) -> + {deprecated,"deprecated (will be removed in R15A); use ssl:peercert/1 and public_key:pkix_decode_cert/2 instead"}; + obsolete_1(_, _, _) -> no. diff --git a/lib/stdlib/src/timer.erl b/lib/stdlib/src/timer.erl index 6ee837c3e6..24e14caa69 100644 --- a/lib/stdlib/src/timer.erl +++ b/lib/stdlib/src/timer.erl @@ -22,7 +22,7 @@ send_after/3, send_after/2, exit_after/3, exit_after/2, kill_after/2, kill_after/1, apply_interval/4, send_interval/3, send_interval/2, - cancel/1, sleep/1, tc/3, now_diff/2, + cancel/1, sleep/1, tc/2, tc/3, now_diff/2, seconds/1, minutes/1, hours/1, hms/3]). -export([start_link/0, start/0, @@ -98,6 +98,17 @@ sleep(T) -> after T -> ok end. + +%% +%% Measure the execution time (in microseconds) for Fun(Args). +%% +-spec tc(function(), [_]) -> {time(), term()}. +tc(F, A) -> + Before = erlang:now(), + Val = (catch apply(F, A)), + After = erlang:now(), + {now_diff(After, Before), Val}. + %% %% Measure the execution time (in microseconds) for an MFA. %% diff --git a/lib/stdlib/test/ets_SUITE.erl b/lib/stdlib/test/ets_SUITE.erl index 13c87ca005..5f26b7d431 100644 --- a/lib/stdlib/test/ets_SUITE.erl +++ b/lib/stdlib/test/ets_SUITE.erl @@ -63,7 +63,8 @@ meta_lookup_unnamed_read/1, meta_lookup_unnamed_write/1, meta_lookup_named_read/1, meta_lookup_named_write/1, meta_newdel_unnamed/1, meta_newdel_named/1]). --export([smp_insert/1, smp_fixed_delete/1, smp_unfix_fix/1, smp_select_delete/1, otp_8166/1]). +-export([smp_insert/1, smp_fixed_delete/1, smp_unfix_fix/1, smp_select_delete/1, + otp_8166/1, otp_8732/1]). -export([exit_large_table_owner/1, exit_many_large_table_owner/1, exit_many_tables_owner/1, @@ -129,7 +130,7 @@ all(suite) -> t_select_delete, t_ets_dets, memory, t_bucket_disappears, select_fail,t_insert_new, t_repair_continuation, otp_5340, otp_6338, - otp_6842_select_1000, otp_7665, + otp_6842_select_1000, otp_7665, otp_8732, meta_wb, grow_shrink, grow_pseudo_deleted, shrink_pseudo_deleted, meta_smp, @@ -391,7 +392,7 @@ memory(Config) when is_list(Config) -> ?line erts_debug:set_internal_state(available_internal_state, true), ?line ok = chk_normal_tab_struct_size(), ?line L = [T1,T2,T3,T4] = fill_sets_int(1000), - ?line XRes1 = adjust_xmem(L, {16862,16072,16072,16078}), + ?line XRes1 = adjust_xmem(L, {14862,14072,14072,14078}), ?line Res1 = {?S(T1),?S(T2),?S(T3),?S(T4)}, ?line lists:foreach(fun(T) -> Before = ets:info(T,size), @@ -402,7 +403,7 @@ memory(Config) when is_list(Config) -> [Key, ets:info(T,type), Before, ets:info(T,size), Objs]) end, L), - ?line XRes2 = adjust_xmem(L, {16849,16060,16048,16054}), + ?line XRes2 = adjust_xmem(L, {14851,14062,14052,14058}), ?line Res2 = {?S(T1),?S(T2),?S(T3),?S(T4)}, ?line lists:foreach(fun(T) -> Before = ets:info(T,size), @@ -413,7 +414,7 @@ memory(Config) when is_list(Config) -> [Key, ets:info(T,type), Before, ets:info(T,size), Objs]) end, L), - ?line XRes3 = adjust_xmem(L, {16836,16048,16024,16030}), + ?line XRes3 = adjust_xmem(L, {14840,14052,14032,14038}), ?line Res3 = {?S(T1),?S(T2),?S(T3),?S(T4)}, ?line lists:foreach(fun(T) -> ?line ets:delete_all_objects(T) @@ -5010,8 +5011,14 @@ verify_table_load(T) -> end. - - +otp_8732(doc) -> ["ets:select on a tree with NIL key object"]; +otp_8732(Config) when is_list(Config) -> + Tab = ets:new(noname,[ordered_set]), + filltabstr(Tab,999), + ets:insert(Tab,{[],"nasty NIL object"}), + ?line [] = ets:match(Tab,{'_',nomatch}), %% Will hang if bug not fixed + ok. + smp_select_delete(suite) -> []; smp_select_delete(doc) -> diff --git a/lib/stdlib/test/timer_simple_SUITE.erl b/lib/stdlib/test/timer_simple_SUITE.erl index 021a22c61b..6aa2b7b945 100644 --- a/lib/stdlib/test/timer_simple_SUITE.erl +++ b/lib/stdlib/test/timer_simple_SUITE.erl @@ -224,11 +224,19 @@ cancel2(Config) when is_list(Config) -> tc(doc) -> "Test sleep/1 and tc/3."; tc(suite) -> []; tc(Config) when is_list(Config) -> - % This should both sleep and tc - ?line {Res, ok} = timer:tc(timer, sleep, [500]), - ?line ok = if - Res < 500*1000 -> {too_early, Res}; % Too early - Res > 800*1000 -> {too_late, Res}; % Too much time + % This should both sleep and tc/3 + ?line {Res1, ok} = timer:tc(timer, sleep, [500]), + ?line ok = if + Res1 < 500*1000 -> {too_early, Res1}; % Too early + Res1 > 800*1000 -> {too_late, Res1}; % Too much time + true -> ok + end, + + % This should both sleep and tc/2 + ?line {Res2, ok} = timer:tc(fun(T) -> timer:sleep(T) end, [500]), + ?line ok = if + Res2 < 500*1000 -> {too_early, Res2}; % Too early + Res2 > 800*1000 -> {too_late, Res2}; % Too much time true -> ok end, diff --git a/lib/stdlib/vsn.mk b/lib/stdlib/vsn.mk index f081bd60e2..02ea6d9d68 100644 --- a/lib/stdlib/vsn.mk +++ b/lib/stdlib/vsn.mk @@ -17,5 +17,4 @@ # %CopyrightEnd% # -STDLIB_VSN = 1.17 - +STDLIB_VSN = 1.17.1 diff --git a/lib/syntax_tools/doc/src/notes.xml b/lib/syntax_tools/doc/src/notes.xml index a3b842b50b..fca93a27d9 100644 --- a/lib/syntax_tools/doc/src/notes.xml +++ b/lib/syntax_tools/doc/src/notes.xml @@ -31,6 +31,21 @@ <p>This document describes the changes made to the Syntax_Tools application.</p> +<section><title>Syntax_Tools 1.6.6</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Minor changes and clean-ups.</p> + <p> + Own Id: OTP-8709</p> + </item> + </list> + </section> + +</section> + <section><title>Syntax_Tools 1.6.5</title> <section><title>Improvements and New Features</title> diff --git a/lib/syntax_tools/vsn.mk b/lib/syntax_tools/vsn.mk index 2ba5eac582..6051fb8e39 100644 --- a/lib/syntax_tools/vsn.mk +++ b/lib/syntax_tools/vsn.mk @@ -1 +1 @@ -SYNTAX_TOOLS_VSN = 1.6.5 +SYNTAX_TOOLS_VSN = 1.6.6 diff --git a/lib/test_server/doc/src/notes.xml b/lib/test_server/doc/src/notes.xml index b6e0a6cefa..dae071311c 100644 --- a/lib/test_server/doc/src/notes.xml +++ b/lib/test_server/doc/src/notes.xml @@ -32,6 +32,70 @@ <file>notes.xml</file> </header> +<section><title>Test_Server 3.4</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Returning {fail,Reason} from the framework end_tc + function was not handled properly by Test Server for all + test suite functions.</p> + <p> + Own Id: OTP-8492 Aux Id: seq11502 </p> + </item> + <item> + <p> + If the framework end_tc function would hang and get + aborted by Test Server, there was no indication of + failure in the logs. This has been fixed.</p> + <p> + Own Id: OTP-8682 Aux Id: seq11504 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + It is now possible for the Test Server framework end_tc + function to change the status of the test case from ok or + auto-skipped to failed by returning {fail,Reason}.</p> + <p> + Own Id: OTP-8495 Aux Id: seq11502 </p> + </item> + <item> + <p> + Test Server will now call the end_per_testcase/2 function + even if the test case has been terminated explicitly + (with abort_current_testcase/1), or after a timetrap + timeout. Under these circumstances the return value of + end_per_testcase is completely ignored. Therefore the + function will not be able to change the reason for test + case termination by returning {fail,Reason}, nor will it + be able to save data with {save_config,Data}.</p> + <p> + Own Id: OTP-8500 Aux Id: seq11521 </p> + </item> + <item> + <p> + Previously, a repeat property of a test case group + specified the number of times the group should be + repeated after the main test run. I.e. {repeat,N} would + case the group to execute 1+N times. To be consistent + with the behaviour of the run_test repeat option, this + has been changed. N now specifies the absolute number of + executions instead.</p> + <p> + Own Id: OTP-8689 Aux Id: seq11502 </p> + </item> + </list> + </section> + +</section> + <section><title>Test_Server 3.3.6</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/test_server/doc/src/test_server.xml b/lib/test_server/doc/src/test_server.xml index 6e75425862..0cae75d692 100644 --- a/lib/test_server/doc/src/test_server.xml +++ b/lib/test_server/doc/src/test_server.xml @@ -167,6 +167,22 @@ </desc> </func> <func> + <name>adjusted_sleep(MSecs) -> ok</name> + <fsummary>Suspens the calling task for a specified time.</fsummary> + <type> + <v>MSecs = integer() | float() | infinity</v> + <d>The default number of milliseconds to sleep</d> + </type> + <desc> + <p>This function suspends the calling process for at least the + supplied number of milliseconds. The function behaves the same + way as <c>test_server:sleep/1</c>, only <c>MSecs</c> + will be multiplied by the 'multiply_timetraps' value, if set, + and also automatically scaled up if 'scale_timetraps' is set + to true (which it is by default).</p> + </desc> + </func> + <func> <name>hours(N) -> MSecs</name> <name>minutes(N) -> MSecs</name> <name>seconds(N) -> MSecs</name> diff --git a/lib/test_server/doc/src/test_server_ctrl.xml b/lib/test_server/doc/src/test_server_ctrl.xml index 8b60849b61..2368c4bacc 100644 --- a/lib/test_server/doc/src/test_server_ctrl.xml +++ b/lib/test_server/doc/src/test_server_ctrl.xml @@ -376,6 +376,31 @@ Optional, if not given the test server controller node </desc> </func> <func> + <name>scale_timetraps(Bool) -> ok</name> + <fsummary>.</fsummary> + <type> + <v>Bool = true | false</v> + </type> + <desc> + <p>This function should be called before a test is started. + The parameter specifies if test_server should attempt + to automatically scale the timetrap value in order to compensate + for delays caused by e.g. the cover tool.</p> + </desc> + </func> + <func> + <name>get_timetrap_parameters() -> {N,Bool} </name> + <fsummary>Read the parameter values that affect timetraps.</fsummary> + <type> + <v>N = integer() | infinity</v> + <v>Bool = true | false</v> + </type> + <desc> + <p>This function may be called to read the values set by + <c>multiply_timetraps/1</c> and <c>scale_timetraps/1</c>.</p> + </desc> + </func> + <func> <name>cover(Application,Analyse) -> ok</name> <name>cover(CoverFile,Analyse) -> ok</name> <name>cover(App,CoverFile,Analyse) -> ok</name> diff --git a/lib/test_server/src/test_server.erl b/lib/test_server/src/test_server.erl index 7db103a4c6..acc9dbaab8 100644 --- a/lib/test_server/src/test_server.erl +++ b/lib/test_server/src/test_server.erl @@ -35,7 +35,7 @@ -export([fail/0,fail/1,format/1,format/2,format/3]). -export([capture_start/0,capture_stop/0,capture_get/0]). -export([messages_get/0]). --export([hours/1,minutes/1,seconds/1,sleep/1,timecall/3]). +-export([hours/1,minutes/1,seconds/1,sleep/1,adjusted_sleep/1,timecall/3]). -export([timetrap_scale_factor/0,timetrap/1,timetrap_cancel/1]). -export([m_out_of_n/3,do_times/4,do_times/2]). -export([call_crash/3,call_crash/4,call_crash/5]). @@ -89,14 +89,14 @@ init(Host,Port,Starter) -> global:register_name(?MODULE,self()), process_flag(trap_exit,true), test_server_sup:cleanup_crash_dumps(), - case gen_tcp:connect(Host,Port, [binary, - {reuseaddr,true}, + case gen_tcp:connect(Host,Port, [binary, + {reuseaddr,true}, {packet,2}]) of - {ok,MainSock} -> + {ok,MainSock} -> Starter ! {self(),started}, request(MainSock,{target_info,init_target_info()}), loop(#state{controller={Host,MainSock}}); - Error -> + Error -> Starter ! {self(),{error, {could_not_contact_controller,Error}}} end. @@ -127,7 +127,7 @@ loop(#state{controller={_,MainSock}} = State) -> halt(); {'EXIT',Pid,Reason} -> case lists:keysearch(Pid,1,State#state.jobs) of - {value,{Pid,Name}} -> + {value,{Pid,Name}} -> case Reason of normal -> ignore; _other -> request(MainSock,{job_proc_killed,Name,Reason}) @@ -157,14 +157,14 @@ init_purify() -> job(Host,Port,Starter) -> process_flag(trap_exit,true), init_purify(), - case gen_tcp:connect(Host,Port, [binary, - {reuseaddr,true}, + case gen_tcp:connect(Host,Port, [binary, + {reuseaddr,true}, {packet,4}, {active,false}]) of {ok,JobSock} -> Starter ! {self(),started}, job(JobSock); - Error -> + Error -> Starter ! {self(),{error, {could_not_contact_controller,Error}}} end. @@ -192,7 +192,7 @@ get_jobdir() -> true -> {ok,Cwd} = file:get_cwd(), Cwd ++ "/" ++ Basename; - false -> + false -> filename:absname(Basename) end. @@ -216,7 +216,7 @@ send_privdir(JobDir,JobSock) -> del_dir(Dir) -> case file:read_file_info(Dir) of - {ok,#file_info{type=directory}} -> + {ok,#file_info{type=directory}} -> {ok,Cont} = file:list_dir(Dir), lists:foreach(fun(F) -> del_dir(filename:join(Dir,F)) end, Cont), ok = file:del_dir(Dir); @@ -227,7 +227,7 @@ del_dir(Dir) -> catch file:delete(Dir), ok end. - + %% %% Receive and decode request on job socket %% @@ -237,7 +237,7 @@ job_loop(JobSock) -> ok -> job_loop(JobSock); {stop,R} -> R end. - + decode_job({{beam,Mod,Which},Beam}) -> % FIXME, shared directory structure on host and target required, % "Library beams" are not loaded from HOST... /Patrik @@ -254,7 +254,7 @@ decode_job({{datadir,Tarfile0},Archive}) -> ok = erl_tar:extract(Tarfile,[compressed,{cwd,JobDir}]), ok = file:delete(Tarfile), ok; -decode_job({test_case,Case}) -> +decode_job({test_case,Case}) -> Result = run_test_case_apply(Case), JobSock = get(test_server_job_sock), request(JobSock,{test_case_result,Result}), @@ -266,11 +266,11 @@ decode_job({test_case,Case}) -> request(JobSock,{{crash_dumps,filename:basename(TarFile)},TarBin}) end, ok; -decode_job({sync_apply,{M,F,A}}) -> +decode_job({sync_apply,{M,F,A}}) -> R = apply(M,F,A), request(get(test_server_job_sock),{sync_result,R}), ok; -decode_job(job_done) -> +decode_job(job_done) -> {stop,stopped}. %% @@ -282,9 +282,9 @@ decode_job(job_done) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% cover_compile({App,Include,Exclude,Cross}) -> +%% cover_compile({App,Include,Exclude,Cross}) -> %% {ok,AnalyseModules} | {error,Reason} -%% +%% %% App = atom() , name of application to be compiled %% Exclude = [atom()], list of modules to exclude %% Include = [atom()], list of modules outside of App that should be included @@ -293,7 +293,7 @@ decode_job(job_done) -> %% in the cover compilation, but that shall not be part of %% the cover analysis for this application. %% -%% Cover compile the given application. Return {ok,AnalyseMods} if application +%% Cover compile the given application. Return {ok,AnalyseMods} if application %% is found, else {error,application_not_found}. cover_compile({none,_Exclude,Include,Cross}) -> @@ -330,7 +330,7 @@ cover_compile({App,all,Include,Cross}) -> end; cover_compile({App,Exclude,Include,Cross}) -> case code:lib_dir(App) of - {error,bad_name} -> + {error,bad_name} -> case Include++Cross of [] -> io:format("\nWARNING: Can't find lib_dir for \'~w\'\n" @@ -366,7 +366,7 @@ cover_compile({App,Exclude,Include,Cross}) -> {ok,AnalyseMods} end end. - + module_names(Beams) -> [list_to_atom(filename:basename(filename:rootname(Beam))) || Beam <- Beams]. @@ -380,11 +380,11 @@ do_cover_compile1([Dont|Rest]) when Dont=:=cover; Dont=:=test_server_ctrl -> do_cover_compile1(Rest); do_cover_compile1([M|Rest]) -> - case {code:is_sticky(M),code:is_loaded(M)} of + case {code:is_sticky(M),code:is_loaded(M)} of {true,_} -> code:unstick_mod(M), case cover:compile_beam(M) of - {ok,_} -> + {ok,_} -> ok; Error -> io:fwrite("\nWARNING: Could not cover compile ~w: ~p\n", @@ -402,7 +402,7 @@ do_cover_compile1([M|Rest]) -> end; {false,_} -> case cover:compile_beam(M) of - {ok,_} -> + {ok,_} -> ok; Error -> io:fwrite("\nWARNING: Could not cover compile ~w: ~p\n", @@ -415,14 +415,14 @@ do_cover_compile1([]) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% cover_analyse(Analyse,Modules) -> [{M,{Cov,NotCov,Details}}] -%% +%% %% Analyse = {details,Dir} | details | {overview,void()} | overview %% Modules = [atom()], the modules to analyse %% %% Cover analysis. If this is a remote target, analyse_to_file can not be used. %% In that case the analyse level 'line' is used instead if Analyse==details. %% -%% If this is a local target, the test directory is given +%% If this is a local target, the test directory is given %% (Analyse=={details,Dir}) and analyse_to_file can be used directly. %% %% If Analyse==overview | {overview,Dir} analyse_to_file is not used, only @@ -432,12 +432,12 @@ do_cover_compile1([]) -> %% all.coverdata in that directory. cover_analyse(Analyse,Modules) -> io:fwrite("Cover analysing...\n",[]), - DetailsFun = + DetailsFun = case Analyse of {details,Dir} -> case cover:export(filename:join(Dir,"all.coverdata")) of ok -> - fun(M) -> + fun(M) -> OutFile = filename:join(Dir, atom_to_list(M) ++ ".COVER.html"), @@ -451,7 +451,7 @@ cover_analyse(Analyse,Modules) -> Error -> fun(_) -> Error end end; - details -> + details -> fun(M) -> case cover:analyse(M,line) of {ok,Lines} -> @@ -489,7 +489,7 @@ cover_analyse(Analyse,Modules) -> unstick_all_sticky(Node) -> lists:filter( - fun(M) -> + fun(M) -> case code:is_sticky(M) of true -> rpc:call(Node,code,unstick_mod,[M]), @@ -502,24 +502,24 @@ unstick_all_sticky(Node) -> stick_all_sticky(Node,Sticky) -> lists:foreach( - fun(M) -> + fun(M) -> rpc:call(Node,code,stick_mod,[M]) end, Sticky). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% run_test_case_apply(Mod,Func,Args,Name,RunInit,MultiplyTimetrap) -> +%% run_test_case_apply(Mod,Func,Args,Name,RunInit,TimetrapData) -> %% {Time,Value,Loc,Opts,Comment} | {died,Reason,unknown,Comment} -%% +%% %% Time = float() (seconds) %% Value = term() %% Loc = term() %% Comment = string() %% Reason = term() %% -%% Spawns off a process (case process) that actually runs the test suite. -%% The case process will have the job process as group leader, which makes +%% Spawns off a process (case process) that actually runs the test suite. +%% The case process will have the job process as group leader, which makes %% it possible to capture all it's output from io:format/2, etc. %% %% The job process then sits down and waits for news from the case process. @@ -535,40 +535,43 @@ stick_all_sticky(Node,Sticky) -> %% called or the comment given by the return value {comment,Comment} from %% a test case. %% -%% {died,Reason,unknown,Comment} is returned if the test case was killed +%% {died,Reason,unknown,Comment} is returned if the test case was killed %% by some other process. Reason is the kill reason provided. %% -%% MultiplyTimetrap indicates a possible extension of all timetraps -%% Timetraps will be multiplied by this integer. If it is infinity, no -%% timetraps will be started at all. +%% TimetrapData = {MultiplyTimetrap,ScaleTimetrap}, which indicates a +%% possible extension of all timetraps. Timetraps will be multiplied by +%% MultiplyTimetrap. If it is infinity, no timetraps will be started at all. +%% ScaleTimetrap indicates if test_server should attemp to automatically +%% compensate timetraps for runtime delays introduced by e.g. tools like +%% cover. -run_test_case_apply({CaseNum,Mod,Func,Args,Name,RunInit,MultiplyTimetrap}) -> +run_test_case_apply({CaseNum,Mod,Func,Args,Name,RunInit,TimetrapData}) -> purify_format("Test case #~w ~w:~w/1", [CaseNum, Mod, Func]), case os:getenv("TS_RUN_VALGRIND") of - false -> + false -> ok; _ -> os:putenv("VALGRIND_LOGFILE_INFIX",atom_to_list(Mod)++"."++ atom_to_list(Func)++"-") end, test_server_h:testcase({Mod,Func,1}), - ProcBef = erlang:system_info(process_count), - Result = run_test_case_apply(Mod, Func, Args, Name, RunInit, MultiplyTimetrap), + ProcBef = erlang:system_info(process_count), + Result = run_test_case_apply(Mod, Func, Args, Name, RunInit, TimetrapData), ProcAft = erlang:system_info(process_count), purify_new_leaks(), DetFail = get(test_server_detected_fail), {Result,DetFail,ProcBef,ProcAft}. - -run_test_case_apply(Mod, Func, Args, Name, RunInit, MultiplyTimetrap) -> + +run_test_case_apply(Mod, Func, Args, Name, RunInit, TimetrapData) -> case get(test_server_job_dir) of undefined -> %% i'm a local target - do_run_test_case_apply(Mod, Func, Args, Name, RunInit, MultiplyTimetrap); + do_run_test_case_apply(Mod, Func, Args, Name, RunInit, TimetrapData); JobDir -> %% i'm a remote target case Args of [Config] when is_list(Config) -> - {value,{data_dir,HostDataDir}} = + {value,{data_dir,HostDataDir}} = lists:keysearch(data_dir, 1, Config), DataBase = filename:basename(HostDataDir), TargetDataDir = filename:join(JobDir, DataBase), @@ -578,18 +581,18 @@ run_test_case_apply(Mod, Func, Args, Name, RunInit, MultiplyTimetrap) -> Config2 = lists:keyreplace(priv_dir, 1, Config1, {priv_dir,TargetPrivDir}), do_run_test_case_apply(Mod, Func, [Config2], Name, RunInit, - MultiplyTimetrap); + TimetrapData); _other -> do_run_test_case_apply(Mod, Func, Args, Name, RunInit, - MultiplyTimetrap) + TimetrapData) end end. -do_run_test_case_apply(Mod, Func, Args, Name, RunInit, MultiplyTimetrap) -> +do_run_test_case_apply(Mod, Func, Args, Name, RunInit, TimetrapData) -> {ok,Cwd} = file:get_cwd(), Args2Print = case Args of - [Args1] when is_list(Args1) -> + [Args1] when is_list(Args1) -> lists:keydelete(tc_group_result, 1, Args1); - _ -> + _ -> Args end, print(minor, "Test case started with:\n~s:~s(~p)\n", [Mod,Func,Args2Print]), @@ -600,16 +603,16 @@ do_run_test_case_apply(Mod, Func, Args, Name, RunInit, MultiplyTimetrap) -> OldGLeader = group_leader(), %% Set ourself to group leader for the spawned process group_leader(self(),self()), - Pid = + Pid = spawn_link( - fun() -> - run_test_case_eval(Mod, Func, Args, Name, Ref, - RunInit, MultiplyTimetrap, + fun() -> + run_test_case_eval(Mod, Func, Args, Name, Ref, + RunInit, TimetrapData, TCCallback) end), group_leader(OldGLeader, self()), put(test_server_detected_fail, []), - run_test_case_msgloop(Ref, Pid, false, false, ""). + run_test_case_msgloop(Ref, Pid, false, false, "", undefined). %% Ugly bug (pre R5A): %% If this process (group leader of the test case) terminates before @@ -620,7 +623,7 @@ do_run_test_case_apply(Mod, Func, Args, Name, RunInit, MultiplyTimetrap) -> %% A test case is known to have failed if it returns {'EXIT', _} tuple, %% or sends a message {failed, File, Line} to it's group_leader %% -run_test_case_msgloop(Ref, Pid, CaptureStdout, Terminate, Comment) -> +run_test_case_msgloop(Ref, Pid, CaptureStdout, Terminate, Comment, CurrConf) -> %% NOTE: Keep job_proxy_msgloop/0 up to date when changes %% are made in this function! {Timeout,ReturnValue} = @@ -641,13 +644,13 @@ run_test_case_msgloop(Ref, Pid, CaptureStdout, Terminate, Comment) -> receive {'DOWN', Mon, process, Pid, _} -> Comment - after 10000 -> + after 10000 -> %% Pid is probably trapping exits, hit it harder... exit(Pid, kill), %% here's the only place we know Reason, so we save %% it as a comment, potentially replacing user data Error = lists:flatten(io_lib:format("Aborted: ~p",[Reason])), - Error1 = lists:flatten([string:strip(S,left) || + Error1 = lists:flatten([string:strip(S,left) || S <- string:tokens(Error,[$\n])]), if length(Error1) > 63 -> string:substr(Error1,1,60) ++ "..."; @@ -655,149 +658,224 @@ run_test_case_msgloop(Ref, Pid, CaptureStdout, Terminate, Comment) -> Error1 end end, - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,NewComment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,NewComment,CurrConf); {io_request,From,ReplyAs,{put_chars,io_lib,Func,[Format,Args]}} when is_list(Format) -> Msg = (catch io_lib:Func(Format,Args)), run_test_case_msgloop_io(ReplyAs,CaptureStdout,Msg,From,Func), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {io_request,From,ReplyAs,{put_chars,io_lib,Func,[Format,Args]}} when is_atom(Format) -> Msg = (catch io_lib:Func(Format,Args)), run_test_case_msgloop_io(ReplyAs,CaptureStdout,Msg,From,Func), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {io_request,From,ReplyAs,{put_chars,Bytes}} -> run_test_case_msgloop_io( ReplyAs,CaptureStdout,Bytes,From,put_chars), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {io_request,From,ReplyAs,{put_chars,unicode,io_lib,Func,[Format,Args]}} when is_list(Format) -> Msg = unicode_to_latin1(catch io_lib:Func(Format,Args)), run_test_case_msgloop_io(ReplyAs,CaptureStdout,Msg,From,Func), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {io_request,From,ReplyAs,{put_chars,latin1,io_lib,Func,[Format,Args]}} when is_list(Format) -> Msg = (catch io_lib:Func(Format,Args)), run_test_case_msgloop_io(ReplyAs,CaptureStdout,Msg,From,Func), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {io_request,From,ReplyAs,{put_chars,unicode,io_lib,Func,[Format,Args]}} when is_atom(Format) -> Msg = unicode_to_latin1(catch io_lib:Func(Format,Args)), run_test_case_msgloop_io(ReplyAs,CaptureStdout,Msg,From,Func), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {io_request,From,ReplyAs,{put_chars,latin1,io_lib,Func,[Format,Args]}} when is_atom(Format) -> Msg = (catch io_lib:Func(Format,Args)), run_test_case_msgloop_io(ReplyAs,CaptureStdout,Msg,From,Func), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {io_request,From,ReplyAs,{put_chars,unicode,Bytes}} -> run_test_case_msgloop_io( ReplyAs,CaptureStdout,unicode_to_latin1(Bytes),From,put_chars), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {io_request,From,ReplyAs,{put_chars,latin1,Bytes}} -> run_test_case_msgloop_io( ReplyAs,CaptureStdout,Bytes,From,put_chars), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); IoReq when element(1, IoReq) == io_request -> %% something else, just pass it on group_leader() ! IoReq, - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {structured_io,ClientPid,Msg} -> output(Msg, ClientPid), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {capture,NewCapture} -> - run_test_case_msgloop(Ref,Pid,NewCapture,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,NewCapture,Terminate,Comment,CurrConf); {sync_apply,From,MFA} -> sync_local_or_remote_apply(false,From,MFA), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {sync_apply_proxy,Proxy,From,MFA} -> sync_local_or_remote_apply(Proxy,From,MFA), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {printout,Detail,Format,Args} -> print(Detail,Format,Args), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {comment,NewComment} -> Terminate1 = case Terminate of - {true,{Time,Value,Loc,Opts,_OldComment}} -> + {true,{Time,Value,Loc,Opts,_OldComment}} -> {true,{Time,Value,mod_loc(Loc),Opts,NewComment}}; Other -> Other end, - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate1,NewComment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate1,NewComment,CurrConf); + {set_curr_conf,NewCurrConf} -> + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,NewCurrConf); {'EXIT',Pid,{Ref,Time,Value,Loc,Opts}} -> RetVal = {Time/1000000,Value,mod_loc(Loc),Opts,Comment}, - run_test_case_msgloop(Ref,Pid,CaptureStdout,{true,RetVal},Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,{true,RetVal},Comment,undefined); {'EXIT',Pid,Reason} -> case Reason of {timetrap_timeout,TVal,Loc} -> %% convert Loc to form that can be formatted - Loc1 = mod_loc(Loc), - {Mod,Func} = get_mf(Loc1), - %% The framework functions mustn't execute on this - %% group leader process or io will cause deadlock, - %% so we spawn a dedicated process for the operation - %% and let the group leader go back to handle io. - spawn_fw_call(Mod,Func,Pid,{timetrap_timeout,TVal}, - Loc1,self(),Comment), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + case mod_loc(Loc) of + {FwMod,FwFunc,framework} -> + %% timout during framework call + spawn_fw_call(FwMod,FwFunc,Pid, + {framework_error,{timetrap,TVal}}, + unknown,self(),Comment), + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate, + Comment,undefined); + Loc1 -> + {Mod,Func} = get_mf(Loc1), + %% call end_per_testcase on a separate process, + %% only so that the user has a chance to clean up + %% after init_per_testcase, even after a timetrap timeout + NewCurrConf = + case CurrConf of + {{Mod,Func},Conf} -> + EndConfPid = + call_end_conf(Mod,Func,Pid, + {timetrap_timeout,TVal}, + Loc1,[{tc_status, + {failed, + timetrap_timeout}}|Conf], + TVal), + {EndConfPid,{Mod,Func},Conf}; + _ -> + %% The framework functions mustn't execute on this + %% group leader process or io will cause deadlock, + %% so we spawn a dedicated process for the operation + %% and let the group leader go back to handle io. + spawn_fw_call(Mod,Func,Pid,{timetrap_timeout,TVal}, + Loc1,self(),Comment), + undefined + end, + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate, + Comment,NewCurrConf) + end; {timetrap_timeout,TVal,Loc,InitOrEnd} -> - Loc1 = mod_loc(Loc), - {Mod,_Func} = get_mf(Loc1), - spawn_fw_call(Mod,InitOrEnd,Pid,{timetrap_timeout,TVal}, - Loc1,self(),Comment), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); - {testcase_aborted,Reason,Loc} -> - Loc1 = mod_loc(Loc), - {Mod,Func} = get_mf(Loc1), - spawn_fw_call(Mod,Func,Pid,{testcase_aborted,Reason}, - Loc1,self(),Comment), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); - killed -> + case mod_loc(Loc) of + {FwMod,FwFunc,framework} -> + %% timout during framework call + spawn_fw_call(FwMod,FwFunc,Pid, + {framework_error,{timetrap,TVal}}, + unknown,self(),Comment); + Loc1 -> + {Mod,_Func} = get_mf(Loc1), + spawn_fw_call(Mod,InitOrEnd,Pid,{timetrap_timeout,TVal}, + Loc1,self(),Comment) + end, + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); + {testcase_aborted,AbortReason,AbortLoc} -> + ErrorMsg = {testcase_aborted,AbortReason}, + case mod_loc(AbortLoc) of + {FwMod,FwFunc,framework} -> + %% abort during framework call + spawn_fw_call(FwMod,FwFunc,Pid, + {framework_error,ErrorMsg}, + unknown,self(),Comment), + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate, + Comment,undefined); + Loc1 -> + {Mod,Func} = get_mf(Loc1), + %% call end_per_testcase on a separate process, only so + %% that the user has a chance to clean up after init_per_testcase, + %% even after abortion + NewCurrConf = + case CurrConf of + {{Mod,Func},Conf} -> + TVal = case lists:keysearch(default_timeout,1,Conf) of + {value,{default_timeout,Tmo}} -> Tmo; + _ -> ?DEFAULT_TIMETRAP_SECS*1000 + end, + EndConfPid = + call_end_conf(Mod,Func,Pid,ErrorMsg, + Loc1, + [{tc_status,{failed,ErrorMsg}}|Conf], + TVal), + {EndConfPid,{Mod,Func},Conf}; + _ -> + spawn_fw_call(Mod,Func,Pid,ErrorMsg, + Loc1,self(),Comment), + undefined + end, + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate, + Comment,NewCurrConf) + end; + killed -> %% result of an exit(TestCase,kill) call, which is the - %% only way to abort a testcase process that traps exits + %% only way to abort a testcase process that traps exits %% (see abort_current_testcase) spawn_fw_call(undefined,undefined,Pid,testcase_aborted_or_killed, unknown,self(),Comment), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); {fw_error,{FwMod,FwFunc,FwError}} -> spawn_fw_call(FwMod,FwFunc,Pid,{framework_error,FwError}, unknown,self(),Comment), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); - _ -> + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); + _Other -> %% the testcase has terminated because of Reason (e.g. an exit %% because a linked process failed) spawn_fw_call(undefined,undefined,Pid,Reason, unknown,self(),Comment), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment) + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf) + end; + {EndConfPid,{call_end_conf,Data,_Result}} -> + case CurrConf of + {EndConfPid,{Mod,Func},_Conf} -> + {_Mod,_Func,TCPid,TCExitReason,Loc} = Data, + spawn_fw_call(Mod,Func,TCPid,TCExitReason,Loc,self(),Comment), + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,undefined); + _ -> + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf) end; {_FwCallPid,fw_notify_done,RetVal} -> %% the framework has been notified, we're finished - run_test_case_msgloop(Ref,Pid,CaptureStdout,{true,RetVal},Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,{true,RetVal},Comment,undefined); {'EXIT',_FwCallPid,{fw_notify_done,Func,Error}} -> %% a framework function failed CB = os:getenv("TEST_SERVER_FRAMEWORK"), Loc = case CB of - false -> + false -> {test_server,Func}; - _ -> + _ -> {list_to_atom(CB),Func} end, RetVal = {died,{framework_error,Loc,Error},Loc,"Framework error"}, - run_test_case_msgloop(Ref,Pid,CaptureStdout,{true,RetVal},Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,{true,RetVal},Comment,undefined); {failed,File,Line} -> - put(test_server_detected_fail, + put(test_server_detected_fail, [{File, Line}| get(test_server_detected_fail)]), - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); _Other when not is_tuple(_Other) -> %% ignore anything not generated by test server - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment); + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf); _Other when element(1, _Other) /= 'EXIT', element(1, _Other) /= started, element(1, _Other) /= finished, element(1, _Other) /= print -> %% ignore anything not generated by test server - run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment) + run_test_case_msgloop(Ref,Pid,CaptureStdout,Terminate,Comment,CurrConf) after Timeout -> ReturnValue end. @@ -819,12 +897,43 @@ run_test_case_msgloop_io(ReplyAs,CaptureStdout,Msg,From,Func) -> output(Msg,Sender) -> local_or_remote_apply({test_server_ctrl,output,[Msg,Sender]}). +call_end_conf(Mod,Func,TCPid,TCExitReason,Loc,Conf,TVal) -> + Starter = self(), + Data = {Mod,Func,TCPid,TCExitReason,Loc}, + EndConfProc = + fun() -> + Supervisor = self(), + EndConfApply = + fun() -> + case catch apply(Mod,end_per_testcase,[Func,Conf]) of + {'EXIT',Why} -> + group_leader() ! {printout,12, + "ERROR! ~p:end_per_testcase(~p, ~p)" + " crashed!\n\tReason: ~p\n", + [Mod,Func,Conf,Why]}; + _ -> + ok + end, + Supervisor ! {self(),end_conf} + end, + Pid = spawn_link(EndConfApply), + receive + {Pid,end_conf} -> + Starter ! {self(),{call_end_conf,Data,ok}}; + {'EXIT',Pid,Reason} -> + Starter ! {self(),{call_end_conf,Data,{error,Reason}}} + after TVal -> + Starter ! {self(),{call_end_conf,Data,{error,timeout}}} + end + end, + spawn_link(EndConfProc). + spawn_fw_call(Mod,{init_per_testcase,Func},Pid,{timetrap_timeout,TVal}=Why, Loc,SendTo,Comment) -> FwCall = fun() -> Skip = {skip,{failed,{Mod,init_per_testcase,Why}}}, - %% if init_per_testcase fails, the test case + %% if init_per_testcase fails, the test case %% should be skipped case catch test_server_sup:framework_call( end_tc,[?pl2a(Mod),Func,{Pid,Skip,[[]]}]) of @@ -838,6 +947,7 @@ spawn_fw_call(Mod,{init_per_testcase,Func},Pid,{timetrap_timeout,TVal}=Why, {TVal/1000,Skip,Loc,[],Comment}} end, spawn_link(FwCall); + spawn_fw_call(Mod,{end_per_testcase,Func},Pid,{timetrap_timeout,TVal}=Why, Loc,SendTo,_Comment) -> FwCall = @@ -869,7 +979,7 @@ spawn_fw_call(FwMod,FwFunc,_Pid,{framework_error,FwError},_,SendTo,_Comment) -> fun() -> test_server_sup:framework_call(report, [framework_error, {{FwMod,FwFunc},FwError}]), - Comment = + Comment = lists:flatten( io_lib:format("<font color=\"red\">" "WARNING! ~w:~w failed!</font>", [FwMod,FwFunc])), @@ -953,9 +1063,10 @@ job_proxy_msgloop() -> %% A test case is known to have failed if it returns {'EXIT', _} tuple, %% or sends a message {failed, File, Line} to it's group_leader -run_test_case_eval(Mod, Func, Args0, Name, Ref, RunInit, - MultiplyTimetrap, TCCallback) -> - put(test_server_multiply_timetraps,MultiplyTimetrap), +run_test_case_eval(Mod, Func, Args0, Name, Ref, RunInit, + TimetrapData, TCCallback) -> + put(test_server_multiply_timetraps,TimetrapData), + {{Time,Value},Loc,Opts} = case test_server_sup:framework_call(init_tc,[?pl2a(Mod),Func,Args0], {ok,Args0}) of @@ -1004,6 +1115,8 @@ run_test_case_eval1(Mod, Func, Args, Name, RunInit, TCCallback) -> put(test_server_init_or_end_conf,undefined), %% call user callback function if defined NewConf1 = user_callback(TCCallback, Mod, Func, init, NewConf), + %% save current state in controller loop + group_leader() ! {set_curr_conf,{{Mod,Func},NewConf1}}, put(test_server_loc, {Mod,Func}), %% execute the test case {{T,Return},Loc} = {ts_tc(Mod, Func, [NewConf1]),get_loc()}, @@ -1025,6 +1138,8 @@ run_test_case_eval1(Mod, Func, Args, Name, RunInit, TCCallback) -> _ -> {[{tc_status,ok}|NewConf1],Return,ok} end, + %% clear current state in controller loop + group_leader() ! {set_curr_conf,undefined}, %% call user callback function if defined EndConf1 = user_callback(TCCallback, Mod, Func, 'end', EndConf), {FWReturn1,TSReturn1,EndConf2} = @@ -1036,9 +1151,10 @@ run_test_case_eval1(Mod, Func, Args, Name, RunInit, TCCallback) -> {{error,ReasonToFail},{failed,ReasonToFail},EndConf1}; {failed,{_,end_per_testcase,_}} = Failure -> % unexpected termination {Failure,TSReturn,EndConf1}; - _ -> + _ -> {FWReturn,TSReturn,EndConf1} end, + put(test_server_init_or_end_conf,undefined), case test_server_sup:framework_call(end_tc, [?pl2a(Mod), Func, {FWReturn1,[EndConf2]}]) of {fail,Reason} -> @@ -1067,7 +1183,7 @@ run_test_case_eval1(Mod, Func, Args, Name, RunInit, TCCallback) -> {{T,Return2},Loc,Opts} end. -%% the return value is a list and we have to check if it contains +%% the return value is a list and we have to check if it contains %% the result of an end conf case or if it's a Config list process_return_val([Return], M,F,A, Loc, Final) when is_list(Return) -> ReturnTags = [skip,skip_and_save,save_config,comment,return_group_result], @@ -1090,16 +1206,20 @@ process_return_val([Return], M,F,A, Loc, Final) when is_list(Return) -> process_return_val(Return, M,F,A, Loc, Final) -> process_return_val1(Return, M,F,A, Loc, Final, []). -process_return_val1([Failed={E,TCError}|_], M,F,A=[Args], Loc, _, SaveOpts) when E=='EXIT'; +process_return_val1([Failed={E,TCError}|_], M,F,A=[Args], Loc, _, SaveOpts) when E=='EXIT'; E==failed -> fw_error_notify(M,F,A, TCError, mod_loc(Loc)), - test_server_sup:framework_call(end_tc, - [?pl2a(M),F,{{error,TCError}, - [[{tc_status,{failed,TCError}}|Args]]}]), - {Failed,SaveOpts}; -process_return_val1([SaveCfg={save_config,_}|Opts], M,F,[Args], Loc, Final, SaveOpts) -> + case test_server_sup:framework_call(end_tc, + [?pl2a(M),F,{{error,TCError}, + [[{tc_status,{failed,TCError}}|Args]]}]) of + {fail,FWReason} -> + {{failed,FWReason},SaveOpts}; + _ -> + {Failed,SaveOpts} + end; +process_return_val1([SaveCfg={save_config,_}|Opts], M,F,[Args], Loc, Final, SaveOpts) -> process_return_val1(Opts, M,F,[[SaveCfg|Args]], Loc, Final, SaveOpts); -process_return_val1([{skip_and_save,Why,SaveCfg}|Opts], M,F,[Args], Loc, _, SaveOpts) -> +process_return_val1([{skip_and_save,Why,SaveCfg}|Opts], M,F,[Args], Loc, _, SaveOpts) -> process_return_val1(Opts, M,F,[[{save_config,SaveCfg}|Args]], Loc, {skip,Why}, SaveOpts); process_return_val1([GR={return_group_result,_}|Opts], M,F,A, Loc, Final, SaveOpts) -> process_return_val1(Opts, M,F,A, Loc, Final, [GR|SaveOpts]); @@ -1109,8 +1229,12 @@ process_return_val1([RetVal={Tag,_}|Opts], M,F,A, Loc, _, SaveOpts) when Tag==sk process_return_val1([_|Opts], M,F,A, Loc, Final, SaveOpts) -> process_return_val1(Opts, M,F,A, Loc, Final, SaveOpts); process_return_val1([], M,F,A, _Loc, Final, SaveOpts) -> - test_server_sup:framework_call(end_tc, [?pl2a(M),F,{Final,A}]), - {Final,lists:reverse(SaveOpts)}. + case test_server_sup:framework_call(end_tc, [?pl2a(M),F,{Final,A}]) of + {fail,FWReason} -> + {{failed,FWReason},SaveOpts}; + _ -> + {Final,lists:reverse(SaveOpts)} + end. user_callback(undefined, _, _, _, Args) -> Args; @@ -1138,7 +1262,7 @@ init_per_testcase(Mod, Func, Args) -> case erlang:function_exported(Mod,init_per_testcase,2) of true -> case catch my_apply(Mod, init_per_testcase, [Func|Args]) of - {'$test_server_ok',{Skip,Reason}} when Skip==skip; + {'$test_server_ok',{Skip,Reason}} when Skip==skip; Skip==skipped -> {skip,Reason}; {'$test_server_ok',Res={skip_and_save,_,_}} -> @@ -1149,31 +1273,31 @@ init_per_testcase(Mod, Func, Args) -> [] -> {ok,NewConf}; Bad -> - group_leader() ! {printout,12, + group_leader() ! {printout,12, "ERROR! init_per_testcase has returned " - "bad elements in Config: ~p\n",[Bad]}, + "bad elements in Config: ~p\n",[Bad]}, {skip,{failed,{Mod,init_per_testcase,bad_return}}} end; {'$test_server_ok',_Other} -> - group_leader() ! {printout,12, + group_leader() ! {printout,12, "ERROR! init_per_testcase did not return " - "a Config list.\n",[]}, + "a Config list.\n",[]}, {skip,{failed,{Mod,init_per_testcase,bad_return}}}; {'EXIT',Reason} -> Line = get_loc(), FormattedLoc = test_server_sup:format_loc(mod_loc(Line)), - group_leader() ! {printout,12, + group_leader() ! {printout,12, "ERROR! init_per_testcase crashed!\n" "\tLocation: ~s\n\tReason: ~p\n", - [FormattedLoc,Reason]}, + [FormattedLoc,Reason]}, {skip,{failed,{Mod,init_per_testcase,Reason}}}; Other -> Line = get_loc(), FormattedLoc = test_server_sup:format_loc(mod_loc(Line)), - group_leader() ! {printout,12, + group_leader() ! {printout,12, "ERROR! init_per_testcase thrown!\n" "\tLocation: ~s\n\tReason: ~p\n", - [FormattedLoc, Other]}, + [FormattedLoc, Other]}, {skip,{failed,{Mod,init_per_testcase,Other}}} end; false -> @@ -1182,7 +1306,7 @@ init_per_testcase(Mod, Func, Args) -> [Config] = Args, {ok, Config} end. - + end_per_testcase(Mod, Func, Conf) -> case erlang:function_exported(Mod,end_per_testcase,2) of true -> @@ -1211,11 +1335,11 @@ do_end_per_testcase(Mod,EndFunc,Func,Conf) -> comment(io_lib:format("<font color=\"red\">" "WARNING: ~w crashed!" "</font>\n",[EndFunc])), - group_leader() ! {printout,12, + group_leader() ! {printout,12, "WARNING: ~w crashed!\n" "Reason: ~p\n" "Line: ~s\n", - [EndFunc, Reason, + [EndFunc, Reason, test_server_sup:format_loc( mod_loc(get_loc()))]}, {failed,{Mod,end_per_testcase,Why}}; @@ -1223,13 +1347,13 @@ do_end_per_testcase(Mod,EndFunc,Func,Conf) -> comment(io_lib:format("<font color=\"red\">" "WARNING: ~w thrown!" "</font>\n",[EndFunc])), - group_leader() ! {printout,12, + group_leader() ! {printout,12, "WARNING: ~w thrown!\n" "Reason: ~p\n" "Line: ~s\n", - [EndFunc, Other, + [EndFunc, Other, test_server_sup:format_loc( - mod_loc(get_loc()))]}, + mod_loc(get_loc()))]}, {failed,{Mod,end_per_testcase,Other}} end. @@ -1254,7 +1378,7 @@ get_mf(_) -> {undefined,undefined}. mod_loc(Loc) -> %% handle diff line num versions - case Loc of + case Loc of [{{_M,_F},_L}|_] -> [{?pl2a(M),F,L} || {{M,F},L} <- Loc]; [{_M,_F}|_] -> @@ -1286,7 +1410,7 @@ fw_error_notify(Mod, Func, Args, Error, Loc) -> %% Args = [term()] %% %% Just like io:format, except that depending on the Detail value, the output -%% is directed to console, major and/or minor log files. +%% is directed to console, major and/or minor log files. print(Detail,Format,Args) -> local_or_remote_apply({test_server_ctrl,print,[Detail,Format,Args]}). @@ -1296,11 +1420,11 @@ print(Detail,Format,Args) -> %% %% Prints Leader followed by a time stamp (date and time). Depending on %% the Detail value, the output is directed to console, major and/or minor -%% log files. +%% log files. print_timestamp(Detail,Leader) -> local_or_remote_apply({test_server_ctrl,print_timestamp,[Detail,Leader]}). - + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% lookup_config(Key,Config) -> {value,{Key,Value}} | undefined @@ -1326,11 +1450,11 @@ ts_tc(M, F, A) -> Val = (catch my_apply(M, F, A)), After = erlang:now(), Result = case Val of - {'$test_server_ok', R} -> + {'$test_server_ok', R} -> R; % test case ok - {'EXIT',_Reason} = R -> + {'EXIT',_Reason} = R -> R; % test case crashed - Other -> + Other -> {failed, {thrown,Other}} % test case was thrown end, Elapsed = @@ -1352,7 +1476,7 @@ my_apply(M, F, A) -> %% in an attempt to keep this modules small (yeah, right!) %% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% unicode_to_latin1(Chars) when is_list(Chars); is_binary(Chars) -> - lists:flatten( + lists:flatten( [ case X of High when High > 255 -> io_lib:format("\\{~.8B}",[X]); @@ -1460,6 +1584,44 @@ sleep(MSecs) -> ok. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% adjusted_sleep(Time) -> ok +%% Time = integer() | float() | infinity +%% +%% Sleeps the specified number of milliseconds, multiplied by the +%% 'multiply_timetraps' value (if set) and possibly also automatically scaled +%% up if 'scale_timetraps' is set to true (which is default). +%% This function also accepts floating point numbers (which are truncated) and +%% the atom 'infinity'. +adjusted_sleep(infinity) -> + receive + after infinity -> + ok + end; +adjusted_sleep(MSecs) -> + {Multiplier,ScaleFactor} = + case test_server_ctrl:get_timetrap_parameters() of + {undefined,undefined} -> + {1,1}; + {undefined,false} -> + {1,1}; + {undefined,true} -> + {1,timetrap_scale_factor()}; + {infinity,_} -> + {infinity,1}; + {Mult,undefined} -> + {Mult,1}; + {Mult,false} -> + {Mult,1}; + {Mult,true} -> + {Mult,timetrap_scale_factor()} + end, + receive + after trunc(MSecs*Multiplier*ScaleFactor) -> + ok + end, + ok. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% fail(Reason) -> exit({suite_failed,Reason}) %% %% Immediately calls exit. Included because test suites are easier @@ -1509,9 +1671,9 @@ break(Comment) -> receive continue -> ok end. spawn_break_process(Pid) -> - spawn(fun() -> + spawn(fun() -> register(test_server_break_process,self()), - receive + receive continue -> continue(Pid); cancel -> ok end @@ -1561,20 +1723,21 @@ timetrap_scale_factor() -> %% timetrap(Timeout) -> Handle %% Handle = term() %% -%% Creates a time trap, that will kill the calling process if the +%% Creates a time trap, that will kill the calling process if the %% trap is not cancelled with timetrap_cancel/1, within Timeout milliseconds. - timetrap(Timeout0) -> Timeout = time_ms(Timeout0), cancel_default_timetrap(), case get(test_server_multiply_timetraps) of - undefined -> timetrap1(Timeout); - infinity -> infinity; - Int -> timetrap1(Timeout*Int) + undefined -> timetrap1(Timeout, true); + {undefined,false} -> timetrap1(Timeout, false); + {undefined,_} -> timetrap1(Timeout, true); + {infinity,_} -> infinity; + {Int,Scale} -> timetrap1(Timeout*Int, Scale) end. -timetrap1(Timeout) -> - Ref = spawn_link(test_server_sup,timetrap,[Timeout,self()]), +timetrap1(Timeout, Scale) -> + Ref = spawn_link(test_server_sup,timetrap,[Timeout,Scale,self()]), case get(test_server_timetraps) of undefined -> put(test_server_timetraps,[Ref]); List -> put(test_server_timetraps,[Ref|List]) @@ -1582,7 +1745,6 @@ timetrap1(Timeout) -> Ref. ensure_timetrap(Config) -> - %format("ensure_timetrap:~p~n",[Config]), case get(test_server_timetraps) of [_|_] -> ok; @@ -1623,7 +1785,7 @@ cancel_default_timetrap() -> time_ms({hours,N}) -> hours(N); time_ms({minutes,N}) -> minutes(N); time_ms({seconds,N}) -> seconds(N); -time_ms({Other,_N}) -> +time_ms({Other,_N}) -> format("=== ERROR: Invalid time specification: ~p. " "Should be seconds, minutes, or hours.~n", [Other]), exit({invalid_time_spec,Other}); @@ -1770,14 +1932,14 @@ call_crash(Time,Crash,M,F,A) -> %% by the test server after completion of the test case %% Therefore it is IMPORTANT that the USER terminates %% the node!! -%% {erl, ReleaseList} - Use an Erlang emulator determined by ReleaseList -%% when starting nodes, instead of the same emulator +%% {erl, ReleaseList} - Use an Erlang emulator determined by ReleaseList +%% when starting nodes, instead of the same emulator %% as the test server is running. ReleaseList is a list -%% of specifiers, where a specifier is either -%% {release, Rel}, {prog, Prog}, or 'this'. Rel is -%% either the name of a release, e.g., "r7a" or -%% 'latest'. 'this' means using the same emulator as -%% the test server. Prog is the name of an emulator +%% of specifiers, where a specifier is either +%% {release, Rel}, {prog, Prog}, or 'this'. Rel is +%% either the name of a release, e.g., "r7a" or +%% 'latest'. 'this' means using the same emulator as +%% the test server. Prog is the name of an emulator %% executable. If the list has more than one element, %% one of them is picked randomly. (Only %% works on Solaris and Linux, and the test @@ -1792,13 +1954,13 @@ call_crash(Time,Crash,M,F,A) -> %% peer nodes. %% Note that slave nodes always act as if they had %% fail_on_error==false. -%% +%% start_node(Name, Type, Options) -> lists:foreach( - fun(N) -> + fun(N) -> case firstname(N) of - Name -> + Name -> format("=== WARNING: Trying to start node \'~w\' when node" " with same first name exists: ~w", [Name, N]); _other -> ok @@ -1817,19 +1979,19 @@ start_node(Name, Type, Options) -> %% Cannot run cover on shielded node or on a node started %% by a shielded node. Cover = case is_cover() of - true -> + true -> not is_shielded(Name) andalso same_version(Node); - false -> + false -> false end, net_adm:ping(Node), case Cover of - true -> + true -> Sticky = unstick_all_sticky(Node), cover:start(Node), stick_all_sticky(Node,Sticky); - _ -> + _ -> ok end, {ok,Node}; @@ -1857,7 +2019,7 @@ wait_for_node(Slave) -> self(), {test_server_ctrl,wait_for_node,[Slave]}}, receive {sync_result,R} -> R end. - + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% stop_node(Name) -> true|false @@ -1867,7 +2029,7 @@ wait_for_node(Slave) -> stop_node(Slave) -> Nocover = is_shielded(Slave) orelse not same_version(Slave), case is_cover() of - true when not Nocover -> + true when not Nocover -> Sticky = unstick_all_sticky(Slave), cover:stop(Slave), stick_all_sticky(Slave,Sticky); @@ -1895,10 +2057,10 @@ stop_node(Slave) -> %% with the {cleanup,false} option, or it was started %% in some other way than test_server:start_node/3 format("=== WARNING: Attempt to stop a nonexisting slavenode (~p)~n" - "=== Trying to kill it anyway!!!", + "=== Trying to kill it anyway!!!", [Slave]), case net_adm:ping(Slave)of - pong -> + pong -> slave:stop(Slave), true; pang -> @@ -1918,7 +2080,7 @@ is_release_available(Release) -> self(), {test_server_ctrl,is_release_available,[Release]}}, receive {sync_result,R} -> R end. - + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% run_on_shielded_node(Fun, CArgs) -> term() @@ -1937,7 +2099,7 @@ is_release_available(Release) -> %% %% Fun - Function to execute %% CArg - Extra command line arguments to use when starting -%% the shielded node. +%% the shielded node. %% %% If Fun is successfully executed, the result is returned. %% @@ -2037,8 +2199,8 @@ is_native(Mod) -> %% The given String will occur in the comment field %% of the table on the test suite result page. If %% called several times, only the last comment is -%% printed. -%% comment/1 is also overwritten by the return value +%% printed. +%% comment/1 is also overwritten by the return value %% {comment,Comment} or fail/1 (which prints Reason %% as a comment). comment(String) -> @@ -2154,7 +2316,7 @@ purify_new_fds_inuse() -> {'EXIT', _} -> false; Inuse when is_integer(Inuse) -> Inuse end. - + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% purify_format(Format, Args) -> ok %% Format = string() @@ -2202,9 +2364,9 @@ local_or_remote_apply({M,F,A} = MFA) -> request(Sock,Request) -> gen_tcp:send(Sock,<<1,(term_to_binary(Request))/binary>>). -%% +%% %% Generic receive function for communication with host -%% +%% recv(Sock) -> case gen_tcp:recv(Sock,0) of {error,closed} -> diff --git a/lib/test_server/src/test_server_ctrl.erl b/lib/test_server/src/test_server_ctrl.erl index 4cb5863955..72f274d63a 100644 --- a/lib/test_server/src/test_server_ctrl.erl +++ b/lib/test_server/src/test_server_ctrl.erl @@ -27,7 +27,7 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% %% MODULE DEPENDENCIES: -%% HARD TO REMOVE: erlang, lists, io_lib, gen_server, file, io, string, +%% HARD TO REMOVE: erlang, lists, io_lib, gen_server, file, io, string, %% code, ets, rpc, gen_tcp, inet, erl_tar, sets, %% test_server, test_server_sup, test_server_node %% EASIER TO REMOVE: filename, filelib, lib, re @@ -36,7 +36,7 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% ARCHITECTURE -%% +%% %% The Erlang Test Server can be run on the target machine (local target) %% or towards a remote target. The execution flow is mainly the same in %% both cases, but with a remote target the test cases are (obviously) @@ -44,11 +44,11 @@ %% socket connections because the host should not be introduced as an %% additional node in the distributed erlang system in which the test %% cases are run. -%% -%% +%% +%% %% Local Target: %% ============= -%% +%% %% ----- %% | | test_server_ctrl ({global,test_server}) %% ----- (test_server_ctrl.erl) @@ -62,33 +62,33 @@ %% ----- %% | | CaseProc %% ----- (test_server.erl) -%% -%% -%% +%% +%% +%% %% test_server_ctrl is the main process in the system. It is a registered %% process, and it will always be alive when testing is ongoing. %% test_server_ctrl initiates testing and monitors JobProc(s). -%% -%% When target is local, and Test Server is *not* being used by a framework -%% application (where it might cause duplicate name problems in a distributed -%% test environment), the process is globally registered as 'test_server' +%% +%% When target is local, and Test Server is *not* being used by a framework +%% application (where it might cause duplicate name problems in a distributed +%% test environment), the process is globally registered as 'test_server' %% to be able to simulate the {global,test_server} process on a remote target. -%% -%% JobProc is spawned for each 'job' added to the test_server_ctrl. +%% +%% JobProc is spawned for each 'job' added to the test_server_ctrl. %% A job can mean one test case, one test suite or one spec. %% JobProc creates and writes logs and presents results from testing. %% JobProc is the group leader for CaseProc. -%% +%% %% CaseProc is spawned for each test case. It runs the test case and %% sends results and any other information to its group leader - JobProc. -%% -%% -%% +%% +%% +%% %% Remote Target: %% ============== -%% +%% %% HOST TARGET -%% +%% %% ----- MainSock ----- %% test_server_ctrl | |- - - - - - -| | {global,test_server} %% (test_server_ctrl.erl) ----- ----- (test_server.erl) @@ -102,36 +102,36 @@ %% ----- %% | | CaseProc %% ----- (test_server.erl) -%% -%% -%% -%% +%% +%% +%% +%% %% A separate test_server process only exists when target is remote. It %% is then the main process on target. It is started when test_server_ctrl %% is started, and a socket connection is established between %% test_server_ctrl and test_server. The following information can be sent %% over MainSock: -%% +%% %% HOST TARGET %% -> {target_info, TargetInfo} (during initiation) %% <- {job_proc_killed,Name,Reason} (if a JobProcT dies unexpectedly) %% -> {job,Port,Name} (to start a new JobProcT) -%% -%% +%% +%% %% When target is remote, JobProc is split into to processes: JobProcH %% executing on Host and JobProcT executing on Target. (The two processes %% execute the same code as JobProc does when target is local.) JobProcH %% and JobProcT communicates over a socket connection. The following %% information can be sent over JobSock: -%% +%% %% HOST TARGET %% -> {test_case, Case} To start a new test case %% -> {beam,Mod} .beam file as binary to be loaded %% on target, e.g. a test suite %% -> {datadir,Tarfile} Content of the datadir for a test suite %% <- {apply,MFA} MFA to be applied on host, ignore return; -%% (apply is used for printing information in -%% log or console) +%% (apply is used for printing information in +%% log or console) %% <- {sync_apply,MFA} MFA to be applied on host, wait for return %% (used for starting and stopping slave nodes) %% -> {sync_apply,MFA} MFA to be applied on target, wait for return @@ -141,7 +141,7 @@ %% <- {crash_dumps,Tarfile} When a test case is finished %% -> job_done When a job is finished %% <- {privdir,Privdir} When a job is finished -%% +%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -151,17 +151,19 @@ %%% OPERATOR INTERFACE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -export([add_spec/1, add_dir/2, add_dir/3]). --export([add_module/1, add_module/2, add_case/2, add_case/3, add_cases/2, - add_cases/3]). +-export([add_module/1, add_module/2, + add_conf/3, + add_case/2, add_case/3, add_cases/2, add_cases/3]). -export([add_dir_with_skip/3, add_dir_with_skip/4, add_tests_with_skip/3]). -export([add_module_with_skip/2, add_module_with_skip/3, + add_conf_with_skip/4, add_case_with_skip/3, add_case_with_skip/4, add_cases_with_skip/3, add_cases_with_skip/4]). -export([jobs/0, run_test/1, wait_finish/0, idle_notify/1, abort_current_testcase/1, abort/0]). -export([start_get_totals/1, stop_get_totals/0]). -export([get_levels/0, set_levels/3]). --export([multiply_timetraps/1]). +-export([multiply_timetraps/1, scale_timetraps/1, get_timetrap_parameters/0]). -export([cover/2, cover/3, cover/7, cross_cover_analyse/1, cross_cover_analyse/2, trc/1, stop_trace/0]). -export([testcase_callback/1]). @@ -207,13 +209,15 @@ -define(pl2a(M), test_server_sup:package_atom(M)). -define(void_fun, fun() -> ok end). --define(mod_result(X), if X == skip -> skipped; - X == auto_skip -> skipped; +-define(mod_result(X), if X == skip -> skipped; + X == auto_skip -> skipped; true -> X end). --record(state,{jobs=[],levels={1,19,10},multiply_timetraps=1,finish=false, +-record(state,{jobs=[],levels={1,19,10}, + multiply_timetraps=1,scale_timetraps=true, + finish=false, target_info, trc=false, cover=false, wait_for_node=[], - testcase_callback=undefined, idle_notify=[], + testcase_callback=undefined, idle_notify=[], get_totals=false, random_seed=undefined}). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -222,21 +226,28 @@ add_dir(Name, Job=[Dir|_Dirs]) when is_list(Dir) -> add_job(cast_to_list(Name), lists:map(fun(D)-> {dir,cast_to_list(D)} end, Job)); -add_dir(Name, Dir) -> +add_dir(Name, Dir) -> add_job(cast_to_list(Name), {dir,cast_to_list(Dir)}). add_dir(Name, Job=[Dir|_Dirs], Pattern) when is_list(Dir) -> add_job(cast_to_list(Name), lists:map(fun(D)-> {dir,cast_to_list(D), cast_to_list(Pattern)} end, Job)); -add_dir(Name, Dir, Pattern) -> +add_dir(Name, Dir, Pattern) -> add_job(cast_to_list(Name), {dir,cast_to_list(Dir),cast_to_list(Pattern)}). add_module(Mod) when is_atom(Mod) -> add_job(atom_to_list(Mod), {Mod,all}). + add_module(Name, Mods) when is_list(Mods) -> add_job(cast_to_list(Name), lists:map(fun(Mod) -> {Mod,all} end, Mods)). +add_conf(Name, Mod, Conf) when is_tuple(Conf) -> + add_job(cast_to_list(Name), {Mod,[Conf]}); + +add_conf(Name, Mod, Confs) when is_list(Confs) -> + add_job(cast_to_list(Name), {Mod,Confs}). + add_case(Mod, Case) when is_atom(Mod), is_atom(Case) -> add_job(atom_to_list(Mod), {Mod,Case}). @@ -256,14 +267,14 @@ add_spec(Spec) -> false -> {error,nofile} end. -%% This version of the interface is to be used if there are +%% This version of the interface is to be used if there are %% suites or cases that should be skipped. add_dir_with_skip(Name, Job=[Dir|_Dirs], Skip) when is_list(Dir) -> add_job(cast_to_list(Name), lists:map(fun(D)-> {dir,cast_to_list(D)} end, Job), Skip); -add_dir_with_skip(Name, Dir, Skip) -> +add_dir_with_skip(Name, Dir, Skip) -> add_job(cast_to_list(Name), {dir,cast_to_list(Dir)}, Skip). add_dir_with_skip(Name, Job=[Dir|_Dirs], Pattern, Skip) when is_list(Dir) -> @@ -271,7 +282,7 @@ add_dir_with_skip(Name, Job=[Dir|_Dirs], Pattern, Skip) when is_list(Dir) -> lists:map(fun(D)-> {dir,cast_to_list(D), cast_to_list(Pattern)} end, Job), Skip); -add_dir_with_skip(Name, Dir, Pattern, Skip) -> +add_dir_with_skip(Name, Dir, Pattern, Skip) -> add_job(cast_to_list(Name), {dir,cast_to_list(Dir),cast_to_list(Pattern)}, Skip). @@ -281,6 +292,12 @@ add_module_with_skip(Mod, Skip) when is_atom(Mod) -> add_module_with_skip(Name, Mods, Skip) when is_list(Mods) -> add_job(cast_to_list(Name), lists:map(fun(Mod) -> {Mod,all} end, Mods), Skip). +add_conf_with_skip(Name, Mod, Conf, Skip) when is_tuple(Conf) -> + add_job(cast_to_list(Name), {Mod,[Conf]}, Skip); + +add_conf_with_skip(Name, Mod, Confs, Skip) when is_list(Confs) -> + add_job(cast_to_list(Name), {Mod,Confs}, Skip). + add_case_with_skip(Mod, Case, Skip) when is_atom(Mod), is_atom(Case) -> add_job(atom_to_list(Mod), {Mod,Case}, Skip). @@ -295,15 +312,14 @@ add_cases_with_skip(Name, Mod, Cases, Skip) when is_atom(Mod), is_list(Cases) -> add_tests_with_skip(LogDir, Tests, Skip) -> add_job(LogDir, - lists:map(fun({Dir,all,all}) -> + lists:map(fun({Dir,all,all}) -> {Dir,{dir,Dir}}; - ({Dir,Mods,all}) -> + ({Dir,Mods,all}) -> {Dir,lists:map(fun(M) -> {M,all} end, Mods)}; ({Dir,Mod,Cases}) -> {Dir,{Mod,Cases}} end, Tests), Skip). - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% COMMAND LINE INTERFACE @@ -315,7 +331,7 @@ parse_cmd_line(['SPEC',Spec|Cmds], SpecList, Names, Param, Trc, Cov, TCCB) -> case file:consult(Spec) of {ok, TermList} -> Name = filename:rootname(Spec), - parse_cmd_line(Cmds, TermList++SpecList, [Name|Names], Param, + parse_cmd_line(Cmds, TermList++SpecList, [Name|Names], Param, Trc, Cov, TCCB); {error,Reason} -> io:format("Can't open ~s: ~p\n", @@ -406,7 +422,7 @@ run_test(CommandLine) -> end, testcase_callback(TCCB), add_job(Name, {command_line,SpecList}), - + %% adding of jobs involves file i/o which may take long time %% when running a nfs mounted file system (VxWorks). case controller_call(get_target_info) of @@ -479,6 +495,12 @@ set_levels(Show, Major, Minor) -> multiply_timetraps(N) -> controller_call({multiply_timetraps,N}). +scale_timetraps(Bool) -> + controller_call({scale_timetraps,Bool}). + +get_timetrap_parameters() -> + controller_call(get_timetrap_parameters). + trc(TraceFile) -> controller_call({trace,TraceFile}, 2*?ACCEPT_TIMEOUT). @@ -551,7 +573,7 @@ controller_call(Arg, Timeout) -> Other -> Other end. - + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -617,7 +639,7 @@ contact_main_target(local) -> case os:getenv("TEST_SERVER_FRAMEWORK") of false -> %% Local target! The global test_server process implemented by - %% test_server.erl will not be started, so we simulate it by + %% test_server.erl will not be started, so we simulate it by %% globally registering this process instead. global:sync(), case global:whereis_name(test_server) of @@ -681,9 +703,9 @@ read_parameters([], Par) when Par#par.type==undefined -> read_parameters([], Par) when Par#par.target==undefined -> {error, {missing_mandatory_parameter,target}}; read_parameters([], Par0) -> - Par = + Par = case {Par0#par.type, Par0#par.master} of - {ose, undefined} -> + {ose, undefined} -> %% Use this node as master and bootserver for target %% and slave nodes Par0#par{master = atom_to_list(node()), @@ -691,10 +713,10 @@ read_parameters([], Par0) -> {ose, _Master} -> %% Master for target and slave nodes was defined in parameterfile Par0; - _ -> + _ -> %% Use target as master for slave nodes, %% (No master is used for target) - Par0#par{master="test_server@" ++ Par0#par.target} + Par0#par{master="test_server@" ++ Par0#par.target} end, {ok,Par}. @@ -708,7 +730,7 @@ naming() -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% handle_call(kill_slavenodes, From, State) -> ok %% -%% Kill all slave nodes that remain after a test case +%% Kill all slave nodes that remain after a test case %% is completed. %% handle_call(kill_slavenodes, _From, State) -> @@ -736,7 +758,7 @@ handle_call(get_hosts, _From, State) -> {reply, Hosts, State}; %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% handle_call({add_job,Dir,Name,TopCase,Skip}, _, State) -> +%% handle_call({add_job,Dir,Name,TopCase,Skip}, _, State) -> %% ok | {error,Reason} %% %% Dir = string() @@ -760,7 +782,7 @@ handle_call(get_hosts, _From, State) -> handle_call({add_job,Dir,Name,TopCase,Skip}, _From, State) -> LogDir = Dir ++ ?logdir_ext, - ExtraTools = + ExtraTools = case State#state.cover of false -> []; {App,Analyse} -> [{cover,App,Analyse}] @@ -776,19 +798,21 @@ handle_call({add_job,Dir,Name,TopCase,Skip}, _From, State) -> {spec,SpecName} -> Pid = spawn_tester( ?MODULE, do_spec, - [SpecName,State#state.multiply_timetraps], - LogDir, Name, State#state.levels, + [SpecName,{State#state.multiply_timetraps, + State#state.scale_timetraps}], + LogDir, Name, State#state.levels, State#state.testcase_callback, ExtraTools1), NewJobs = [{Name,Pid}|State#state.jobs], - {reply, ok, State#state{jobs=NewJobs}}; + {reply, ok, State#state{jobs=NewJobs}}; {command_line,SpecList} -> Pid = spawn_tester( ?MODULE, do_spec_list, - [SpecList,State#state.multiply_timetraps], - LogDir, Name, State#state.levels, + [SpecList,{State#state.multiply_timetraps, + State#state.scale_timetraps}], + LogDir, Name, State#state.levels, State#state.testcase_callback, ExtraTools1), NewJobs = [{Name,Pid}|State#state.jobs], - {reply, ok, State#state{jobs=NewJobs}}; + {reply, ok, State#state{jobs=NewJobs}}; TopCase -> case State#state.get_totals of {CliPid,Fun} -> @@ -798,10 +822,11 @@ handle_call({add_job,Dir,Name,TopCase,Skip}, _From, State) -> _ -> Cfg = make_config([]), Pid = spawn_tester( - ?MODULE, do_test_cases, + ?MODULE, do_test_cases, [TopCase,Skip,Cfg, - State#state.multiply_timetraps], - LogDir, Name, State#state.levels, + {State#state.multiply_timetraps, + State#state.scale_timetraps}], + LogDir, Name, State#state.levels, State#state.testcase_callback, ExtraTools1), NewJobs = [{Name,Pid}|State#state.jobs], {reply, ok, State#state{jobs=NewJobs}} @@ -827,7 +852,7 @@ handle_call(jobs, _From, State) -> %% handle_call({abort_current_testcase,Reason}, _, State) -> Result %% Reason = term() %% Result = ok | {error,no_testcase_running} -%% +%% %% Attempts to abort the test case that's currently running. handle_call({abort_current_testcase,Reason}, _From, State) -> @@ -855,7 +880,7 @@ handle_call({abort_current_testcase,Reason}, _From, State) -> handle_call({finish,Fini}, _From, State) -> case State#state.jobs of [] -> - lists:foreach(fun({Cli,Fun}) -> Fun(Cli) end, + lists:foreach(fun({Cli,Fun}) -> Fun(Cli) end, State#state.idle_notify), State2 = State#state{finish=false}, {stop,shutdown,{ok,self()}, State2}; @@ -878,7 +903,7 @@ handle_call({idle_notify,Fun}, {Cli,_Ref}, State) -> {reply, {ok,self()}, State}; _ -> Subscribed = State#state.idle_notify, - {reply, {ok,self()}, + {reply, {ok,self()}, State#state{idle_notify=[{Cli,Fun}|Subscribed]}} end; @@ -891,7 +916,7 @@ handle_call({idle_notify,Fun}, {Cli,_Ref}, State) -> handle_call({start_get_totals,Fun}, {Cli,_Ref}, State) -> {reply, {ok,self()}, State#state{get_totals={Cli,Fun}}}; - + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% handle_call(stop_get_totals, From, State) -> ok %% @@ -942,11 +967,31 @@ handle_call({multiply_timetraps,N}, _From, State) -> {reply,ok,State#state{multiply_timetraps=N}}; %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% handle_call({scale_timetraps,Bool}, _, State) -> ok +%% Bool = true | false +%% +%% Specifies if test_server should scale the timetrap value +%% automatically if e.g. cover is running. + +handle_call({scale_timetraps,Bool}, _From, State) -> + {reply,ok,State#state{scale_timetraps=Bool}}; + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%% handle_call(get_timetrap_parameters, _, State) -> {Multiplier,Scale} +%% Multiplier = integer() | infinity +%% Scale = true | false +%% +%% Returns the parameter values that affect timetraps. + +handle_call(get_timetrap_parameters, _From, State) -> + {reply,{State#state.multiply_timetraps,State#state.scale_timetraps},State}; + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% handle_call({trace,TraceFile}, _, State) -> ok | {error,Reason} %% -%% Starts a separate node (trace control node) which +%% Starts a separate node (trace control node) which %% starts tracing on target and all slave nodes -%% +%% %% TraceFile is a text file with elements of type %% {Trace,Mod,TracePattern}. %% {Trace,Mod,Func,TracePattern}. @@ -955,10 +1000,10 @@ handle_call({multiply_timetraps,N}, _From, State) -> %% Trace = tp | tpl; local or global call trace %% Mod,Func = atom(), Arity=integer(); defines what to trace %% TracePattern = [] | match_spec() -%% +%% %% The 'call' trace flag is set on all processes, and then %% the given trace patterns are set. - + handle_call({trace,TraceFile}, _From, State=#state{trc=false}) -> TI = State#state.target_info, case test_server_node:start_tracer_node(TraceFile, TI) of @@ -993,7 +1038,7 @@ handle_call({cover,App,Analyse}, _From, State) -> %% handle_call({testcase_callback,{Mod,Func}}, _, State) -> ok | {error,Reason} %% %% Add a callback function that will be called before and after every -%% test case (on the test case process): +%% test case (on the test case process): %% %% Mod:Func(Suite,TestCase,InitOrEnd,Config) %% @@ -1001,9 +1046,9 @@ handle_call({cover,App,Analyse}, _From, State) -> handle_call({testcase_callback,ModFunc}, _From, State) -> case ModFunc of - {Mod,Func} -> + {Mod,Func} -> case code:is_loaded(Mod) of - {file,_} -> + {file,_} -> ok; false -> code:load_file(Mod) @@ -1065,15 +1110,15 @@ handle_call({start_node, Name, Type, Options}, From, State) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% handle_call({wait_for_node,Node}, _, State) -> ok %% -%% Waits for a new node to take contact. Used if +%% Waits for a new node to take contact. Used if %% node is started with option {wait,false} handle_call({wait_for_node, Node}, From, State) -> - NewWaitList = + NewWaitList = case ets:lookup(slave_tab,Node) of - [] -> + [] -> [{Node,From}|State#state.wait_for_node]; - _ -> + _ -> gen_server:reply(From,ok), State#state.wait_for_node end, @@ -1086,7 +1131,7 @@ handle_call({wait_for_node, Node}, From, State) -> %% - the node is really stopped by test_server when this returns. handle_call({stop_node, Name}, _From, State) -> - R = test_server_node:stop_node(Name, State#state.target_info), + R = test_server_node:stop_node(Name, State#state.target_info), {reply, R, State}; %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1112,7 +1157,7 @@ handle_cast({node_started,Node}, State) -> false -> ok; Trc -> test_server_node:trace_nodes(Trc, [Node]) end, - NewWaitList = + NewWaitList = case lists:keysearch(Node,1,State#state.wait_for_node) of {value,{Node,From}} -> gen_server:reply(From, ok), @@ -1128,10 +1173,10 @@ handle_cast({node_started,Node}, State) -> %% Reason = term() %% %% Handles exit messages from linked processes. Only test suites and -%% possibly a target client are expected to be linked. +%% possibly a target client are expected to be linked. %% When a test suite terminates, it is removed from the job queue. %% If a target client terminates it means that we lost contact with -%% target. The test_server_ctrl process is terminated, and teminate/2 +%% target. The test_server_ctrl process is terminated, and teminate/2 %% will do the cleanup handle_info({'EXIT',Pid,Reason}, State) -> @@ -1139,7 +1184,7 @@ handle_info({'EXIT',Pid,Reason}, State) -> false -> TI = State#state.target_info, case TI#target_info.target_client of - Pid -> + Pid -> %% The target client died - lost contact with target {stop,{lost_contact_with_target,Reason},State}; _other -> @@ -1160,13 +1205,13 @@ handle_info({'EXIT',Pid,Reason}, State) -> State2 = State#state{jobs=NewJobs}, case NewJobs of [] -> - lists:foreach(fun({Cli,Fun}) -> Fun(Cli) end, + lists:foreach(fun({Cli,Fun}) -> Fun(Cli) end, State2#state.idle_notify), case State2#state.finish of false -> {noreply,State2#state{idle_notify=[]}}; _ -> % true | abort - %% test_server:finish() has been called and + %% test_server:finish() has been called and %% there are no jobs in the job queue => %% stop the test_server_ctrl {stop,shutdown,State2#state{finish=false}} @@ -1174,7 +1219,7 @@ handle_info({'EXIT',Pid,Reason}, State) -> _ -> % pending jobs case State2#state.finish of abort -> % abort test now! - lists:foreach(fun({Cli,Fun}) -> Fun(Cli) end, + lists:foreach(fun({Cli,Fun}) -> Fun(Cli) end, State2#state.idle_notify), {stop,shutdown,State2#state{finish=false}}; _ -> % true | false @@ -1194,9 +1239,9 @@ handle_info({tcp,_MainSock,<<1,Request/binary>>}, State) -> case binary_to_term(Request) of {job_proc_killed,Name,Reason} -> %% The only purpose of this is to inform the user about what - %% happened on target. + %% happened on target. %% The local job proc will soon be killed by the closed socket or - %% because the job is finished. Then the above clause ('EXIT') will + %% because the job is finished. Then the above clause ('EXIT') will %% handle the problem. io:format("Suite ~s was killed on remote target with reason" " ~p\n", [Name,Reason]); @@ -1204,13 +1249,13 @@ handle_info({tcp,_MainSock,<<1,Request/binary>>}, State) -> ignore end, {noreply,State}; - + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% handle_info({tcp_closed,Sock}, State) %% %% A Socket was closed. This indicates that a node died. -%% This can be +%% This can be %% *Target node (if remote) %% *Slave or peer node started by a test suite %% *Trace controll node @@ -1221,10 +1266,10 @@ handle_info({tcp_closed,Sock}, State=#state{trc=Sock}) -> {noreply,State#state{trc=false}}; handle_info({tcp_closed,Sock}, State) -> case test_server_node:nodedown(Sock,State#state.target_info) of - target_died -> + target_died -> %% terminate/2 will do the cleanup {stop,target_died,State}; - _ -> + _ -> {noreply,State} end; @@ -1260,7 +1305,7 @@ kill_all_jobs([]) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% spawn_tester(Mod, Func, Args, Dir, Name, Levels, +%% spawn_tester(Mod, Func, Args, Dir, Name, Levels, %% TestCaseCallback, ExtraTools) -> Pid %% Mod = atom() %% Func = atom() @@ -1268,23 +1313,23 @@ kill_all_jobs([]) -> %% Dir = string() %% Name = string() %% Levels = {integer(),integer(),integer()} -%% TestCaseCallback = {CBMod,CBFunc} | undefined +%% TestCaseCallback = {CBMod,CBFunc} | undefined %% ExtraTools = [ExtraTool,...] %% ExtraTool = CoverInfo | TraceInfo | RandomSeed %% %% Spawns a test suite execute-process, just an ordinary spawn, except %% that it will set a lot of dictionary information before starting the %% named function. Also, the execution is timed and protected by a catch. -%% When the named function is done executing, a summary of the results +%% When the named function is done executing, a summary of the results %% is printed to the log files. spawn_tester(Mod, Func, Args, Dir, Name, Levels, TCCallback, ExtraTools) -> spawn_link( - fun() -> init_tester(Mod, Func, Args, Dir, Name, Levels, - TCCallback, ExtraTools) + fun() -> init_tester(Mod, Func, Args, Dir, Name, Levels, + TCCallback, ExtraTools) end). -init_tester(Mod, Func, Args, Dir, Name, {SumLev,MajLev,MinLev}, +init_tester(Mod, Func, Args, Dir, Name, {SumLev,MajLev,MinLev}, TCCallback, ExtraTools) -> process_flag(trap_exit, true), put(test_server_name, Name), @@ -1324,7 +1369,7 @@ init_tester(Mod, Func, Args, Dir, Name, {SumLev,MajLev,MinLev}, {Skipped,_} -> {Skipped,io_lib:format(", ~p Skipped", [Skipped])} end, OkN = get(test_server_ok), - FailedN = get(test_server_failed), + FailedN = get(test_server_failed), print(html,"<tr><td></td><td><b>TOTAL</b></td><td></td><td></td>" "<td>~.3fs</td><td><b>~s</b></td><td>~p Ok, ~p Failed~s of ~p</td></tr>\n", [Time,SuccessStr,OkN,FailedN,SkipStr,OkN+FailedN+SkippedN]). @@ -1338,9 +1383,9 @@ ts_tc(M, F, A) -> {Elapsed,Val}. elapsed_time(Before, After) -> - (element(1,After)*1000000000000 + + (element(1,After)*1000000000000 + element(2,After)*1000000 + element(3,After)) - - (element(1,Before)*1000000000000 + + (element(1,Before)*1000000000000 + element(2,Before)*1000000 + element(3,Before)). start_extra_tools(ExtraTools) -> @@ -1378,28 +1423,32 @@ stop_extra_tools([], _) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% do_spec(SpecName, MultiplyTimetrap) -> {error,Reason} | exit(Result) +%% do_spec(SpecName, TimetrapSpec) -> {error,Reason} | exit(Result) %% SpecName = string() +%% TimetrapSpec = MultiplyTimetrap | {MultiplyTimetrap,ScaleTimetrap} %% MultiplyTimetrap = integer() | infinity +%% ScaleTimetrap = bool() %% %% Reads the named test suite specification file, and executes it. %% -%% This function is meant to be called by a process created by +%% This function is meant to be called by a process created by %% spawn_tester/7, which sets up some necessary dictionary values. -do_spec(SpecName, MultiplyTimetrap) when is_list(SpecName) -> +do_spec(SpecName, TimetrapSpec) when is_list(SpecName) -> case file:consult(SpecName) of {ok,TermList} -> - do_spec_list(TermList,MultiplyTimetrap); + do_spec_list(TermList,TimetrapSpec); {error,Reason} -> io:format("Can't open ~s: ~p\n", [SpecName,Reason]), {error,{cant_open_spec,Reason}} end. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% do_spec_list(TermList) -> exit(Result) +%% do_spec_list(TermList, TimetrapSpec) -> exit(Result) %% TermList = [term()|...] +%% TimetrapSpec = MultiplyTimetrap | {MultiplyTimetrap,ScaleTimetrap} %% MultiplyTimetrap = integer() | infinity +%% ScaleTimetrap = bool() %% %% Executes a list of test suite specification commands. The following %% commands are available, and may occur zero or more times (if several, @@ -1422,21 +1471,21 @@ do_spec(SpecName, MultiplyTimetrap) when is_list(SpecName) -> %% nodenames will be generated from the local host. %% %% {hosts, Hosts} Specifies a list of available hosts on which to start -%% slave nodes. It is used when the {remote, true} option is given to the +%% slave nodes. It is used when the {remote, true} option is given to the %% test_server:start_node/3 function. Also, if {require_nodenames, Num} is -%% contained in the TermList, the generated nodenames will be spread over +%% contained in the TermList, the generated nodenames will be spread over %% all hosts given in this Hosts list. The hostnames are given as atoms or %% strings. -%% +%% %% {diskless, true}</c></tag> is kept for backwards compatiblilty and %% should not be used. Use a configuration test case instead. -%% -%% This function is meant to be called by a process created by +%% +%% This function is meant to be called by a process created by %% spawn_tester/7, which sets up some necessary dictionary values. -do_spec_list(TermList0, MultiplyTimetrap) -> +do_spec_list(TermList0, TimetrapSpec) -> Nodes = [], - TermList = + TermList = case lists:keysearch(hosts, 1, TermList0) of {value, {hosts, Hosts0}} -> Hosts = lists:map(fun(H) -> cast_to_list(H) end, Hosts0), @@ -1447,7 +1496,7 @@ do_spec_list(TermList0, MultiplyTimetrap) -> end, DefaultConfig = make_config([{nodes,Nodes}]), {TopCases,SkipList,Config} = do_spec_terms(TermList, [], [], DefaultConfig), - do_test_cases(TopCases, SkipList, Config, MultiplyTimetrap). + do_test_cases(TopCases, SkipList, Config, TimetrapSpec). do_spec_terms([], TopCases, SkipList, Config) -> {TopCases,SkipList,Config}; @@ -1470,21 +1519,21 @@ do_spec_terms([{default_timeout,Tmo}|Terms], TopCases, SkipList, Config) -> do_spec_terms([{require_nodenames,NumNames}|Terms], TopCases, SkipList, Config) -> NodeNames0=generate_nodenames(NumNames), NodeNames=lists:delete([], NodeNames0), - do_spec_terms(Terms, TopCases, SkipList, + do_spec_terms(Terms, TopCases, SkipList, update_config(Config, {nodenames,NodeNames})); do_spec_terms([Other|Terms], TopCases, SkipList, Config) -> io:format("** WARNING: Spec file contains unknown directive ~p\n", [Other]), do_spec_terms(Terms, TopCases, SkipList, Config). - + generate_nodenames(Num) -> Hosts = case controller_call(get_hosts) of - [] -> + [] -> TI = controller_call(get_target_info), [TI#target_info.host]; - List -> + List -> List end, generate_nodenames2(Num, Hosts, []). @@ -1511,25 +1560,25 @@ temp_nodename([Chr|Base], Acc) -> %% NoOfCases = integer() | unknown %% %% Counts the test cases that are about to run and returns that number. -%% If there's a conf group in TestSpec with a repeat property, the total number +%% If there's a conf group in TestSpec with a repeat property, the total number %% of cases can not be calculated and NoOfCases = unknown. count_test_cases(TopCases, SkipCases) when is_list(TopCases) -> case collect_all_cases(TopCases, SkipCases) of - {error,_} -> - error; + {error,_Why} = Error -> + Error; TestSpec -> {get_suites(TestSpec, []), case remove_conf(TestSpec) of {repeats,_} -> unknown; - TestSpec1 -> + TestSpec1 -> length(TestSpec1) end} end; count_test_cases(TopCase, SkipCases) -> count_test_cases([TopCase], SkipCases). - + remove_conf(Cases) -> remove_conf(Cases, [], false). @@ -1538,13 +1587,15 @@ remove_conf([{conf, _Ref, Props, _MF}|Cases], NoConf, Repeats) -> case get_repeat(Props) of undefined -> remove_conf(Cases, NoConf, Repeats); + {_RepType,1} -> + remove_conf(Cases, NoConf, Repeats); _ -> remove_conf(Cases, NoConf, true) end; remove_conf([{make,_Ref,_MF}|Cases], NoConf, Repeats) -> remove_conf(Cases, NoConf, Repeats); -remove_conf([{skip_case,{Type,_Ref,_MF,_Cmt}}|Cases], - NoConf, Repeats) when Type==conf; +remove_conf([{skip_case,{Type,_Ref,_MF,_Cmt}}|Cases], + NoConf, Repeats) when Type==conf; Type==make -> remove_conf(Cases, NoConf, Repeats); remove_conf([C|Cases], NoConf, Repeats) -> @@ -1582,22 +1633,30 @@ add_mod(Mod, Mods) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% do_test_cases(TopCases, SkipCases, Config, MultiplyTimetrap) -> +%% do_test_cases(TopCases, SkipCases, Config, TimetrapSpec) -> %% exit(Result) %% %% TopCases = term() (See collect_cases/3) %% SkipCases = term() (See collect_cases/3) %% Config = term() (See collect_cases/3) +%% TimetrapSpec = MultiplyTimetrap | {MultiplyTimetrap,ScaleTimetrap} %% MultiplyTimetrap = integer() | infinity +%% ScaleTimetrap = bool() %% %% Initializes and starts the test run, for "ordinary" test suites. %% Creates log directories and log files, inserts initial timestamps and %% configuration information into the log files. %% -%% This function is meant to be called by a process created by +%% This function is meant to be called by a process created by %% spawn_tester/7, which sets up some necessary dictionary values. - -do_test_cases(TopCases, SkipCases, Config, MultiplyTimetrap) when is_list(TopCases) -> +do_test_cases(TopCases, SkipCases, + Config, MultiplyTimetrap) when is_integer(MultiplyTimetrap); + MultiplyTimetrap == infinity -> + do_test_cases(TopCases, SkipCases, Config, {MultiplyTimetrap,true}); + +do_test_cases(TopCases, SkipCases, + Config, TimetrapData) when is_list(TopCases), + is_tuple(TimetrapData) -> start_log_file(), case collect_all_cases(TopCases, SkipCases) of {error,Why} -> @@ -1607,27 +1666,36 @@ do_test_cases(TopCases, SkipCases, Config, MultiplyTimetrap) when is_list(TopCas N = case remove_conf(TestSpec0) of {repeats,_} -> unknown; TS -> length(TS) - end, + end, put(test_server_cases, N), put(test_server_case_num, 0), - TestSpec = + TestSpec = add_init_and_end_per_suite(TestSpec0, undefined, undefined), + TI = get_target_info(), print(1, "Starting test~s", [print_if_known(N, {", ~w test cases",[N]}, {" (with repeated test cases)",[]})]), - test_server_sup:framework_call(report, [tests_start, - {get(test_server_name),N}]), - print(html, - "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n" - "<!-- autogenerated by '"++atom_to_list(?MODULE)++"'. -->\n" - "<html>\n" - "<head><title>Test ~p results</title>\n" - "<meta http-equiv=\"cache-control\" content=\"no-cache\">\n" - "</head>\n" - "<body bgcolor=\"white\" text=\"black\" " - "link=\"blue\" vlink=\"purple\" alink=\"red\">" - "<h2>Results from test ~p</h2>\n", - [get(test_server_name),get(test_server_name)]), + Test = get(test_server_name), + test_server_sup:framework_call(report, [tests_start,{Test,N}]), + + Header = + case test_server_sup:framework_call(overview_html_header, [Test], "") of + "" -> + TestName = lists:flatten(io_lib:format("~p", [Test])), + ["<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n", + "<!-- autogenerated by '", atom_to_list(?MODULE), "'. -->\n", + "<html>\n", + "<head><title>Test ", TestName, " results</title>\n", + "<meta http-equiv=\"cache-control\" content=\"no-cache\">\n", + "</head>\n", + "<body bgcolor=\"white\" text=\"black\" ", + "link=\"blue\" vlink=\"purple\" alink=\"red\">", + "<h2>Results from test ", TestName, "</h2>\n"]; + Html -> + ["<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n", + "<!-- autogenerated by '", atom_to_list(?MODULE), "'. -->\n" | Html] + end, + print(html, Header, []), print_timestamp(html, "Test started at "), print(html, "<p>Host:<br>\n"), @@ -1643,7 +1711,7 @@ do_test_cases(TopCases, SkipCases, Config, MultiplyTimetrap) when is_list(TopCas [TI#target_info.version, TI#target_info.root_dir]); _ -> case test_server_sup:framework_call(target_info, []) of - TargetInfo when is_list(TargetInfo), + TargetInfo when is_list(TargetInfo), length(TargetInfo) > 0 -> print(html, "<p>Target:<br>\n"), print(html, "~s\n", [TargetInfo]); @@ -1658,7 +1726,7 @@ do_test_cases(TopCases, SkipCases, Config, MultiplyTimetrap) when is_list(TopCas [?suitelog_name,?coverlog_name]), print(html,"<p>~s" "<p>\n" - "<table border=3 cellpadding=5>" + "<table bgcolor=\"white\" border=\"3\" cellpadding=\"5\">" "<tr><th>Num</th><th>Module</th><th>Case</th><th>Log</th>" "<th>Time</th><th>Result</th><th>Comment</th></tr>\n", [print_if_known(N, {"Suite contains ~p test cases.\n",[N]}, @@ -1681,12 +1749,12 @@ do_test_cases(TopCases, SkipCases, Config, MultiplyTimetrap) when is_list(TopCas print(major, "=otp_release ~s", [TI#target_info.otp_release]), print(major, "=started ~s", [lists:flatten(timestamp_get(""))]), - run_test_cases(TestSpec, Config, MultiplyTimetrap) + run_test_cases(TestSpec, Config, TimetrapData) end; -do_test_cases(TopCase, SkipCases, Config, MultiplyTimetrap) -> +do_test_cases(TopCase, SkipCases, Config, TimetrapSpec) -> %% when not list(TopCase) - do_test_cases([TopCase], SkipCases, Config, MultiplyTimetrap). + do_test_cases([TopCase], SkipCases, Config, TimetrapSpec). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -1741,8 +1809,8 @@ start_log_file() -> ok. make_html_link(LinkName, Target, Explanation) -> - %% if possible use a relative reference�to�Target. - TargetL = filename:split(Target), + %% if possible use a relative reference to Target. + TargetL = filename:split(Target), PwdL = filename:split(filename:dirname(LinkName)), Href = case lists:prefix(PwdL, TargetL) of true -> @@ -1782,7 +1850,7 @@ start_minor_log_file(Mod, Func) -> start_minor_log_file1(Mod, Func, LogDir, AbsName); {ok,_} -> %% special case, duplicate names {_,S,Us} = now(), - Name1_0 = + Name1_0 = lists:flatten(io_lib:format("~s.~s.~w.~w~s", [Mod,Func,S, trunc(Us/1000), ?html_ext])), @@ -1853,7 +1921,7 @@ html_convert_modules(TestSpec, _Config) -> %% Retrieve a list of modules out of the test spec. html_isolate_modules(List) -> html_isolate_modules(List, sets:new()). - + html_isolate_modules([], Set) -> sets:to_list(Set); html_isolate_modules([{skip_case,_}|Cases], Set) -> html_isolate_modules(Cases, Set); @@ -1919,36 +1987,36 @@ add_init_and_end_per_suite([{make,_,_}=Case|Cases], LastMod, LastRef) -> [Case|add_init_and_end_per_suite(Cases, LastMod, LastRef)]; add_init_and_end_per_suite([{skip_case,{{Mod,all},_}}=Case|Cases], LastMod, LastRef) when Mod =/= LastMod -> - {PreCases, NextMod, NextRef} = + {PreCases, NextMod, NextRef} = do_add_end_per_suite_and_skip(LastMod, LastRef, Mod), PreCases ++ [Case|add_init_and_end_per_suite(Cases, NextMod, NextRef)]; add_init_and_end_per_suite([{skip_case,{{Mod,_},_}}=Case|Cases], LastMod, LastRef) when Mod =/= LastMod -> - {PreCases, NextMod, NextRef} = + {PreCases, NextMod, NextRef} = do_add_init_and_end_per_suite(LastMod, LastRef, Mod), PreCases ++ [Case|add_init_and_end_per_suite(Cases, NextMod, NextRef)]; add_init_and_end_per_suite([{skip_case,{conf,_,{Mod,_},_}}=Case|Cases], LastMod, LastRef) when Mod =/= LastMod -> - {PreCases, NextMod, NextRef} = + {PreCases, NextMod, NextRef} = do_add_init_and_end_per_suite(LastMod, LastRef, Mod), PreCases ++ [Case|add_init_and_end_per_suite(Cases, NextMod, NextRef)]; add_init_and_end_per_suite([{skip_case,_}=Case|Cases], LastMod, LastRef) -> [Case|add_init_and_end_per_suite(Cases, LastMod, LastRef)]; -add_init_and_end_per_suite([{conf,_,_,{Mod,_}}=Case|Cases], LastMod, LastRef) +add_init_and_end_per_suite([{conf,_,_,{Mod,_}}=Case|Cases], LastMod, LastRef) when Mod =/= LastMod -> - {PreCases, NextMod, NextRef} = + {PreCases, NextMod, NextRef} = do_add_init_and_end_per_suite(LastMod, LastRef, Mod), PreCases ++ [Case|add_init_and_end_per_suite(Cases, NextMod, NextRef)]; add_init_and_end_per_suite([{conf,_,_,_}=Case|Cases], LastMod, LastRef) -> [Case|add_init_and_end_per_suite(Cases, LastMod, LastRef)]; -add_init_and_end_per_suite([{Mod,_}=Case|Cases], LastMod, LastRef) +add_init_and_end_per_suite([{Mod,_}=Case|Cases], LastMod, LastRef) when Mod =/= LastMod -> - {PreCases, NextMod, NextRef} = + {PreCases, NextMod, NextRef} = do_add_init_and_end_per_suite(LastMod, LastRef, Mod), PreCases ++ [Case|add_init_and_end_per_suite(Cases, NextMod, NextRef)]; add_init_and_end_per_suite([{Mod,_,_}=Case|Cases], LastMod, LastRef) when Mod =/= LastMod -> - {PreCases, NextMod, NextRef} = + {PreCases, NextMod, NextRef} = do_add_init_and_end_per_suite(LastMod, LastRef, Mod), PreCases ++ [Case|add_init_and_end_per_suite(Cases, NextMod, NextRef)]; add_init_and_end_per_suite([Case|Cases], LastMod, LastRef)-> @@ -1965,7 +2033,7 @@ do_add_init_and_end_per_suite(LastMod, LastRef, Mod) -> false -> code:load_file(Mod); _ -> ok end, - {Init,NextMod,NextRef} = + {Init,NextMod,NextRef} = case erlang:function_exported(Mod, init_per_suite, 1) of true -> Ref = make_ref(), @@ -1973,15 +2041,15 @@ do_add_init_and_end_per_suite(LastMod, LastRef, Mod) -> false -> {[],Mod,undefined} end, - Cases = + Cases = if LastRef==undefined -> Init; LastRef==skipped_suite -> Init; true -> - %% Adding end_per_suite here without checking if the + %% Adding end_per_suite here without checking if the %% function is actually exported. This is because a - %% conf case must have an end case - so if it doesn't + %% conf case must have an end case - so if it doesn't %% exist, it will only fail... [{conf,LastRef,[],{LastMod,end_per_suite}}|Init] end, @@ -1997,23 +2065,19 @@ do_add_end_per_suite_and_skip(LastMod, LastRef, Mod) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% run_test_cases(TestSpec, Config, MultiplyTimetrap) -> exit(Result) +%% run_test_cases(TestSpec, Config, TimetrapData) -> exit(Result) %% %% If remote target, a socket connection is established. %% Runs the specified tests, then displays/logs the summary. -run_test_cases(TestSpec, Config, MultiplyTimetrap) -> +run_test_cases(TestSpec, Config, TimetrapData) -> maybe_open_job_sock(), html_convert_modules(TestSpec, Config), - %%! For readable tracing... - %%! Config1 = [{data_dir,""},{priv_dir,""},{nodes,[]}], - %%! run_test_cases_loop(TestSpec, [[]], MultiplyTimetrap, [], []), + run_test_cases_loop(TestSpec, [Config], TimetrapData, [], []), - run_test_cases_loop(TestSpec, [Config], MultiplyTimetrap, [], []), - maybe_get_privdir(), {AllSkippedN,UserSkipN,AutoSkipN,SkipStr} = @@ -2060,10 +2124,10 @@ maybe_open_job_sock() -> %% tar packet containing the privdir created by the test case. maybe_get_privdir() -> case get(test_server_ctrl_job_sock) of - undefined -> + undefined -> %% local target ok; - Sock -> + Sock -> %% remote target request(Sock, job_done), gen_tcp:close(Sock) @@ -2071,37 +2135,39 @@ maybe_get_privdir() -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% run_test_cases_loop(TestCases, Config, MultiplyTimetrap, Mode, Status) -> ok +%% run_test_cases_loop(TestCases, Config, TimetrapData, Mode, Status) -> ok %% TestCases = [Test,...] %% Config = [[{Key,Val},...],...] +%% TimetrapData = {MultiplyTimetrap,ScaleTimetrap} %% MultiplyTimetrap = integer() | infinity +%% ScaleTimetrap = bool() %% Mode = [{Ref,[Prop,..],StartTime}] %% Ref = reference() -%% Prop = {name,Name} | sequence | parallel | +%% Prop = {name,Name} | sequence | parallel | %% shuffle | {shuffle,Seed} | -%% repeat | {repeat,N} | +%% repeat | {repeat,N} | %% repeat_until_all_ok | {repeat_until_all_ok,N} | -%% repeat_until_any_ok | {repeat_until_any_ok,N} | -%% repeat_until_any_fail | {repeat_until_any_fail,N} | -%% repeat_until_all_fail | {repeat_until_all_fail,N} +%% repeat_until_any_ok | {repeat_until_any_ok,N} | +%% repeat_until_any_fail | {repeat_until_any_fail,N} | +%% repeat_until_all_fail | {repeat_until_all_fail,N} %% Status = [{Ref,{{Ok,Skipped,Failed},CopiedCases}}] %% Ok = Skipped = Failed = [Case,...] %% %% Execute the TestCases under configuration Config. Config is a list %% of lists, where hd(Config) holds the config tuples for the current -%% conf case and tl(Config) is the data for the higher level conf cases. -%% Config data is "inherited" from top to nested conf cases, but +%% conf case and tl(Config) is the data for the higher level conf cases. +%% Config data is "inherited" from top to nested conf cases, but %% never the other way around. if length(Config) == 1, Config contains %% only the initial config data for the suite. %% %% Test may be one of the following: %% -%% {conf,Ref,Props,{Mod,Func}} Mod:Func is a configuration modification +%% {conf,Ref,Props,{Mod,Func}} Mod:Func is a configuration modification %% function, call it with the current configuration as argument. It will %% return a new configuration. %% -%% {make,Ref,{Mod,Func,Args}} Mod:Func is a make function, and it is called -%% with the given arguments. This function will *always* be called on the host +%% {make,Ref,{Mod,Func,Args}} Mod:Func is a make function, and it is called +%% with the given arguments. This function will *always* be called on the host %% - not on target. %% %% {Mod,Case} This is a normal test case. Determine the correct @@ -2114,16 +2180,16 @@ maybe_get_privdir() -> %% {skip_case,{conf,Ref,Case,Comment}} An init conf case gets skipped %% by the user. This will also cause the end conf case to be skipped. %% Note that it is not possible to skip an end conf case directly (it -%% can only be skipped indirectly by a skipped init conf case). The -%% comment (which gets printed in the log files) describes why the case +%% can only be skipped indirectly by a skipped init conf case). The +%% comment (which gets printed in the log files) describes why the case %% was skipped. %% -%% {skip_case,{Case,Comment}} A normal test case skipped by the user. -%% The comment (which gets printed in the log files) describes why the +%% {skip_case,{Case,Comment}} A normal test case skipped by the user. +%% The comment (which gets printed in the log files) describes why the %% case was skipped. %% %% {auto_skip_case,{conf,Ref,Case,Comment},Mode} This is the result of -%% an end conf case being automatically skipped due to a failing init +%% an end conf case being automatically skipped due to a failing init %% conf case. It could also be a nested conf case that gets skipped %% because of a failed or skipped top level conf. %% @@ -2151,25 +2217,25 @@ maybe_get_privdir() -> %% messages to the main process instead of writing the data to file %% (only true for printouts to common log files). %% -%% If a conf group nested under a parallel group in the test +%% If a conf group nested under a parallel group in the test %% specification should be started, the 'test_server_common_io_handler' %% value gets set also on the main process. This causes all printouts -%% to common files - both from parallel test cases and from cases +%% to common files - both from parallel test cases and from cases %% executed by the main process - to all end up as messages in the -%% inbox of the main process. +%% inbox of the main process. %% %% During execution of a parallel group (or of a group nested under a -%% parallel group), *any* new test case being started gets registered +%% parallel group), *any* new test case being started gets registered %% in a list saved in the dictionary with 'test_server_queued_io' as key. %% When the top level parallel group is finished (only then can we be %% sure all parallel test cases have finished and "reported in"), the -%% list of test cases is traversed in order and printout messages from -%% each process - including the main process - are handled in turn. See +%% list of test cases is traversed in order and printout messages from +%% each process - including the main process - are handled in turn. See %% handle_test_case_io_and_status/0 for details. %% %% To be able to handle nested conf groups with different properties, %% the Mode argument specifies a list of {Ref,Properties} tuples. -%% The head of the Mode list at any given time identifies the group +%% The head of the Mode list at any given time identifies the group %% currently being processed. The tail of the list identifies groups %% on higher level. %% @@ -2179,13 +2245,13 @@ maybe_get_privdir() -> %% %% A group nested under a parallel group will start executing in %% parallel with previous (parallel) test cases (no matter what -%% properties the nested group has). Test cases are however never +%% properties the nested group has). Test cases are however never %% executed in parallel with the start or end conf case of the same %% group! Because of this, the test_server_ctrl loop waits at %% the end conf of a group for all parallel cases to finish %% before the end conf case actually executes. This has the effect %% that it's only after a nested group has finished that any -%% remaining parallel cases in the previous group get spawned (*). +%% remaining parallel cases in the previous group get spawned (*). %% Example (all parallel cases): %% %% group1_init |----> @@ -2201,8 +2267,8 @@ maybe_get_privdir() -> %% run_test_cases_loop([{auto_skip_case,{Type,Ref,Case,Comment},SkipMode}|Cases], - Config, MultiplyTimetrap, Mode, Status) when Type==conf; - Type==make -> + Config, TimetrapData, Mode, Status) when Type==conf; + Type==make -> file:set_cwd(filename:dirname(get(test_server_dir))), CurrIOHandler = get(test_server_common_io_handler), @@ -2217,24 +2283,24 @@ run_test_cases_loop([{auto_skip_case,{Type,Ref,Case,Comment},SkipMode}|Cases], set_io_buffering(undefined), {Mod,Func} = skip_case(auto, Ref, 0, Case, Comment, false, SkipMode), test_server_sup:framework_call(report, [tc_auto_skip,{?pl2a(Mod),Func,Comment}]), - run_test_cases_loop(Cases, Config, MultiplyTimetrap, tl(Mode), + run_test_cases_loop(Cases, Config, TimetrapData, tl(Mode), delete_status(Ref, Status)); _ -> - %% this is a skipped end conf for a parallel group nested under a + %% this is a skipped end conf for a parallel group nested under a %% parallel group (io buffering is active) wait_for_cases(Ref), {Mod,Func} = skip_case(auto, Ref, 0, Case, Comment, true, SkipMode), test_server_sup:framework_call(report, [tc_auto_skip,{?pl2a(Mod),Func,Comment}]), case CurrIOHandler of - {Ref,_} -> + {Ref,_} -> %% current_io_handler was set by start conf of this - %% group, so we can unset it now (no more io from main + %% group, so we can unset it now (no more io from main %% process needs to be buffered) set_io_buffering(undefined); - _ -> + _ -> ok end, - run_test_cases_loop(Cases, Config, MultiplyTimetrap, tl(Mode), + run_test_cases_loop(Cases, Config, TimetrapData, tl(Mode), delete_status(Ref, Status)) end; {Ref,false} -> @@ -2242,7 +2308,7 @@ run_test_cases_loop([{auto_skip_case,{Type,Ref,Case,Comment},SkipMode}|Cases], %% nested under a parallel group {Mod,Func} = skip_case(auto, Ref, 0, Case, Comment, false, SkipMode), test_server_sup:framework_call(report, [tc_auto_skip,{?pl2a(Mod),Func,Comment}]), - run_test_cases_loop(Cases, Config, MultiplyTimetrap, tl(Mode), + run_test_cases_loop(Cases, Config, TimetrapData, tl(Mode), delete_status(Ref, Status)); {Ref,_} -> %% this is a skipped end conf for a non-parallel group nested under @@ -2250,22 +2316,22 @@ run_test_cases_loop([{auto_skip_case,{Type,Ref,Case,Comment},SkipMode}|Cases], {Mod,Func} = skip_case(auto, Ref, 0, Case, Comment, true, SkipMode), test_server_sup:framework_call(report, [tc_auto_skip,{?pl2a(Mod),Func,Comment}]), case CurrIOHandler of - {Ref,_} -> + {Ref,_} -> %% current_io_handler was set by start conf of this - %% group, so we can unset it now (no more io from main + %% group, so we can unset it now (no more io from main %% process needs to be buffered) set_io_buffering(undefined); - _ -> + _ -> ok end, - run_test_cases_loop(Cases, Config, MultiplyTimetrap, tl(Mode), + run_test_cases_loop(Cases, Config, TimetrapData, tl(Mode), delete_status(Ref, Status)); {_,false} -> %% this is a skipped start conf for a group which is not nested %% under a parallel group {Mod,Func} = skip_case(auto, Ref, 0, Case, Comment, false, SkipMode), test_server_sup:framework_call(report, [tc_auto_skip,{?pl2a(Mod),Func,Comment}]), - run_test_cases_loop(Cases, Config, MultiplyTimetrap, [conf(Ref,[])|Mode], Status); + run_test_cases_loop(Cases, Config, TimetrapData, [conf(Ref,[])|Mode], Status); {_,Ref0} when is_reference(Ref0) -> %% this is a skipped start conf for a group nested under a parallel group %% and if this is the first nested group, io buffering must be activated @@ -2276,19 +2342,19 @@ run_test_cases_loop([{auto_skip_case,{Type,Ref,Case,Comment},SkipMode}|Cases], end, {Mod,Func} = skip_case(auto, Ref, 0, Case, Comment, true, SkipMode), test_server_sup:framework_call(report, [tc_auto_skip,{?pl2a(Mod),Func,Comment}]), - run_test_cases_loop(Cases, Config, MultiplyTimetrap, [conf(Ref,[])|Mode], Status) - end; + run_test_cases_loop(Cases, Config, TimetrapData, [conf(Ref,[])|Mode], Status) + end; run_test_cases_loop([{auto_skip_case,{Case,Comment},SkipMode}|Cases], - Config, MultiplyTimetrap, Mode, Status) -> + Config, TimetrapData, Mode, Status) -> {Mod,Func} = skip_case(auto, undefined, get(test_server_case_num)+1, Case, Comment, (undefined /= get(test_server_common_io_handler)), SkipMode), test_server_sup:framework_call(report, [tc_auto_skip,{?pl2a(Mod),Func,Comment}]), - run_test_cases_loop(Cases, Config, MultiplyTimetrap, Mode, + run_test_cases_loop(Cases, Config, TimetrapData, Mode, update_status(skipped, Mod, Func, Status)); run_test_cases_loop([{skip_case,{conf,Ref,Case,Comment}}|Cases0], - Config, MultiplyTimetrap, Mode, Status) -> + Config, TimetrapData, Mode, Status) -> {Mod,Func} = skip_case(user, Ref, 0, Case, Comment, (undefined /= get(test_server_common_io_handler))), {Cases,Config1} = @@ -2301,24 +2367,24 @@ run_test_cases_loop([{skip_case,{conf,Ref,Case,Comment}}|Cases0], {skip_cases_upto(Ref, Cases0, Comment, conf, Mode),Config} end, test_server_sup:framework_call(report, [tc_user_skip,{?pl2a(Mod),Func,Comment}]), - run_test_cases_loop(Cases, Config1, MultiplyTimetrap, Mode, + run_test_cases_loop(Cases, Config1, TimetrapData, Mode, update_status(skipped, Mod, Func, Status)); run_test_cases_loop([{skip_case,{Case,Comment}}|Cases], - Config, MultiplyTimetrap, Mode, Status) -> + Config, TimetrapData, Mode, Status) -> {Mod,Func} = skip_case(user, undefined, get(test_server_case_num)+1, Case, Comment, (undefined /= get(test_server_common_io_handler))), test_server_sup:framework_call(report, [tc_user_skip,{?pl2a(Mod),Func,Comment}]), - run_test_cases_loop(Cases, Config, MultiplyTimetrap, Mode, + run_test_cases_loop(Cases, Config, TimetrapData, Mode, update_status(skipped, Mod, Func, Status)); %% a start *or* end conf case, wrapping test cases or other conf cases -run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, - Config, MultiplyTimetrap, Mode0, Status) -> - +run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, + Config, TimetrapData, Mode0, Status) -> + CurrIOHandler = get(test_server_common_io_handler), %% check and update the mode for test case execution and io msg handling - {StartConf,Mode,IOHandler,ConfTime,Status1} = + {StartConf,Mode,IOHandler,ConfTime,Status1} = case {curr_ref(Mode0),check_props(parallel, Mode0)} of {Ref,Ref} -> case check_props(parallel, tl(Mode0)) of @@ -2334,19 +2400,19 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, {false,tl(Mode0),undefined,Elapsed, update_status(Ref, OkSkipFail, Status)}; _ -> - %% this is an end conf for a parallel group nested under a + %% this is an end conf for a parallel group nested under a %% parallel group (io buffering is active) OkSkipFail = wait_for_cases(Ref), queue_test_case_io(Ref, self(), 0, Mod, Func), Elapsed = elapsed_time(conf_start(Ref, Mode0),?now)/1000000, case CurrIOHandler of - {Ref,_} -> + {Ref,_} -> %% current_io_handler was set by start conf of this %% group, so we can unset it after this case (no %% more io from main process needs to be buffered) {false,tl(Mode0),undefined,Elapsed, update_status(Ref, OkSkipFail, Status)}; - _ -> + _ -> {false,tl(Mode0),CurrIOHandler,Elapsed, update_status(Ref, OkSkipFail, Status)} end @@ -2362,16 +2428,16 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, queue_test_case_io(Ref, self(), 0, Mod, Func), Elapsed = elapsed_time(conf_start(Ref, Mode0),?now)/1000000, case CurrIOHandler of - {Ref,_} -> + {Ref,_} -> %% current_io_handler was set by start conf of this %% group, so we can unset it after this case (no %% more io from main process needs to be buffered) {false,tl(Mode0),undefined,Elapsed,Status}; - _ -> + _ -> {false,tl(Mode0),CurrIOHandler,Elapsed,Status} end; {_,false} -> - %% this is a start conf for a group which is not nested under a + %% this is a start conf for a group which is not nested under a %% parallel group, check if this case starts a new parallel group case lists:member(parallel, Props) of true -> @@ -2424,9 +2490,9 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, end; NumStr -> %% Ex: "123 456 789" or "123,456,789" -> {123,456,789} - list_to_tuple([list_to_integer(NS) || + list_to_tuple([list_to_integer(NS) || NS <- string:tokens(NumStr, [$ ,$:,$,])]) - end, + end, {shuffle_cases(Ref, Cs0, UseSeed),{shuffle,UseSeed}} end; not StartConf -> @@ -2440,17 +2506,19 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, if StartConf -> case get_repeat(Props) of undefined -> - %% we *must* have a status entry for every conf since we + %% we *must* have a status entry for every conf since we %% will continously update status with test case results %% without knowing the Ref (but update hd(Status)) {false,new_status(Ref, Status1),Cases1,?void_fun}; - _ -> + {_RepType,N} when N =< 1 -> + {false,new_status(Ref, Status1),Cases1,?void_fun}; + _ -> {Copied,_} = copy_cases(Ref, make_ref(), Cs1), {true,new_status(Ref, Copied, Status1),Cases1,?void_fun} end; not StartConf -> RepVal = get_repeat(get_props(Mode0)), - ReportStop = + ReportStop = fun() -> print(minor, "~n*** Stopping repeat operation ~w", [RepVal]), print(1, "Stopping repeat operation ~w", [RepVal]) @@ -2461,21 +2529,23 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, case RepVal of undefined -> {false,EndStatus,Cases1,?void_fun}; + {_RepType,N} when N =< 1 -> + {false,EndStatus,Cases1,?void_fun}; {repeat,_} -> {true,EndStatus,CopiedCases++Cases1,?void_fun}; {repeat_until_all_ok,_} -> {RestCs,Fun} = case get_tc_results(Status1) of - {_,_,[]} -> + {_,_,[]} -> {Cases1,ReportStop}; - _ -> + _ -> {CopiedCases++Cases1,?void_fun} end, {true,EndStatus,RestCs,Fun}; {repeat_until_any_ok,_} -> {RestCs,Fun} = case get_tc_results(Status1) of - {Ok,_,_} when length(Ok) > 0 -> + {Ok,_,_} when length(Ok) > 0 -> {Cases1,ReportStop}; - _ -> + _ -> {CopiedCases++Cases1,?void_fun} end, {true,EndStatus,RestCs,Fun}; @@ -2483,15 +2553,15 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, {RestCs,Fun} = case get_tc_results(Status1) of {_,_,Fails} when length(Fails) > 0 -> {Cases1,ReportStop}; - _ -> + _ -> {CopiedCases++Cases1,?void_fun} end, {true,EndStatus,RestCs,Fun}; {repeat_until_all_fail,_} -> {RestCs,Fun} = case get_tc_results(Status1) of - {[],_,_} -> + {[],_,_} -> {Cases1,ReportStop}; - _ -> + _ -> {CopiedCases++Cases1,?void_fun} end, {true,EndStatus,RestCs,Fun} @@ -2517,13 +2587,13 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, [{tc_group_properties,get_props(Mode0)}, {tc_group_result,[{ok,TcOk},{skipped,TcSkip},{failed,TcFail}]}] end, - ActualCfg = + ActualCfg = update_config(hd(Config), [{priv_dir,get(test_server_priv_dir)}, {data_dir,get_data_dir(Mod)}] ++ CfgProps), CurrMode = curr_mode(Ref, Mode0, Mode), - ConfCaseResult = run_test_case(Ref, 0, Mod, Func, [ActualCfg], skip_init, target, - MultiplyTimetrap, CurrMode), + ConfCaseResult = run_test_case(Ref, 0, Mod, Func, [ActualCfg], skip_init, target, + TimetrapData, CurrMode), case ConfCaseResult of {_,NewCfg,_} when Func == init_per_suite, is_list(NewCfg) -> @@ -2533,8 +2603,8 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, [] -> set_io_buffering(IOHandler), stop_minor_log_file(), - run_test_cases_loop(Cases, [NewCfg|Config], - MultiplyTimetrap, Mode, Status2); + run_test_cases_loop(Cases, [NewCfg|Config], + TimetrapData, Mode, Status2); Bad -> print(minor, "~n*** ~p returned bad elements in Config: ~p.~n", [Func,Bad]), @@ -2542,22 +2612,22 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, Cases2 = skip_cases_upto(Ref, Cases, Reason, conf, CurrMode), set_io_buffering(IOHandler), stop_minor_log_file(), - run_test_cases_loop(Cases2, Config, MultiplyTimetrap, Mode, + run_test_cases_loop(Cases2, Config, TimetrapData, Mode, delete_status(Ref, Status2)) - end; + end; {_,NewCfg,_} when StartConf, is_list(NewCfg) -> print_conf_time(ConfTime), set_io_buffering(IOHandler), stop_minor_log_file(), - run_test_cases_loop(Cases, [NewCfg|Config], MultiplyTimetrap, Mode, Status2); + run_test_cases_loop(Cases, [NewCfg|Config], TimetrapData, Mode, Status2); {_,{framework_error,{FwMod,FwFunc},Reason},_} -> print(minor, "~n*** ~p failed in ~p. Reason: ~p~n", [FwMod,FwFunc,Reason]), print(1, "~p failed in ~p. Reason: ~p~n", [FwMod,FwFunc,Reason]), exit(framework_error); - {_,Fail,_} when element(1,Fail) == 'EXIT'; + {_,Fail,_} when element(1,Fail) == 'EXIT'; element(1,Fail) == timetrap_timeout; element(1,Fail) == failed -> - {Cases2,Config1} = + {Cases2,Config1} = if StartConf -> ReportAbortRepeat(failed), print(minor, "~n*** ~p failed.~n" @@ -2571,7 +2641,7 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, end, set_io_buffering(IOHandler), stop_minor_log_file(), - run_test_cases_loop(Cases2, Config1, MultiplyTimetrap, Mode, + run_test_cases_loop(Cases2, Config1, TimetrapData, Mode, delete_status(Ref, Status2)); {died,Why,_} when Func == init_per_suite -> print(minor, "~n*** Unexpected exit during init_per_suite.~n", []), @@ -2579,16 +2649,16 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, Cases2 = skip_cases_upto(Ref, Cases, Reason, conf, CurrMode), set_io_buffering(IOHandler), stop_minor_log_file(), - run_test_cases_loop(Cases2, Config, MultiplyTimetrap, Mode, - delete_status(Ref, Status2)); + run_test_cases_loop(Cases2, Config, TimetrapData, Mode, + delete_status(Ref, Status2)); {_,{Skip,Reason},_} when StartConf and ((Skip==skip) or (Skip==skipped)) -> ReportAbortRepeat(skipped), print(minor, "~n*** ~p skipped.~n" " Skipping all cases.", [Func]), set_io_buffering(IOHandler), stop_minor_log_file(), - run_test_cases_loop(skip_cases_upto(Ref, Cases, Reason, conf, CurrMode), - Config, MultiplyTimetrap, Mode, + run_test_cases_loop(skip_cases_upto(Ref, Cases, Reason, conf, CurrMode), + Config, TimetrapData, Mode, delete_status(Ref, Status2)); {_,{skip_and_save,Reason,_SavedConfig},_} when StartConf -> ReportAbortRepeat(skipped), @@ -2596,8 +2666,8 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, " Skipping all cases.", [Func]), set_io_buffering(IOHandler), stop_minor_log_file(), - run_test_cases_loop(skip_cases_upto(Ref, Cases, Reason, conf, CurrMode), - Config, MultiplyTimetrap, Mode, + run_test_cases_loop(skip_cases_upto(Ref, Cases, Reason, conf, CurrMode), + Config, TimetrapData, Mode, delete_status(Ref, Status2)); {_,_Other,_} when Func == init_per_suite -> print(minor, "~n*** init_per_suite failed to return a Config list.~n", []), @@ -2605,16 +2675,16 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, Cases2 = skip_cases_upto(Ref, Cases, Reason, conf, CurrMode), set_io_buffering(IOHandler), stop_minor_log_file(), - run_test_cases_loop(Cases2, Config, MultiplyTimetrap, Mode, + run_test_cases_loop(Cases2, Config, TimetrapData, Mode, delete_status(Ref, Status2)); {_,_Other,_} when StartConf -> print_conf_time(ConfTime), set_io_buffering(IOHandler), ReportRepeatStop(), stop_minor_log_file(), - run_test_cases_loop(Cases, [hd(Config)|Config], MultiplyTimetrap, + run_test_cases_loop(Cases, [hd(Config)|Config], TimetrapData, Mode, Status2); - + {_,_EndConfRetVal,Opts} -> %% check if return_group_result is set (ok, skipped or failed) and %% if so return the value to the group "above" so that result may be @@ -2631,35 +2701,35 @@ run_test_cases_loop([{conf,Ref,Props,{Mod,Func}}|_Cases]=Cs0, ReportRepeatStop(), set_io_buffering(IOHandler), stop_minor_log_file(), - run_test_cases_loop(Cases, tl(Config), MultiplyTimetrap, Mode, Status3) + run_test_cases_loop(Cases, tl(Config), TimetrapData, Mode, Status3) end; -run_test_cases_loop([{make,Ref,{Mod,Func,Args}}|Cases0], Config, MultiplyTimetrap, Mode, Status) -> - case run_test_case(Ref, 0, Mod, Func, Args, skip_init, host, MultiplyTimetrap) of +run_test_cases_loop([{make,Ref,{Mod,Func,Args}}|Cases0], Config, TimetrapData, Mode, Status) -> + case run_test_case(Ref, 0, Mod, Func, Args, skip_init, host, TimetrapData) of {_,Why={'EXIT',_},_} -> print(minor, "~n*** ~p failed.~n" " Skipping all cases.", [Func]), Reason = {failed,{Mod,Func,Why}}, Cases = skip_cases_upto(Ref, Cases0, Reason, conf, Mode), stop_minor_log_file(), - run_test_cases_loop(Cases, Config, MultiplyTimetrap, Mode, Status); + run_test_cases_loop(Cases, Config, TimetrapData, Mode, Status); {_,_Whatever,_} -> stop_minor_log_file(), - run_test_cases_loop(Cases0, Config, MultiplyTimetrap, Mode, Status) + run_test_cases_loop(Cases0, Config, TimetrapData, Mode, Status) end; -run_test_cases_loop([{conf,_Ref,_Props,_X}=Conf|_Cases0], - Config, _MultiplyTimetrap, _Mode, _Status) -> +run_test_cases_loop([{conf,_Ref,_Props,_X}=Conf|_Cases0], + Config, _TimetrapData, _Mode, _Status) -> erlang:error(badarg, [Conf,Config]); -run_test_cases_loop([{Mod,Case}|Cases], Config, MultiplyTimetrap, Mode, Status) -> - ActualCfg = +run_test_cases_loop([{Mod,Case}|Cases], Config, TimetrapData, Mode, Status) -> + ActualCfg = update_config(hd(Config), [{priv_dir,get(test_server_priv_dir)}, {data_dir,get_data_dir(Mod)}]), run_test_cases_loop([{Mod,Case,[ActualCfg]}|Cases], Config, - MultiplyTimetrap, Mode, Status); + TimetrapData, Mode, Status); -run_test_cases_loop([{Mod,Func,Args}|Cases], Config, MultiplyTimetrap, Mode, Status) -> +run_test_cases_loop([{Mod,Func,Args}|Cases], Config, TimetrapData, Mode, Status) -> Num = put(test_server_case_num, get(test_server_case_num)+1), %% check the current execution mode and save info about the case if %% detected that printouts to common log files is handled later @@ -2669,15 +2739,15 @@ run_test_cases_loop([{Mod,Func,Args}|Cases], Config, MultiplyTimetrap, Mode, Sta undefined -> %% io printouts are written to straight to file ok; - _ -> + _ -> %% io messages are buffered, put test case in queue queue_test_case_io(undefined, self(), Num+1, Mod, Func) end; _ -> ok end, - case run_test_case(undefined, Num+1, Mod, Func, Args, - run_init, target, MultiplyTimetrap, Mode) of + case run_test_case(undefined, Num+1, Mod, Func, Args, + run_init, target, TimetrapData, Mode) of %% callback to framework module failed, exit immediately {_,{framework_error,{FwMod,FwFunc},Reason},_} -> print(minor, "~n*** ~p failed in ~p. Reason: ~p~n", [FwMod,FwFunc,Reason]), @@ -2688,50 +2758,50 @@ run_test_cases_loop([{Mod,Func,Args}|Cases], Config, MultiplyTimetrap, Mode, Sta {Time,RetVal,_} -> {Failed,Status1} = case Time of - died -> + died -> {true,update_status(failed, Mod, Func, Status)}; _ when is_tuple(RetVal) -> case element(1, RetVal) of - R when R=='EXIT'; R==failed -> + R when R=='EXIT'; R==failed -> {true,update_status(failed, Mod, Func, Status)}; - R when R==skip; R==skipped -> + R when R==skip; R==skipped -> {false,update_status(skipped, Mod, Func, Status)}; - _ -> + _ -> {false,update_status(ok, Mod, Func, Status)} end; - _ -> + _ -> {false,update_status(ok, Mod, Func, Status)} - end, + end, case check_prop(sequence, Mode) of false -> stop_minor_log_file(), - run_test_cases_loop(Cases, Config, MultiplyTimetrap, Mode, Status1); - Ref -> - %% the case is in a sequence; we must check the result and + run_test_cases_loop(Cases, Config, TimetrapData, Mode, Status1); + Ref -> + %% the case is in a sequence; we must check the result and %% determine if the following cases should run or be skipped if not Failed -> % proceed with next case stop_minor_log_file(), - run_test_cases_loop(Cases, Config, MultiplyTimetrap, Mode, Status1); + run_test_cases_loop(Cases, Config, TimetrapData, Mode, Status1); true -> % skip rest of cases in sequence print(minor, "~n*** ~p failed.~n" " Skipping all other cases in sequence.", [Func]), Reason = {failed,{Mod,Func}}, Cases2 = skip_cases_upto(Ref, Cases, Reason, tc, Mode), stop_minor_log_file(), - run_test_cases_loop(Cases2, Config, MultiplyTimetrap, Mode, Status1) + run_test_cases_loop(Cases2, Config, TimetrapData, Mode, Status1) end end; %% the test case is being executed in parallel with the main process (and %% other test cases) and Pid is the dedicated process executing the case Pid -> - %% io from Pid will be buffered in the main process inbox and handled + %% io from Pid will be buffered in the main process inbox and handled %% later, so we have to save info about the case queue_test_case_io(undefined, Pid, Num+1, Mod, Func), - run_test_cases_loop(Cases, Config, MultiplyTimetrap, Mode, Status) + run_test_cases_loop(Cases, Config, TimetrapData, Mode, Status) end; %% TestSpec processing finished -run_test_cases_loop([], _Config, _MultiplyTimetrap, _, _) -> +run_test_cases_loop([], _Config, _TimetrapData, _, _) -> ok. %%-------------------------------------------------------------------- @@ -2798,12 +2868,12 @@ check_props(Attrib, Mode) -> case [R || {R,Ps,_} <- Mode, lists:member(Attrib, Ps)] of [] -> false; [Ref|_] -> Ref - end. + end. get_name([{_Ref,Props,_}|_]) -> proplists:get_value(name, Props); get_name([]) -> - undefined. + undefined. conf_start(Ref, Mode) -> case lists:keysearch(Ref, 1, Mode) of @@ -2826,10 +2896,10 @@ print_conf_time(0) -> print_conf_time(ConfTime) -> print(major, "=group_time ~.3fs", [ConfTime]), print(minor, "~n=== Total execution time of group: ~.3fs~n", [ConfTime]). - -print_props(_, []) -> + +print_props(_, []) -> ok; -print_props(true, Props) -> +print_props(true, Props) -> print(major, "=group_props ~p", [Props]), print(minor, "Group properties: ~p~n", [Props]); print_props(_, _) -> @@ -2853,12 +2923,12 @@ update_repeat(Props) -> Props1 = if N == forever -> [{RepType,N}|lists:keydelete(RepType, 1, Props)]; - N < 2 -> + N < 3 -> lists:keydelete(RepType, 1, Props); - N >= 2 -> + N >= 3 -> [{RepType,N-1}|lists:keydelete(RepType, 1, Props)] end, - %% if shuffle is used in combination with repeat, a new + %% if shuffle is used in combination with repeat, a new %% seed shouldn't be set every new turn case get_shuffle(Props1) of undefined -> @@ -2874,13 +2944,13 @@ get_shuffle(Props) -> delete_shuffle(Props) -> delete_prop([shuffle], Props). -%% Return {Item,Value} if found, else if Item alone +%% Return {Item,Value} if found, else if Item alone %% is found, return {Item,Default} get_prop([Item|Items], Default, Props) -> case lists:keysearch(Item, 1, Props) of - {value,R} -> + {value,R} -> R; - false -> + false -> case lists:member(Item, Props) of true -> {Item,Default}; @@ -2940,8 +3010,8 @@ random_order(1, {_Pos,Seed}, [{_Ix,CaseOrGroup}], Shuffled) -> put(test_server_curr_random_seed, Seed), Shuffled++CaseOrGroup; random_order(N, {Pos,NewSeed}, IxCases, Shuffled) -> - {First,[{_Ix,CaseOrGroup}|Rest]} = lists:split(Pos-1, IxCases), - random_order(N-1, random:uniform_s(N-1, NewSeed), + {First,[{_Ix,CaseOrGroup}|Rest]} = lists:split(Pos-1, IxCases), + random_order(N-1, random:uniform_s(N-1, NewSeed), First++Rest, Shuffled++CaseOrGroup). @@ -2949,7 +3019,7 @@ random_order(N, {Pos,NewSeed}, IxCases, Shuffled) -> %% skip_case(Type, Ref, CaseNum, Case, Comment, SendSync) -> {Mod,Func} %% %% Prints info about a skipped case in the major and html log files. -%% SendSync determines if start and finished messages must be sent so +%% SendSync determines if start and finished messages must be sent so %% that the printouts can be buffered and handled in order with io from %% parallel processes. @@ -2969,13 +3039,13 @@ skip_case(Type, Ref, CaseNum, Case, Comment, SendSync, Mode) -> not SendSync -> skip_case1(Type, CaseNum, Mod, Func, Comment, Mode) end, - MF. + MF. skip_case1(Type, CaseNum, Mod, Func, Comment, Mode) -> {{Col0,Col1},_} = get_font_style((CaseNum > 0), Mode), ResultCol = if Type == auto -> "#ffcc99"; Type == user -> "#ff9933" - end, + end, Comment1 = reason_to_string(Comment), @@ -3084,7 +3154,7 @@ modify_cases_upto1(Ref, ModOp, [{skip_case,{_F,_Cmt}}=MF|T], Orig, Alt) -> %% next is a normal case (possibly in a sequence), mark as skipped, or copy, and proceed modify_cases_upto1(Ref, {skip,Reason,_,Mode}=Op, [{_M,_F}=MF|T], Orig, Alt) -> - modify_cases_upto1(Ref, Op, T, Orig, [{auto_skip_case,{MF,Reason},Mode}|Alt]); + modify_cases_upto1(Ref, Op, T, Orig, [{auto_skip_case,{MF,Reason},Mode}|Alt]); modify_cases_upto1(Ref, CopyOp, [{_M,_F}=MF|T], Orig, Alt) -> modify_cases_upto1(Ref, CopyOp, T, [MF|Orig], [MF|Alt]); @@ -3110,7 +3180,7 @@ set_io_buffering(IOHandler) -> %% queue_test_case_io(Pid, Num, Mod, Func) -> ok %% %% Save info about test case that gets its io buffered. This can -%% be a parallel test case or it can be a test case (conf or normal) +%% be a parallel test case or it can be a test case (conf or normal) %% that belongs to a group nested under a parallel group. The queue %% is processed after io buffering is disabled. See run_test_cases_loop/4 %% and handle_test_case_io_and_status/0 for more info. @@ -3124,10 +3194,10 @@ queue_test_case_io(Ref, Pid, Num, Mod, Func) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% wait_for_cases(Ref) -> {Ok,Skipped,Failed} %% -%% At the end of a nested parallel group, we have to wait for the test +%% At the end of a nested parallel group, we have to wait for the test %% cases to terminate before we can go on (since test cases never execute -%% in parallel with the end conf case of the group). When a top level -%% parallel group is finished, buffered io messages must be handled and +%% in parallel with the end conf case of the group). When a top level +%% parallel group is finished, buffered io messages must be handled and %% this is taken care of by handle_test_case_io_and_status/0. wait_for_cases(Ref) -> @@ -3135,15 +3205,15 @@ wait_for_cases(Ref) -> [] -> {[],[],[]}; Cases -> - [_Start|TCs] = + [_Start|TCs] = lists:dropwhile(fun({R,_,_,_,_}) when R == Ref -> false; (_) -> true end, Cases), wait_and_resend(Ref, TCs, [],[],[]) end. -wait_and_resend(Ref, [{OtherRef,_,0,_,_}|Ps], - Ok,Skip,Fail) when is_reference(OtherRef), +wait_and_resend(Ref, [{OtherRef,_,0,_,_}|Ps], + Ok,Skip,Fail) when is_reference(OtherRef), OtherRef /= Ref -> %% ignore cases that belong to nested group Ps1 = rm_cases_upto(OtherRef, Ps), @@ -3152,7 +3222,7 @@ wait_and_resend(Ref, [{OtherRef,_,0,_,_}|Ps], wait_and_resend(Ref, [{_,CurrPid,CaseNum,Mod,Func}|Ps] = Cases, Ok,Skip,Fail) -> receive {finished,_Ref,CurrPid,CaseNum,Mod,Func,Result,_RetVal} = Msg -> - %% resend message to main process so that it can be used + %% resend message to main process so that it can be used %% to handle buffered io messages later self() ! Msg, MF = {Mod,Func}, @@ -3163,7 +3233,7 @@ wait_and_resend(Ref, [{_,CurrPid,CaseNum,Mod,Func}|Ps] = Cases, Ok,Skip,Fail) -> failed -> {Ok,Skip,[MF|Fail]} end, wait_and_resend(Ref, Ps, Ok1,Skip1,Fail1); - {'EXIT',CurrPid,Reason} when Reason /= normal -> + {'EXIT',CurrPid,Reason} when Reason /= normal -> %% unexpected termination of test case process {value,{_,_,CaseNum,Mod,Func}} = lists:keysearch(CurrPid, 2, Cases), print(1, "Error! Process for test case #~p (~p:~p) died! Reason: ~p", @@ -3186,17 +3256,17 @@ rm_cases_upto(Ref, [_|Ps]) -> %% execution. The common log files (major, html etc) must however be %% written to sequentially. The test case processes send print requests %% to the main (starting) process (the same process executing -%% run_test_cases_loop/4), which handles these requests in the same +%% run_test_cases_loop/4), which handles these requests in the same %% order that the test case processes were started. %% %% An io session is always started with a {started,Ref,Pid,Num,Mod,Func} %% message and terminated with {finished,Ref,Pid,Num,Mod,Func,Result,RetVal}. %% The result shipped with the finished message from a parallel process -%% is used to update status data of the current test run. An 'EXIT' -%% message from each parallel test case process (after finishing and +%% is used to update status data of the current test run. An 'EXIT' +%% message from each parallel test case process (after finishing and %% terminating) is also received and handled here. %% -%% During execution of a parallel group, any cases (conf or normal) +%% During execution of a parallel group, any cases (conf or normal) %% belonging to a nested group will also get its io printouts buffered. %% This is necessary to get the major and html log files written in %% correct sequence. This function handles also the print messages @@ -3207,7 +3277,7 @@ rm_cases_upto(Ref, [_|Ps]) -> %% See the header comment for run_test_cases_loop/4 for more %% info about IO handling. %% -%% Note: It is important that the type of messages handled here +%% Note: It is important that the type of messages handled here %% do not get consumated by test_server:run_test_case_msgloop/5 %% during the test case execution (e.g. in the catch clause of %% the receive)! @@ -3231,7 +3301,7 @@ handle_test_case_io_and_status() -> ok end, Cases), Result - end. + end. %% Handle cases (without Ref) that belong to the top parallel group (i.e. when Refs = []) handle_io_and_exit_loop([], [{undefined,CurrPid,CaseNum,Mod,Func}|Ps] = Cases, Ok,Skip,Fail) -> @@ -3249,7 +3319,7 @@ handle_io_and_exit_loop([], [{undefined,CurrPid,CaseNum,Mod,Func}|Ps] = Cases, O 1000 -> exit({testcase_failed_to_start,Mod,Func}) end; - + %% Handle cases that belong to groups nested under top parallel group handle_io_and_exit_loop(Refs, [{Ref,CurrPid,CaseNum,Mod,Func}|Ps] = Cases, Ok,Skip,Fail) -> receive @@ -3269,7 +3339,7 @@ handle_io_and_exit_loop(Refs, [{Ref,CurrPid,CaseNum,Mod,Func}|Ps] = Cases, Ok,Sk 1000 -> exit({testcase_failed_to_start,Mod,Func}) end; - + handle_io_and_exit_loop(_, [], Ok,Skip,Fail) -> {lists:reverse(Ok),lists:reverse(Skip),lists:reverse(Fail)}. @@ -3286,7 +3356,7 @@ handle_io_and_exits(Main, CurrPid, CaseNum, Mod, Func, Cases) -> failed -> put(test_server_failed, get(test_server_failed)+1); skipped -> - SkipCounters = + SkipCounters = update_skip_counters(RetVal, get(test_server_skipped)), put(test_server_skipped, SkipCounters) end, @@ -3298,7 +3368,7 @@ handle_io_and_exits(Main, CurrPid, CaseNum, Mod, Func, Cases) -> handle_io_and_exits(Main, CurrPid, CaseNum, Mod, Func, Cases); %% unexpected termination of test case process - {'EXIT',TCPid,Reason} when Reason /= normal -> + {'EXIT',TCPid,Reason} when Reason /= normal -> {value,{_,_,Num,M,F}} = lists:keysearch(TCPid, 2, Cases), print(1, "Error! Process for test case #~p (~p:~p) died! Reason: ~p", [Num, M, F, Reason]), @@ -3307,65 +3377,65 @@ handle_io_and_exits(Main, CurrPid, CaseNum, Mod, Func, Cases) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% run_test_case(Ref, Num, Mod, Func, Args, RunInit, -%% Where, MultiplyTimetrap, Mode) -> RetVal +%% run_test_case(Ref, Num, Mod, Func, Args, RunInit, +%% Where, TimetrapData, Mode) -> RetVal %% %% Creates the minor log file and inserts some test case specific headers -%% and footers into the log files. If a remote target is used, the test +%% and footers into the log files. If a remote target is used, the test %% suite (binary) and the content of data_dir is sent. Then the test case -%% is executed and the result is printed to the log files (also info +%% is executed and the result is printed to the log files (also info %% about lingering processes & slave nodes in the system is presented). -%% +%% %% RunInit decides if the per test case init is to be run (true for all %% but conf cases). %% -%% Where specifies if the test case should run on target or on the host. +%% Where specifies if the test case should run on target or on the host. %% (Note that 'make' test cases always run on host). -%% +%% %% Mode specifies if the test case should be executed by a dedicated, %% parallel, process rather than sequentially by the main process. If %% the former, the new process is spawned and the dictionary of the main %% process is copied to the test case process. -%% -%% RetVal is the result of executing the test case. It contains info +%% +%% RetVal is the result of executing the test case. It contains info %% about the execution time and the return value of the test case function. -run_test_case(Ref, Num, Mod, Func, Args, RunInit, Where, MultiplyTimetrap) -> +run_test_case(Ref, Num, Mod, Func, Args, RunInit, Where, TimetrapData) -> file:set_cwd(filename:dirname(get(test_server_dir))), - run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, - MultiplyTimetrap, [], [], self()). + run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, + TimetrapData, [], [], self()). -run_test_case(Ref, Num, Mod, Func, Args, skip_init, Where, MultiplyTimetrap, Mode) -> +run_test_case(Ref, Num, Mod, Func, Args, skip_init, Where, TimetrapData, Mode) -> %% a conf case is always executed by the main process - run_test_case1(Ref, Num, Mod, Func, Args, skip_init, Where, - MultiplyTimetrap, [], Mode, self()); + run_test_case1(Ref, Num, Mod, Func, Args, skip_init, Where, + TimetrapData, [], Mode, self()); -run_test_case(Ref, Num, Mod, Func, Args, RunInit, Where, MultiplyTimetrap, Mode) -> +run_test_case(Ref, Num, Mod, Func, Args, RunInit, Where, TimetrapData, Mode) -> file:set_cwd(filename:dirname(get(test_server_dir))), case check_prop(parallel, Mode) of false -> %% this is a sequential test case - run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, - MultiplyTimetrap, [], Mode, self()); + run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, + TimetrapData, [], Mode, self()); _Ref -> %% this a parallel test case, spawn the new process Main = self(), - {dictionary,State} = process_info(self(), dictionary), + {dictionary,State} = process_info(self(), dictionary), spawn_link(fun() -> - run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, - MultiplyTimetrap, State, Mode, Main) - end) + run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, + TimetrapData, State, Mode, Main) + end) end. -run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, - MultiplyTimetrap, State, Mode, Main) -> - %% if this runs on a parallel test case process, +run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, + TimetrapData, State, Mode, Main) -> + %% if this runs on a parallel test case process, %% copy the dictionary from the main process do_if_parallel(Main, fun() -> process_flag(trap_exit, true) end, ok), CopyDict = fun() -> lists:foreach(fun({Key,Val}) -> put(Key, Val) end, State) end, do_if_parallel(Main, CopyDict, ok), do_if_parallel(Main, fun() -> put(test_server_common_io_handler, {tc,Main}) end, ok), - %% if io is being buffered, send start io session message + %% if io is being buffered, send start io session message %% (no matter if case runs on parallel or main process) case get(test_server_common_io_handler) of undefined -> ok; @@ -3373,7 +3443,7 @@ run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, end, TSDir = get(test_server_dir), case Where of - target -> + target -> maybe_send_beam_and_datadir(Mod); host -> ok @@ -3396,8 +3466,8 @@ run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, do_if_parallel(Main, ok, fun erlang:yield/0), %% run the test case {Result,DetectedFail,ProcsBefore,ProcsAfter} = - run_test_case_apply(Num, Mod, Func, Args, get_name(Mode), - RunInit, Where, MultiplyTimetrap), + run_test_case_apply(Num, Mod, Func, Args, get_name(Mode), + RunInit, Where, TimetrapData), {Time,RetVal,Loc,Opts,Comment} = case Result of Normal={_Time,_RetVal,_Loc,_Opts,_Comment} -> Normal; @@ -3409,7 +3479,7 @@ run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, print(major, "=ended ~s", [lists:flatten(timestamp_get(""))]), do_if_parallel(Main, ok, fun() -> file:set_cwd(filename:dirname(TSDir)) end), - + %% call the appropriate progress function clause to print the results to log Status = case {Time,RetVal} of @@ -3423,16 +3493,16 @@ run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, progress(skip, Num, Mod, Func, Loc, Reason, Time, Comment, Style); {_,{'EXIT',_Pid,{Skip,Reason}}} when Skip==skip; Skip==skipped -> - progress(skip, Num, Mod, Func, Loc, Reason, + progress(skip, Num, Mod, Func, Loc, Reason, Time, Comment, Style); {_,{'EXIT',_Pid,Reason}} -> - progress(failed, Num, Mod, Func, Loc, Reason, + progress(failed, Num, Mod, Func, Loc, Reason, Time, Comment, Style); {_,{'EXIT',Reason}} -> - progress(failed, Num, Mod, Func, Loc, Reason, + progress(failed, Num, Mod, Func, Loc, Reason, Time, Comment, Style); {_, {failed, Reason}} -> - progress(failed, Num, Mod, Func, Loc, Reason, + progress(failed, Num, Mod, Func, Loc, Reason, Time, Comment, Style); {_, {Skip, Reason}} when Skip==skip; Skip==skipped -> progress(skip, Num, Mod, Func, Loc, Reason, @@ -3442,7 +3512,7 @@ run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, [] -> progress(ok, Num, Mod, Func, Loc, RetVal, Time, Comment, Style); - + Reason -> progress(failed, Num, Mod, Func, Loc, Reason, Time, Comment, Style) @@ -3465,18 +3535,18 @@ run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, {US,AS} = get(test_server_skipped), put(test_server_skipped, {US,AS+1}) end, - %% only if test case execution is sequential do we care about the + %% only if test case execution is sequential do we care about the %% remaining processes and slave nodes count case self() of Main -> case test_server_sup:framework_call(warn, [processes], true) of true -> if ProcsBefore < ProcsAfter -> - print(minor, + print(minor, "WARNING: ~w more processes in system after test case", [ProcsAfter-ProcsBefore]); ProcsBefore > ProcsAfter -> - print(minor, + print(minor, "WARNING: ~w less processes in system after test case", [ProcsBefore-ProcsAfter]); true -> ok @@ -3487,13 +3557,13 @@ run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, case test_server_sup:framework_call(warn, [nodes], true) of true -> case catch controller_call(kill_slavenodes) of - {'EXIT',_}=Exit -> + {'EXIT',_} = Exit -> print(minor, "WARNING: There might be slavenodes left in the" " system. I tried to kill them, but I failed: ~p\n", [Exit]); [] -> ok; - List -> + List -> print(minor, "WARNING: ~w slave nodes in system after test"++ "case. Tried to killed them.~n"++ " Names:~p", @@ -3505,8 +3575,8 @@ run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, _ -> ok end, - %% if the test case was executed sequentially, this updates the execution - %% time count on the main process (adding execution time of parallel test + %% if the test case was executed sequentially, this updates the execution + %% time count on the main process (adding execution time of parallel test %% case groups is done in run_test_cases_loop/4) if is_number(Time) -> put(test_server_total_time, get(test_server_total_time)+Time); @@ -3515,7 +3585,7 @@ run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, end, check_new_crash_dumps(Where), - %% if io is being buffered, send finished message + %% if io is being buffered, send finished message %% (no matter if case runs on parallel or main process) case get(test_server_common_io_handler) of undefined -> ok; @@ -3528,7 +3598,7 @@ run_test_case1(Ref, Num, Mod, Func, Args, RunInit, Where, %%-------------------------------------------------------------------- %% various help functions -%% Call If() if we're on parallel process, or +%% Call If() if we're on parallel process, or %% call Else() if we're on main process do_if_parallel(Pid, If, Else) -> case self() of @@ -3536,7 +3606,7 @@ do_if_parallel(Pid, If, Else) -> if is_function(Else) -> Else(); true -> Else end; - _ -> + _ -> if is_function(If) -> If(); true -> If end @@ -3549,13 +3619,13 @@ num2str(N) -> integer_to_list(N). %% and the content of datadir til target. maybe_send_beam_and_datadir(Mod) -> case get(test_server_ctrl_job_sock) of - undefined -> + undefined -> %% local target ok; JobSock -> %% remote target case get(test_server_downloaded_suites) of - undefined -> + undefined -> send_beam_and_datadir(Mod, JobSock), put(test_server_downloaded_suites, [Mod]); Suites -> @@ -3571,10 +3641,10 @@ maybe_send_beam_and_datadir(Mod) -> send_beam_and_datadir(Mod, JobSock) -> case code:which(Mod) of - non_existing -> + non_existing -> io:format("** WARNING: Suite ~w could not be found on host\n", [Mod]); - BeamFile -> + BeamFile -> send_beam(JobSock, Mod, BeamFile) end, DataDir = get_data_dir(Mod), @@ -3589,7 +3659,7 @@ send_beam_and_datadir(Mod, JobSock) -> ModsInDatadir = filelib:wildcard(Wc), SendBeamFun = fun(X) -> send_beam(JobSock, X) end, lists:foreach(SendBeamFun, ModsInDatadir), - %% No need to send C code or makefiles since + %% No need to send C code or makefiles since %% no compilation can be done on target anyway. %% Compiled C code must exist on target. %% Beam files are already sent as binaries. @@ -3597,7 +3667,7 @@ send_beam_and_datadir(Mod, JobSock) -> %% is to compile it. Filter = fun("Makefile") -> false; ("Makefile.src") -> false; - (Y) -> + (Y) -> case filename:extension(Y) of ".c" -> false; ObjExt -> false; @@ -3611,7 +3681,7 @@ send_beam_and_datadir(Mod, JobSock) -> Tarfile = "data_dir.tar.gz", {ok,Tar} = erl_tar:open(Tarfile, [write,compressed]), ShortDataDir = filename:basename(DataDir), - AddTarFun = + AddTarFun = fun(File) -> Long = filename:join(DataDir, File), Short = filename:join(ShortDataDir, File), @@ -3628,11 +3698,11 @@ send_beam_and_datadir(Mod, JobSock) -> send_beam(JobSock, BeamFile) -> Mod=filename:rootname(filename:basename(BeamFile), code:objfile_extension()), - send_beam(JobSock, list_to_atom(Mod), BeamFile). + send_beam(JobSock, list_to_atom(Mod), BeamFile). send_beam(JobSock, Mod, BeamFile) -> {ok,BeamBin} = file:read_file(BeamFile), request(JobSock, {{beam,Mod,BeamFile}, BeamBin}). - + check_new_crash_dumps(Where) -> case Where of target -> @@ -3649,25 +3719,25 @@ check_new_crash_dumps(Where) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% progress(Result, CaseNum, Mod, Func, Location, Reason, Time, +%% progress(Result, CaseNum, Mod, Func, Location, Reason, Time, %% Comment, TimeFormat) -> Result %% %% Prints the result of the test case to log file. %% Note: Strings that are to be written to the minor log must %% be prefixed with "=== " here, or the indentation will be wrong. -progress(skip, CaseNum, Mod, Func, Loc, Reason, Time, +progress(skip, CaseNum, Mod, Func, Loc, Reason, Time, Comment, {St0,St1}) -> - {Reason1,{Color,Ret}} = if_auto_skip(Reason, + {Reason1,{Color,Ret}} = if_auto_skip(Reason, fun() -> {"#ffcc99",auto_skip} end, fun() -> {"#ff9933",skip} end), print(major, "=result skipped", []), - print(1, "*** SKIPPED *** ~s", + print(1, "*** SKIPPED *** ~s", [get_info_str(Func, CaseNum, get(test_server_cases))]), test_server_sup:framework_call(report, [tc_done,{?pl2a(Mod),Func, {skipped,Reason1}}]), ReasonStr = reason_to_string(Reason1), - ReasonStr1 = lists:flatten([string:strip(S,left) || + ReasonStr1 = lists:flatten([string:strip(S,left) || S <- string:tokens(ReasonStr,[$\n])]), ReasonStr2 = if length(ReasonStr1) > 80 -> @@ -3686,10 +3756,10 @@ progress(skip, CaseNum, Mod, Func, Loc, Reason, Time, [Time,Color,ReasonStr2,Comment1]), FormatLoc = test_server_sup:format_loc(Loc), print(minor, "=== location ~s", [FormatLoc]), - print(minor, "=== reason = ~s", [ReasonStr1]), + print(minor, "=== reason = ~s", [ReasonStr1]), Ret; - -progress(failed, CaseNum, Mod, Func, Loc, timetrap_timeout, T, + +progress(failed, CaseNum, Mod, Func, Loc, timetrap_timeout, T, Comment0, {St0,St1}) -> print(major, "=result failed: timeout, ~p", [Loc]), print(1, "*** FAILED *** ~s", @@ -3699,23 +3769,23 @@ progress(failed, CaseNum, Mod, Func, Loc, timetrap_timeout, T, {failed,timetrap_timeout}}]), FormatLastLoc = test_server_sup:format_loc(get_last_loc(Loc)), ErrorReason = io_lib:format("{timetrap_timeout,~s}", [FormatLastLoc]), - Comment = + Comment = case Comment0 of "" -> "<font color=\"red\">" ++ ErrorReason ++ "</font>"; - _ -> "<font color=\"red\">" ++ ErrorReason ++ "</font><br>" ++ + _ -> "<font color=\"red\">" ++ ErrorReason ++ "</font><br>" ++ to_string(Comment0) end, - print(html, + print(html, "<td>" ++ St0 ++ "~.3fs" ++ St1 ++ "</td>" "<td><font color=\"red\">FAILED</font></td>" - "<td>~s</td></tr>\n", + "<td>~s</td></tr>\n", [T/1000,Comment]), FormatLoc = test_server_sup:format_loc(Loc), print(minor, "=== location ~s", [FormatLoc]), print(minor, "=== reason = timetrap timeout", []), failed; -progress(failed, CaseNum, Mod, Func, Loc, {testcase_aborted,Reason}, _T, +progress(failed, CaseNum, Mod, Func, Loc, {testcase_aborted,Reason}, _T, Comment0, {St0,St1}) -> print(major, "=result failed: testcase_aborted, ~p", [Loc]), print(1, "*** FAILED *** ~s", @@ -3725,23 +3795,23 @@ progress(failed, CaseNum, Mod, Func, Loc, {testcase_aborted,Reason}, _T, {failed,testcase_aborted}}]), FormatLastLoc = test_server_sup:format_loc(get_last_loc(Loc)), ErrorReason = io_lib:format("{testcase_aborted,~s}", [FormatLastLoc]), - Comment = + Comment = case Comment0 of "" -> "<font color=\"red\">" ++ ErrorReason ++ "</font>"; - _ -> "<font color=\"red\">" ++ ErrorReason ++ "</font><br>" ++ + _ -> "<font color=\"red\">" ++ ErrorReason ++ "</font><br>" ++ to_string(Comment0) end, - print(html, + print(html, "<td>" ++ St0 ++ "died" ++ St1 ++ "</td>" "<td><font color=\"red\">FAILED</font></td>" - "<td>~s</td></tr>\n", + "<td>~s</td></tr>\n", [Comment]), FormatLoc = test_server_sup:format_loc(Loc), print(minor, "=== location ~s", [FormatLoc]), print(minor, "=== reason = {testcase_aborted,~p}", [Reason]), failed; -progress(failed, CaseNum, Mod, Func, unknown, Reason, Time, +progress(failed, CaseNum, Mod, Func, unknown, Reason, Time, Comment0, {St0,St1}) -> print(major, "=result failed: ~p, ~p", [Reason,unknown]), print(1, "*** FAILED *** ~s", @@ -3749,10 +3819,10 @@ progress(failed, CaseNum, Mod, Func, unknown, Reason, Time, test_server_sup:framework_call(report, [tc_done,{?pl2a(Mod),Func, {failed,Reason}}]), TimeStr = io_lib:format(if is_float(Time) -> "~.3fs"; - true -> "~w" + true -> "~w" end, [Time]), ErrorReason = lists:flatten(io_lib:format("~p", [Reason])), - ErrorReason1 = lists:flatten([string:strip(S,left) || + ErrorReason1 = lists:flatten([string:strip(S,left) || S <- string:tokens(ErrorReason,[$\n])]), ErrorReason2 = if length(ErrorReason1) > 63 -> @@ -3760,13 +3830,13 @@ progress(failed, CaseNum, Mod, Func, unknown, Reason, Time, true -> ErrorReason1 end, - Comment = + Comment = case Comment0 of "" -> "<font color=\"red\">" ++ ErrorReason2 ++ "</font>"; - _ -> "<font color=\"red\">" ++ ErrorReason2 ++ "</font><br>" ++ + _ -> "<font color=\"red\">" ++ ErrorReason2 ++ "</font><br>" ++ to_string(Comment0) end, - print(html, + print(html, "<td>" ++ St0 ++ "~s" ++ St1 ++ "</td>" "<td><font color=\"red\">FAILED</font></td>" "<td>~s</td></tr>\n", @@ -3776,7 +3846,7 @@ progress(failed, CaseNum, Mod, Func, unknown, Reason, Time, print(minor, "=== reason = "++FStr, [FormattedReason]), failed; -progress(failed, CaseNum, Mod, Func, Loc, Reason, Time, +progress(failed, CaseNum, Mod, Func, Loc, Reason, Time, Comment0, {St0,St1}) -> print(major, "=result failed: ~p, ~p", [Reason,Loc]), print(1, "*** FAILED *** ~s", @@ -3784,18 +3854,18 @@ progress(failed, CaseNum, Mod, Func, Loc, Reason, Time, test_server_sup:framework_call(report, [tc_done,{?pl2a(Mod),Func, {failed,Reason}}]), TimeStr = io_lib:format(if is_float(Time) -> "~.3fs"; - true -> "~w" + true -> "~w" end, [Time]), - Comment = + Comment = case Comment0 of "" -> ""; _ -> "<br>" ++ to_string(Comment0) end, FormatLastLoc = test_server_sup:format_loc(get_last_loc(Loc)), - print(html, + print(html, "<td>" ++ St0 ++ "~s" ++ St1 ++ "</td>" "<td><font color=\"red\">FAILED</font></td>" - "<td><font color=\"red\">~s</font>~s</td></tr>\n", + "<td><font color=\"red\">~s</font>~s</td></tr>\n", [TimeStr,FormatLastLoc,Comment]), FormatLoc = test_server_sup:format_loc(Loc), print(minor, "=== location ~s", [FormatLoc]), @@ -3803,7 +3873,7 @@ progress(failed, CaseNum, Mod, Func, Loc, Reason, Time, print(minor, "=== reason = "++FStr, [FormattedReason]), failed; -progress(ok, _CaseNum, Mod, Func, _Loc, RetVal, Time, +progress(ok, _CaseNum, Mod, Func, _Loc, RetVal, Time, Comment0, {St0,St1}) -> print(minor, "successfully completed test case", []), test_server_sup:framework_call(report, [tc_done,{?pl2a(Mod),Func,ok}]), @@ -3852,9 +3922,9 @@ get_info_str(Func, 0, _Cases) -> get_info_str(_Func, CaseNum, unknown) -> "test case " ++ integer_to_list(CaseNum); get_info_str(_Func, CaseNum, Cases) -> - "test case " ++ integer_to_list(CaseNum) ++ + "test case " ++ integer_to_list(CaseNum) ++ " of " ++ integer_to_list(Cases). - + print_if_known(Known, {SK,AK}, {SU,AU}) -> {S,A} = if Known == unknown -> {SU,AU}; true -> {SK,AK} @@ -3880,7 +3950,7 @@ reason_to_string({failed,{_,FailFunc,bad_return}}) -> atom_to_list(FailFunc) ++ " bad return value"; reason_to_string({failed,{_,FailFunc,{timetrap_timeout,_}}}) -> atom_to_list(FailFunc) ++ " timed out"; -reason_to_string(FWInitFail = {failed,{_CB,init_tc,_Reason}}) -> +reason_to_string(FWInitFail = {failed,{_CB,init_tc,_Reason}}) -> to_string(FWInitFail); reason_to_string({failed,{_,FailFunc,_}}) -> atom_to_list(FailFunc) ++ " failed"; @@ -3889,29 +3959,29 @@ reason_to_string(Other) -> %get_font_style(Prop) -> % {Col,St0,St1} = get_font_style1(Prop), -% {{"<font color="++Col++">","</font>"}, +% {{"<font color="++Col++">","</font>"}, % {"<font color="++Col++">"++St0,St1++"</font>"}}. - + get_font_style(NormalCase, Mode) -> - Prop = if not NormalCase -> + Prop = if not NormalCase -> default; true -> case check_prop(parallel, Mode) of - false -> + false -> case check_prop(sequence, Mode) of - false -> + false -> default; - _ -> + _ -> sequence end; - _ -> + _ -> parallel end end, {Col,St0,St1} = get_font_style1(Prop), - {{"<font color="++Col++">","</font>"}, + {{"<font color="++Col++">","</font>"}, {"<font color="++Col++">"++St0,St1++"</font>"}}. - + get_font_style1(parallel) -> {"\"darkslategray\"","<i>","</i>"}; get_font_style1(sequence) -> @@ -3931,7 +4001,7 @@ get_font_style1(default) -> %% The framework application can switch this feature off by setting %% *its* application environment variable 'format_exception' to false. %% It is also possible to switch formatting off by starting the -%% test_server node with init argument 'test_server_format_exception' +%% test_server node with init argument 'test_server_format_exception' %% set to false. format_exception(Reason={_Error,Stack}) when is_list(Stack) -> @@ -3950,17 +4020,17 @@ format_exception(Reason={_Error,Stack}) when is_list(Stack) -> _ -> do_format_exception(Reason) end - end; + end; format_exception(Error) -> format_exception({Error,[]}). do_format_exception(Reason={Error,Stack}) -> StackFun = fun(_, _, _) -> false end, - PF = fun(Term, I) -> - io_lib:format("~." ++ integer_to_list(I) ++ "p", [Term]) + PF = fun(Term, I) -> + io_lib:format("~." ++ integer_to_list(I) ++ "p", [Term]) end, case catch lib:format_exception(1, error, Error, Stack, StackFun, PF) of - {'EXIT',_} -> + {'EXIT',_} -> {"~p",Reason}; Formatted -> Formatted1 = re:replace(Formatted, "exception error: ", "", [{return,list}]), @@ -3969,8 +4039,8 @@ do_format_exception(Reason={Error,Stack}) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% run_test_case_apply(CaseNum, Mod, Func, Args, Name, RunInit, -%% Where, MultiplyTimetrap) -> +%% run_test_case_apply(CaseNum, Mod, Func, Args, Name, RunInit, +%% Where, TimetrapData) -> %% {{Time,RetVal,Loc,Opts,Comment},DetectedFail,ProcessesBefore,ProcessesAfter} | %% {{died,Reason,unknown,Comment},DetectedFail,ProcessesBefore,ProcessesAfter} %% Name = atom() @@ -3984,24 +4054,24 @@ do_format_exception(Reason={Error,Stack}) -> %% ProcessesBefore = ProcessesAfter = integer() %% %% Where indicates if the test should run on target or always on the host. -%% -%% If test is to be run on target, and target is remote the request is +%% +%% If test is to be run on target, and target is remote the request is %% sent over socket to target, and test_server runs the case and sends the %% result back over the socket. Else test_server runs the case directly on host. -run_test_case_apply(CaseNum, Mod, Func, Args, Name, RunInit, host, MultiplyTimetrap) -> +run_test_case_apply(CaseNum, Mod, Func, Args, Name, RunInit, host, TimetrapData) -> test_server:run_test_case_apply({CaseNum,Mod,Func,Args,Name,RunInit, - MultiplyTimetrap}); -run_test_case_apply(CaseNum, Mod, Func, Args, Name, RunInit, target, MultiplyTimetrap) -> + TimetrapData}); +run_test_case_apply(CaseNum, Mod, Func, Args, Name, RunInit, target, TimetrapData) -> case get(test_server_ctrl_job_sock) of undefined -> %% local target test_server:run_test_case_apply({CaseNum,Mod,Func,Args,Name,RunInit, - MultiplyTimetrap}); + TimetrapData}); JobSock -> %% remote target request(JobSock, {test_case,{CaseNum,Mod,Func,Args,Name,RunInit, - MultiplyTimetrap}}), + TimetrapData}}), read_job_sock_loop(JobSock) end. @@ -4012,15 +4082,15 @@ run_test_case_apply(CaseNum, Mod, Func, Args, Name, RunInit, target, MultiplyTim %% Args = [term()] %% %% Just like io:format, except that depending on the Detail value, the output -%% is directed to console, major and/or minor log files. +%% is directed to console, major and/or minor log files. %% %% To handle printouts to common (not minor) log files from parallel test %% case processes, the test_server_common_io_handler value is checked. If %% set, the data is sent to the main controlling process. Note that test %% cases that belong to a conf group nested under a parallel group will also %% get its io data sent to main rather than immediately printed out, even -%% if the test cases are executed by the same, main, process (ie the main -%% process sends messages to itself then). +%% if the test cases are executed by the same, main, process (ie the main +%% process sends messages to itself then). %% %% Buffered io is handled by the handle_test_case_io_and_status/0 function. @@ -4040,21 +4110,21 @@ print_or_buffer(Detail, Msg, Printer) -> output({Detail,Msg}, Printer); MinLevel when is_number(Detail), Detail >= MinLevel -> output({Detail,Msg}, Printer); - _ -> % Detail < Minor | major | html + _ -> % Detail < Minor | major | html case get(test_server_common_io_handler) of - undefined -> + undefined -> output({Detail,Msg}, Printer); {_,MainPid} -> MainPid ! {print,self(),Detail,Msg} end - end. + end. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% print_timestamp(Detail, Leader) -> ok %% %% Prints Leader followed by a time stamp (date and time). Depending on %% the Detail value, the output is directed to console, major and/or minor -%% log files. +%% log files. print_timestamp(Detail, Leader) -> print(Detail, timestamp_get(Leader), []). @@ -4288,7 +4358,7 @@ update_config(Config, []) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% collect_cases(CurMod, TopCase, SkipList) -> +%% collect_cases(CurMod, TopCase, SkipList) -> %% BasicCaseList | {error,Reason} %% %% CurMod = atom() @@ -4319,18 +4389,18 @@ update_config(Config, []) -> %% are listed, and each Module:all(suite) is called %% {dir,Dir,Pattern} All modules <Pattern>_SUITE in the named dir %% are listed, and each Module:all(suite) is called -%% {conf,InitMF,Cases,FinMF} -%% {conf,Props,InitMF,Cases,FinMF} +%% {conf,InitMF,Cases,FinMF} +%% {conf,Props,InitMF,Cases,FinMF} %% InitMF is placed in the BasicCaseList, then %% Cases is treated according to this table, then %% FinMF is placed in the BasicCaseList. InitMF %% and FinMF are configuration manipulation %% functions. See below. -%% {make,InitMFA,Cases,FinMFA} +%% {make,InitMFA,Cases,FinMFA} %% InitMFA is placed in the BasicCaseList, then %% Cases is treated according to this table, then %% FinMFA is placed in the BasicCaseList. InitMFA -%% and FinMFA are make/unmake functions. If InitMFA +%% and FinMFA are make/unmake functions. If InitMFA %% fails, Cases are not run. InitMFA and FinMFA are %% always run on the host - not on target. %% @@ -4339,7 +4409,7 @@ update_config(Config, []) -> %% %% [] Leaf case %% {req,ReqList} Kept for backwards compatibility - same as [] -%% {req,ReqList,Cases} Kept for backwards compatibility - +%% {req,ReqList,Cases} Kept for backwards compatibility - %% Cases parsed recursively with collect_cases/3 %% Cases (list) Recursively parsed with collect_cases/3 %% @@ -4351,7 +4421,7 @@ update_config(Config, []) -> %% Configuration manipulation functions are called with the current %% configuration list as only argument, and are expected to return a new %% configuration list. Such a pair of function may, for example, start a -%% server and stop it after a serie of test cases. +%% server and stop it after a serie of test cases. %% %% SkipCases is expected to be in the format: %% @@ -4364,10 +4434,10 @@ update_config(Config, []) -> skip}). % skip list collect_all_cases(Top, Skip) when is_list(Skip) -> - Result = + Result = case collect_cases(Top, #cc{mod=[],skip=Skip}) of {ok,Cases,_St} -> Cases; - Other -> Other + Other -> Other end, Result. @@ -4379,12 +4449,12 @@ collect_cases([Case|Cs0], St0) -> case collect_cases(Cs0, St1) of {ok,FlatCases2,St} -> {ok,FlatCases1 ++ FlatCases2,St}; - {error,_Reason}=Error -> Error + {error,_Reason} = Error -> Error end; - {error,_Reason}=Error -> Error + {error,_Reason} = Error -> Error end; - + collect_cases({module,Case}, St) when is_atom(Case), is_atom(St#cc.mod) -> collect_case({St#cc.mod,Case}, St); collect_cases({module,Mod,Case}, St) -> @@ -4404,38 +4474,71 @@ collect_cases({conf,InitMF,CaseList,FinF}, St) when is_atom(FinF) -> collect_cases({conf,InitMF,CaseList,FinMF}, St0) -> collect_cases({conf,[],InitMF,CaseList,FinMF}, St0); collect_cases({conf,Props,InitF,CaseList,FinMF}, St) when is_atom(InitF) -> - collect_cases({conf,Props,{St#cc.mod,InitF},CaseList,FinMF}, St); + case init_props(Props) of + {error,_} -> + {ok,[],St}; + Props1 -> + collect_cases({conf,Props1,{St#cc.mod,InitF},CaseList,FinMF}, St) + end; collect_cases({conf,Props,InitMF,CaseList,FinF}, St) when is_atom(FinF) -> - collect_cases({conf,Props,InitMF,CaseList,{St#cc.mod,FinF}}, St); -collect_cases({conf,Props,InitMF,CaseList,FinMF}, St0) -> - case collect_cases(CaseList, St0) of - {ok,[],_St}=Empty -> - Empty; - {ok,FlatCases,St} -> + case init_props(Props) of + {error,_} -> + {ok,[],St}; + Props1 -> + collect_cases({conf,Props1,InitMF,CaseList,{St#cc.mod,FinF}}, St) + end; +collect_cases({conf,Props,InitMF,CaseList,FinMF} = Conf, St) -> + case init_props(Props) of + {error,_} -> + {ok,[],St}; + Props1 -> Ref = make_ref(), - case in_skip_list(InitMF, St#cc.skip) of - {true,Comment} -> - {ok,[{skip_case,{conf,Ref,InitMF,Comment}} | - FlatCases ++ [{conf,Ref,[],FinMF}]],St}; + Skips = St#cc.skip, + case in_skip_list({St#cc.mod,Conf}, Skips) of + {true,Comment} -> % conf init skipped + {ok,[{skip_case,{conf,Ref,InitMF,Comment}} | + [] ++ [{conf,Ref,[],FinMF}]],St}; + {true,Name,Comment} when is_atom(Name) -> % all cases skipped + {ok,[{skip_case,{{St#cc.mod,{group,Name}},Comment}}],St}; + {true,ToSkip,_} when is_list(ToSkip) -> % some cases skipped + case collect_cases(CaseList, + St#cc{skip=ToSkip++Skips}) of + {ok,[],_St} = Empty -> + Empty; + {ok,FlatCases,St1} -> + {ok,[{conf,Ref,Props1,InitMF} | + FlatCases ++ [{conf,Ref, + keep_name(Props1), + FinMF}]],St1#cc{skip=Skips}}; + {error,_Reason} = Error -> + Error + end; false -> - {ok,[{conf,Ref,Props,InitMF} | - FlatCases ++ [{conf,Ref,keep_name(Props),FinMF}]],St} - end; - {error,_Reason}=Error -> - Error + case collect_cases(CaseList, St) of + {ok,[],_St} = Empty -> + Empty; + {ok,FlatCases,St1} -> + {ok,[{conf,Ref,Props1,InitMF} | + FlatCases ++ [{conf,Ref, + keep_name(Props1), + FinMF}]],St1}; + {error,_Reason} = Error -> + Error + end + end end; collect_cases({make,InitMFA,CaseList,FinMFA}, St0) -> case collect_cases(CaseList, St0) of - {ok,[],_St}=Empty -> Empty; + {ok,[],_St} = Empty -> Empty; {ok,FlatCases,St} -> Ref = make_ref(), - {ok,[{make,Ref,InitMFA}|FlatCases ++ + {ok,[{make,Ref,InitMFA}|FlatCases ++ [{make,Ref,FinMFA}]],St}; - {error,_Reason}=Error -> Error + {error,_Reason} = Error -> Error end; -collect_cases({Module, Cases}, St) when is_list(Cases) -> +collect_cases({Module, Cases}, St) when is_list(Cases) -> case (catch collect_case(Cases, St#cc{mod=Module}, [])) of {ok, NewCases, NewSt} -> {ok, NewCases, NewSt}; @@ -4450,8 +4553,11 @@ collect_cases({_Mod,_Case,_Args}=Spec, St) -> collect_case(Spec, St); collect_cases(Case, St) when is_atom(Case), is_atom(St#cc.mod) -> collect_case({St#cc.mod,Case}, St); -collect_cases(Other, _St) -> - {error,{bad_subtest_spec,Other}}. +collect_cases(Other, St) -> + {error,{bad_subtest_spec,St#cc.mod,Other}}. + +collect_case({Mod,{conf,_,_,_,_}=Conf}, St) -> + collect_case_invoke(Mod, Conf, [], St); collect_case(MFA, St) -> case in_skip_list(MFA, St#cc.skip) of @@ -4475,9 +4581,9 @@ collect_case_invoke(Mod, Case, MFA, St) -> case os:getenv("TEST_SERVER_FRAMEWORK") of false -> case catch apply(Mod, Case, [suite]) of - {'EXIT',_} -> + {'EXIT',_} -> {ok,[MFA],St}; - Suite -> + Suite -> collect_subcases(Mod, Case, MFA, St, Suite) end; _ -> @@ -4485,9 +4591,10 @@ collect_case_invoke(Mod, Case, MFA, St) -> collect_subcases(Mod, Case, MFA, St, Suite) end. -collect_subcases(Mod, Case, MFA, St, Suite) -> +collect_subcases(Mod, Case, MFA, St, Suite) -> case Suite of [] when Case == all -> {ok,[],St}; + [] when element(1, Case) == conf -> {ok,[],St}; [] -> {ok,[MFA],St}; %%%! --- START Kept for backwards compatibilty --- %%%! Requirements are not used @@ -4498,6 +4605,8 @@ collect_subcases(Mod, Case, MFA, St, Suite) -> %%%! --- END Kept for backwards compatibilty --- {Skip,Reason} when Skip==skip; Skip==skipped -> {ok,[{skip_case,{MFA,Reason}}],St}; + {error,Reason} -> + throw(Reason); SubCases -> collect_case_subcases(Mod, Case, SubCases, St) end. @@ -4536,7 +4645,7 @@ collect_case_deny(Mod, Case, MFA, ReqList, SubCases, St) -> {granted,SubCases} -> collect_case_subcases(Mod, Case, SubCases, St) end. - + check_deny([Req|Reqs], DenyList) -> case check_deny_req(Req, DenyList) of {denied,_Comment}=Denied -> Denied; @@ -4559,8 +4668,49 @@ check_deny_req(Req, DenyList) -> false -> granted end. +in_skip_list({Mod,{conf,Props,InitMF,_CaseList,_FinMF}}, SkipList) -> + case in_skip_list(InitMF, SkipList) of + {true,_} = Yes -> + Yes; + _ -> + case proplists:get_value(name, Props) of + undefined -> + false; + Name -> + ToSkip = + lists:flatmap( + fun({M,{conf,SProps,_,SCaseList,_},Cmt}) when + M == Mod -> + case proplists:get_value(name, SProps) of + all -> + [{M,all,Cmt}]; + Name -> + case SCaseList of + all -> + [{M,all,Cmt}]; + _ -> + [{M,F,Cmt} || F <- SCaseList] + end; + _ -> + [] + end; + (_) -> + [] + end, SkipList), + case ToSkip of + [] -> + false; + _ -> + case lists:keysearch(all, 2, ToSkip) of + {value,{_,_,Cmt}} -> {true,Name,Cmt}; + _ -> {true,ToSkip,""} + end + end + end + end; + in_skip_list({Mod,Func,_Args}, SkipList) -> - in_skip_list({Mod,Func}, SkipList); + in_skip_list({Mod,Func}, SkipList); in_skip_list({Mod,Func}, [{Mod,Funcs,Comment}|SkipList]) when is_list(Funcs) -> case lists:member(Func, Funcs) of true -> @@ -4577,9 +4727,21 @@ in_skip_list({Mod,Func}, [_|SkipList]) -> in_skip_list(_, []) -> false. +%% remove unnecessary properties +init_props(Props) -> + case get_repeat(Props) of + Repeat = {_RepType,N} when N < 2 -> + if N == 0 -> + {error,{invalid_property,Repeat}}; + true -> + lists:delete(Repeat, Props) + end; + _ -> + Props + end. + keep_name(Props) -> lists:filter(fun({name,_}) -> true; (_) -> false end, Props). - %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Target node handling functions %% @@ -4615,13 +4777,13 @@ start_node(Name, Type, Options) -> end, case Warning of [] -> ok; - _ -> + _ -> format(1, Warning), format(minor, Warning) end, {ok, Nodename}; {fail,{Ret, Host, Cmd}} -> - format(minor, + format(minor, "Failed to start node ~p on ~p with command: ~p~n" "Reason: ~p", [Name, Host, Cmd, Ret]), @@ -4630,7 +4792,7 @@ start_node(Name, Type, Options) -> format(minor, "Failed to start node ~p: ~p", [Name,Ret]), Ret; {Ret, Host, Cmd} -> - format(minor, + format(minor, "Failed to start node ~p on ~p with command: ~p~n" "Reason: ~p", [Name, Host, Cmd, Ret]), @@ -4685,7 +4847,7 @@ read_job_sock_loop(Sock) -> exit({controller,connection_lost,Reason}); {ok,<<1,Request/binary>>} -> case decode(binary_to_term(Request)) of - ok -> + ok -> read_job_sock_loop(Sock); {stop,Result} -> Result @@ -4695,14 +4857,14 @@ read_job_sock_loop(Sock) -> decode({apply,{M,F,A}}) -> apply(M,F,A), ok; -decode({sync_apply,{M,F,A}}) -> +decode({sync_apply,{M,F,A}}) -> R = apply(M,F,A), request(get(test_server_ctrl_job_sock),{sync_result,R}), ok; decode({sync_result,Result}) -> {stop,Result}; decode({test_case_result,Result}) -> - {stop,Result}; + {stop,Result}; decode({privdir,empty_priv_dir}) -> {stop,ok}; decode({{privdir,PrivDirTar},TarBin}) -> @@ -4742,7 +4904,7 @@ p({A,B,C}) -> p(X) -> pinfo(X). -t() -> +t() -> t(wall_clock). t(X) -> element(1, statistics(X)). @@ -4781,7 +4943,7 @@ display_info([Pid|T], R, M) -> Other -> Other end, - Reds = fetch(reductions, Info), + Reds = fetch(reductions, Info), LM = length(fetch(messages, Info)), pformat(io_lib:format("~w", [Pid]), io_lib:format("~w", [Call]), @@ -4822,12 +4984,12 @@ pinfo(P) -> %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% %% A module is included in the cover analysis if -%% - it belongs to the tested application and is not listed in the +%% - it belongs to the tested application and is not listed in the %% {exclude,List} part of the App.cover file %% - it does not belong to the application, but is listed in the %% {include,List} part of the App.cover file -%% - it does not belong to the application, but is listed in the -%% cross.cover file (in the test_server application) under 'all' +%% - it does not belong to the application, but is listed in the +%% cross.cover file (in the test_server application) under 'all' %% or under the tested application. %% %% The modules listed in the cross.cover file are modules that are @@ -4893,7 +5055,7 @@ read_cover_file(CoverFile) -> io:fwrite("Faulty format of CoverFile ~p\n", [CoverFile]), {[],[]} end; - {error,Reason} -> + {error,Reason} -> io:fwrite("Can't read CoverFile ~p\nReason: ~p\n", [CoverFile,Reason]), {[],[]} @@ -4958,7 +5120,7 @@ cover_analyse({App,CoverInfo}, Analyse, AnalyseMods, TestDir) -> end, io:fwrite(CoverLog, "<p>Excluded module(s): <code>~p</code>\n", [Excluded]), - + Coverage = cover_analyse(Analyse, AnalyseMods), case lists:filter(fun({_M,{_,_,_}}) -> false; @@ -4968,7 +5130,7 @@ cover_analyse({App,CoverInfo}, Analyse, AnalyseMods, TestDir) -> ok; Bad -> io:fwrite(CoverLog, "<p>Analysis failed for ~w module(s): " - "<code>~w</code>\n", + "<code>~w</code>\n", [length(Bad),[BadM || {BadM,{_,_Why}} <- Bad]]) end, @@ -5002,10 +5164,10 @@ cross_cover_analyse(Analyse, CrossModules) -> CoverdataFiles = get_coverdata_files(), lists:foreach(fun(CDF) -> cover:import(CDF) end, CoverdataFiles), io:fwrite("Cover analysing... ", []), - DetailsFun = + DetailsFun = case Analyse of details -> - fun(Dir,M) -> + fun(Dir,M) -> OutFile = filename:join(Dir, atom_to_list(M) ++ ".CROSS_COVER.html"), @@ -5018,7 +5180,7 @@ cross_cover_analyse(Analyse, CrossModules) -> SortedModules = case CrossModules of undefined -> - sort_modules([Mod || Mod <- get_all_cross_modules(), + sort_modules([Mod || Mod <- get_all_cross_modules(), lists:member(Mod, cover:imported_modules())], []); _ -> sort_modules(CrossModules, []) @@ -5031,7 +5193,7 @@ cross_cover_analyse(Analyse, CrossModules) -> %% cross.cover, write a cross cover log (cross_cover.html). write_cross_cover_logs([{App,Coverage}|T]) -> case last_test_for_app(App) of - false -> + false -> ok; Dir -> CoverLogName = filename:join(Dir,?cross_coverlog_name), @@ -5045,13 +5207,13 @@ write_cross_cover_logs([{App,Coverage}|T]) -> end, write_cross_cover_logs(T); write_cross_cover_logs([]) -> - io:fwrite("done\n", []). + io:fwrite("done\n", []). %% Find all exported coverdata files. First find all the latest %% run.<timestamp> directories, and the check if there is a file named %% all.coverdata. get_coverdata_files() -> - PossibleFiles = [last_coverdata_file(Dir) || + PossibleFiles = [last_coverdata_file(Dir) || Dir <- filelib:wildcard([$*|?logdir_ext]), filelib:is_dir(Dir)], [File || File <- PossibleFiles, filelib:is_file(File)]. @@ -5074,12 +5236,12 @@ last_test([_|Rest], Latest) -> last_test(Rest, Latest); last_test([], Latest) -> Latest. - + %% Sort modules according to the application they belong to. %% Return [{App,LastTestDir,ModuleList}] sort_modules([M|Modules], Acc) -> App = get_app(M), - Acc1 = + Acc1 = case lists:keysearch(App, 1, Acc) of {value,{App,LastTest,List}} -> lists:keyreplace(App, 1, Acc, {App,LastTest,[M|List]}); @@ -5120,9 +5282,9 @@ get_all_cross_modules() -> get_cross_modules(all). get_cross_modules(App) -> case file:consult(?cross_cover_file) of - {ok,List} -> + {ok,List} -> get_cross_modules(App, List, []); - _X -> + _X -> [] end. @@ -5134,11 +5296,11 @@ get_cross_modules(App, [_H|T], Acc) -> get_cross_modules(App, T, Acc); get_cross_modules(_App, [], Acc) -> Acc. - + %% Support functions for writing the cover logs (both cross and normal) write_coverlog_header(CoverLog) -> - case catch + case catch io:fwrite(CoverLog, "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n" "<!-- autogenerated by '~w'. -->\n" @@ -5162,13 +5324,13 @@ format_analyse(M,Cov,NotCov,undefined) -> io_lib:fwrite("<tr><td>~w</td>" "<td align=right>~w %</td>" "<td align=right>~w</td>" - "<td align=right>~w</td></tr>\n", + "<td align=right>~w</td></tr>\n", [M,pc(Cov,NotCov),Cov,NotCov]); format_analyse(M,Cov,NotCov,{file,File}) -> io_lib:fwrite("<tr><td><a href=\"~s\">~w</a></td>" "<td align=right>~w %</td>" "<td align=right>~w</td>" - "<td align=right>~w</td></tr>\n", + "<td align=right>~w</td></tr>\n", [filename:basename(File),M,pc(Cov,NotCov),Cov,NotCov]); format_analyse(M,Cov,NotCov,{lines,Lines}) -> CoverOutName = atom_to_list(M)++".COVER.html", @@ -5177,15 +5339,15 @@ format_analyse(M,Cov,NotCov,{lines,Lines}) -> io_lib:fwrite("<tr><td><a href=\"~s\">~w</a></td>" "<td align=right>~w %</td>" "<td align=right>~w</td>" - "<td align=right>~w</td></tr>\n", + "<td align=right>~w</td></tr>\n", [CoverOutName,M,pc(Cov,NotCov),Cov,NotCov]); format_analyse(M,Cov,NotCov,{error,_}) -> io_lib:fwrite("<tr><td>~w</td>" "<td align=right>~w %</td>" "<td align=right>~w</td>" - "<td align=right>~w</td></tr>\n", + "<td align=right>~w</td></tr>\n", [M,pc(Cov,NotCov),Cov,NotCov]). - + pc(0,0) -> 0; @@ -5200,9 +5362,9 @@ write_not_covered(CoverOut,M,Lines) -> "<table border=3 cellpadding=5>\n" "<th>Line Number</th>\n", [M]), - lists:foreach(fun({{_M,Line},{0,1}}) -> + lists:foreach(fun({{_M,Line},{0,1}}) -> io:fwrite(CoverOut,"<tr><td>~w</td></tr>\n", [Line]); - (_) -> + (_) -> ok end, Lines), @@ -5216,7 +5378,7 @@ write_default_coverlog(TestDir) -> file:close(CoverLog). write_default_cross_coverlog(TestDir) -> - {ok,CrossCoverLog} = + {ok,CrossCoverLog} = file:open(filename:join(TestDir,?cross_coverlog_name), [write]), write_coverlog_header(CrossCoverLog), io:fwrite(CrossCoverLog, @@ -5232,7 +5394,7 @@ write_cover_result_table(CoverLog,Coverage) -> "<th>Not covered (Lines)</th>\n", []), {TotCov,TotNotCov} = - lists:foldl(fun({M,{Cov,NotCov,Details}},{AccCov,AccNotCov}) -> + lists:foldl(fun({M,{Cov,NotCov,Details}},{AccCov,AccNotCov}) -> Str = format_analyse(M,Cov,NotCov,Details), io:fwrite(CoverLog,"~s", [Str]), {AccCov+Cov,AccNotCov+NotCov}; diff --git a/lib/test_server/src/test_server_sup.erl b/lib/test_server/src/test_server_sup.erl index 89edb0f881..625724fbb5 100644 --- a/lib/test_server/src/test_server_sup.erl +++ b/lib/test_server/src/test_server_sup.erl @@ -21,7 +21,7 @@ %%% Purpose: Test server support functions. %%%------------------------------------------------------------------- -module(test_server_sup). --export([timetrap/2, timetrap_cancel/1, capture_get/1, messages_get/1, +-export([timetrap/2, timetrap/3, timetrap_cancel/1, capture_get/1, messages_get/1, timecall/3, call_crash/5, app_test/2, check_new_crash_dumps/0, cleanup_crash_dumps/0, crash_dump_dir/0, tar_crash_dumps/0, get_username/0, get_os_family/0, @@ -34,16 +34,23 @@ -define(src_listing_ext, ".src.html"). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -%% timetrap(Timeout,Pid) -> Handle +%% timetrap(Timeout,Scale,Pid) -> Handle %% Handle = term() %% %% Creates a time trap, that will kill the given process if the %% trap is not cancelled with timetrap_cancel/1, within Timeout %% milliseconds. +%% Scale says if the time should be scaled up to compensate for +%% delays during the test (e.g. if cover is running). timetrap(Timeout0, Pid) -> + timetrap(Timeout0, true, Pid). + +timetrap(Timeout0, Scale, Pid) -> process_flag(priority, max), - Timeout = test_server:timetrap_scale_factor() * Timeout0, + Timeout = if not Scale -> Timeout0; + true -> test_server:timetrap_scale_factor() * Timeout0 + end, receive after trunc(Timeout) -> Line = test_server:get_loc(Pid), @@ -497,6 +504,7 @@ framework_call(Callback,Func,Args,DefaultReturn) -> end, case erlang:function_exported(Mod,Func,length(Args)) of true -> + put(test_server_loc, {Mod,Func,framework}), EH = fun(Reason) -> exit({fw_error,{Mod,Func,Reason}}) end, try apply(Mod,Func,Args) of Result -> diff --git a/lib/test_server/src/ts_erl_config.erl b/lib/test_server/src/ts_erl_config.erl index 5cdbf0fbb8..14c36a2c35 100644 --- a/lib/test_server/src/ts_erl_config.erl +++ b/lib/test_server/src/ts_erl_config.erl @@ -70,18 +70,18 @@ dl_vars(Vars, _) -> ShlibRules = ts_lib:subst(ShlibRules0, Vars), [{'SHLIB_RULES', ShlibRules}|Vars]. -erts_lib_name(multi_threaded, win32) -> +erts_lib_name(multi_threaded, {win32, V}) -> link_library("erts_MD" ++ case is_debug_build() of true -> "d"; false -> "" end, - win32); -erts_lib_name(single_threaded, win32) -> + {win32, V}); +erts_lib_name(single_threaded, {win32, V}) -> link_library("erts_ML" ++ case is_debug_build() of true -> "d"; false -> "" end, - win32); + {win32, V}); erts_lib_name(multi_threaded, OsType) -> link_library("erts_r", OsType); erts_lib_name(single_threaded, OsType) -> @@ -349,10 +349,7 @@ sock_libraries({unix, _}) -> sock_libraries(vxworks) -> ""; sock_libraries(ose) -> - ""; -sock_libraries(_Other) -> - exit({sock_libraries, not_supported}). - + "". link_library(LibName,{win32, _}) -> LibName ++ ".lib"; diff --git a/lib/test_server/test/test_server_SUITE.erl b/lib/test_server/test/test_server_SUITE.erl index dfe1028d3a..0563e1104f 100644 --- a/lib/test_server/test/test_server_SUITE.erl +++ b/lib/test_server/test/test_server_SUITE.erl @@ -183,7 +183,7 @@ multiply_timetrap(suite) -> []; multiply_timetrap(doc) -> ["Test multiply timetrap"]; multiply_timetrap(Config) when is_list(Config) -> %% This simulates the call to test_server_ctrl:multiply_timetraps/1: - put(test_server_multiply_timetraps,2), + put(test_server_multiply_timetraps,{2,true}), Dog = ?t:timetrap(500), timer:sleep(800), diff --git a/lib/test_server/vsn.mk b/lib/test_server/vsn.mk index 3c6efeffde..4c3df28814 100644 --- a/lib/test_server/vsn.mk +++ b/lib/test_server/vsn.mk @@ -1,2 +1,2 @@ -TEST_SERVER_VSN = 3.3.7 +TEST_SERVER_VSN = 3.4.1 diff --git a/lib/tools/doc/src/eprof.xml b/lib/tools/doc/src/eprof.xml index ae1033f2d0..6d68c90768 100644 --- a/lib/tools/doc/src/eprof.xml +++ b/lib/tools/doc/src/eprof.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1996</year><year>2009</year> + <year>1996</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>eprof</title> @@ -35,8 +35,7 @@ used. The profiling is done using the Erlang trace BIFs. Tracing of local function calls for a specified set of processes is enabled when profiling is begun, and disabled when profiling is stopped.</p> - <p>When using Eprof, expect a significant slowdown in program execution, - in most cases at least 100 percent.</p> + <p>When using Eprof expect a slowdown in program execution.</p> </description> <funcs> <func> @@ -47,15 +46,19 @@ <v>Reason = {already_started,Pid}</v> </type> <desc> - <p>Starts the Eprof server which owns the Eprof internal database.</p> + <p>Starts the Eprof server which holds the internal state of the collected data.</p> </desc> </func> <func> - <name>start_profiling(Rootset) -> profiling | error</name> - <name>profile(Rootset) -> profiling | error</name> + <name>start_profiling(Rootset) -> profiling | {error, Reason}</name> + <name>start_profiling(Rootset,Pattern) -> profiling | {error, Reason}</name> <fsummary>Start profiling.</fsummary> <type> <v>Rootset = [atom() | pid()]</v> + <v>Pattern = {Module, Function, Arity}</v> + <v>Module = Function = atom()</v> + <v>Arity = integer()</v> + <v>Reason = term()</v> </type> <desc> <p>Starts profiling for the processes in <c>Rootset</c> (and any new @@ -64,6 +67,9 @@ <p><c>Rootset</c> is a list of pids and registered names.</p> <p>The function returns <c>profiling</c> if tracing could be enabled for all processes in <c>Rootset</c>, or <c>error</c> otherwise.</p> + <p>A pattern can be selected to narrow the profiling. For instance ca a specific + module be selected and only the code processes executes in that module will be + profiled.</p> </desc> </func> <func> @@ -75,14 +81,20 @@ </desc> </func> <func> - <name>profile(Rootset,Fun) -> {ok,Value} | {error,Reason} | error</name> - <name>profile(Rootset,Module,Function,Args) -> {ok,Value} | {error,Reason} | error</name> + <name>profile(Fun) -> profiling | {error, Reason}</name> + <name>profile(Rootset) -> profiling | {error, Reason}</name> + <name>profile(Rootset,Fun) -> {ok, Value} | {error,Reason}</name> + <name>profile(Rootset,Fun,Pattern) -> {ok, Value} | {error, Reason}</name> + <name>profile(Rootset,Module,Function,Args) -> {ok, Value} | {error, Reason}</name> + <name>profile(Rootset,Module,Function,Args,Pattern) -> {ok, Value} | {error, Reason}</name> <fsummary>Start profiling.</fsummary> <type> <v>Rootset = [atom() | pid()]</v> <v>Fun = fun() -> term()</v> + <v>Pattern = {Module, Function, Arity}</v> <v>Module = Function = atom()</v> <v>Args = [term()]</v> + <v>Arity = integer()</v> <v>Value = Reason = term()</v> </type> <desc> @@ -96,7 +108,7 @@ <c>Rootset</c>, the function returns <c>{ok,Value}</c> when <c>Fun()</c>/<c>apply</c> returns with the value <c>Value</c>, or <c>{error,Reason}</c> if <c>Fun()</c>/<c>apply</c> fails with - exit reason <c>Reason</c>. Otherwise it returns <c>error</c> + exit reason <c>Reason</c>. Otherwise it returns <c>{error, Reason}</c> immediately.</p> <p>The programmer must ensure that the function given as argument is truly synchronous and that no work continues after @@ -104,7 +116,15 @@ </desc> </func> <func> - <name>analyse()</name> + <name>analyze() -> ok</name> + <name>analyze(Type) -> ok</name> + <name>analyze(Type,Options) -> ok</name> + <type> + <v>Type = procs | total</v> + <v>Options = [{filter, Filter} | {sort, Sort}</v> + <v>Filter = [{calls, integer()} | {time, float()}]</v> + <v>Sort = time | calls | mfa</v> + </type> <fsummary>Display profiling results per process.</fsummary> <desc> <p>Call this function when profiling has been stopped to display @@ -113,17 +133,10 @@ <item>how much time has been used by each process, and</item> <item>in which function calls this time has been spent.</item> </list> - <p>Time is shown as percentage of total time, not as absolute time.</p> - </desc> - </func> - <func> - <name>total_analyse()</name> - <fsummary>Display profiling results per function call.</fsummary> - <desc> - <p>Call this function when profiling has been stopped to display + <p>Call <c>analyze</c> with <c>total</c> option when profiling has been stopped to display the results per function call, that is in which function calls the time has been spent.</p> - <p>Time is shown as percentage of total time, not as absolute time.</p> + <p>Time is shown as percentage of total time and as absolute time.</p> </desc> </func> <func> diff --git a/lib/tools/doc/src/notes.xml b/lib/tools/doc/src/notes.xml index e7d1ae150c..b9e1ef2959 100644 --- a/lib/tools/doc/src/notes.xml +++ b/lib/tools/doc/src/notes.xml @@ -30,6 +30,76 @@ </header> <p>This document describes the changes made to the Tools application.</p> +<section><title>Tools 2.6.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>A race condition affecting Cover has been removed.</p> + <p> + Own Id: OTP-8469</p> + </item> + <item> + <p> + Emacs improvements:</p> + <p> + Fixed emacs-mode installation problems.</p> + <p> + Fixed a couple of -spec and -type indentation and + font-lock problems.</p> + <p> + Fixed error messages on emacs-21.</p> + <p> + Magnus Henoch fixed several issues.</p> + <p> + Ralf Doering, Klas Johansson and Chris Bernard + contributed various emacs-eunit improvements.</p> + <p> + Klas Johansson and Dave Peticolas added emacs-flymake + support.</p> + <p> + Own Id: OTP-8530</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Xref has been updated to use the <c>re</c> module + instead of the deprecated <c>regexp</c> module.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8472</p> + </item> + <item> + <p>When given the option <c>{builtins,true}</c> Xref now + adds calls to operators.</p> + <p> + Own Id: OTP-8647</p> + </item> + <item> + <p><c>eprof</c> has been reimplemented with support in + the Erlang virtual machine and is now both faster (i.e. + slows down the code being measured less) and scales much + better. In measurements we saw speed-ups compared to the + old eprof ranging from 6 times (for sequential code that + only uses one scheduler/core) up to 84 times (for + parallel code that uses 8 cores).</p> + <p>Note: The API for the <c>eprof</c> has been cleaned up + and extended. See the documentation.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8706</p> + </item> + </list> + </section> + +</section> + <section><title>Tools 2.6.5.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/tools/src/eprof.erl b/lib/tools/src/eprof.erl index b4313d6888..f7c1b76364 100644 --- a/lib/tools/src/eprof.erl +++ b/lib/tools/src/eprof.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 1996-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 1996-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% %% Purpose: Profile a system in order to figure out where the @@ -23,456 +23,464 @@ -module(eprof). -behaviour(gen_server). --export([start/0, stop/0, dump/0, total_analyse/0, - start_profiling/1, profile/2, profile/4, profile/1, - stop_profiling/0, analyse/0, log/1]). +-export([start/0, + stop/0, + dump/0, + start_profiling/1, start_profiling/2, + profile/1, profile/2, profile/3, profile/4, profile/5, + stop_profiling/0, + analyze/0, analyze/1, analyze/2, + log/1]). %% Internal exports -export([init/1, - call/4, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). +-record(bpd, { + n = 0, % number of total calls + us = 0, % sum of uS for all calls + p = gb_trees:empty(), % tree of {Pid, {Mfa, {Count, Us}}} + mfa = [] % list of {Mfa, {Count, Us}} + }). + +-record(state, { + profiling = false, + pattern = {'_','_','_'}, + rootset = [], + fd = undefined, + start_ts = undefined, + reply = undefined, + bpd = #bpd{} + }). + + + +%% -------------------------------------------------------------------- %% +%% +%% API +%% +%% -------------------------------------------------------------------- %% --include_lib("stdlib/include/qlc.hrl"). - --import(lists, [flatten/1,reverse/1,keysort/2]). - - --record(state, {table = notable, - proc = noproc, - profiling = false, - pfunc = undefined, - pop = running, - ptime = 0, - overhead = 0, - rootset = []}). - -%%%%%%%%%%%%%% - -start() -> gen_server:start({local, eprof}, eprof, [], []). -stop() -> gen_server:call(eprof, stop, infinity). +start() -> gen_server:start({local, ?MODULE}, ?MODULE, [], []). +stop() -> gen_server:call(?MODULE, stop, infinity). +profile(Fun) when is_function(Fun) -> + profile([], Fun); +profile(Rs) when is_list(Rs) -> + start_profiling(Rs). profile(Pids, Fun) -> - start(), - gen_server:call(eprof, {profile,Pids,erlang,apply,[Fun,[]]},infinity). + profile(Pids, Fun, {'_','_','_'}). + +profile(Pids, Fun, Pattern) -> + profile(Pids, erlang, apply, [Fun,[]], Pattern). profile(Pids, M, F, A) -> + profile(Pids, M, F, A, {'_','_','_'}). + +profile(Pids, M, F, A, Pattern) -> start(), - gen_server:call(eprof, {profile,Pids,M,F,A},infinity). + gen_server:call(?MODULE, {profile,Pids,Pattern,M,F,A},infinity). dump() -> - gen_server:call(eprof, dump, infinity). + gen_server:call(?MODULE, dump, infinity). -analyse() -> - gen_server:call(eprof, analyse, infinity). +analyze() -> + analyze(procs). -log(File) -> - gen_server:call(eprof, {logfile, File}, infinity). +analyze(Type) when is_atom(Type) -> + analyze(Type, []); +analyze(Opts) when is_list(Opts) -> + analyze(procs, Opts). +analyze(Type, Opts) when is_list(Opts) -> + gen_server:call(?MODULE, {analyze, Type, Opts}, infinity). -total_analyse() -> - gen_server:call(eprof, total_analyse, infinity). +log(File) -> + gen_server:call(?MODULE, {logfile, File}, infinity). start_profiling(Rootset) -> + start_profiling(Rootset, {'_','_','_'}). +start_profiling(Rootset, Pattern) -> start(), - gen_server:call(eprof, {profile, Rootset}, infinity). + gen_server:call(?MODULE, {profile, Rootset, Pattern}, infinity). stop_profiling() -> - gen_server:call(eprof, stop_profiling, infinity). + gen_server:call(?MODULE, stop_profiling, infinity). -profile(Rs) -> - start_profiling(Rs). -%%%%%%%%%%%%%%%% +%% -------------------------------------------------------------------- %% +%% +%% init +%% +%% -------------------------------------------------------------------- %% -init(_) -> +init([]) -> process_flag(trap_exit, true), - process_flag(priority, max), - put(three_one, {3,1}), %To avoid building garbage. {ok, #state{}}. -subtr({X1,Y1,Z1}, {X1,Y1,Z2}) -> - Z1 - Z2; -subtr({X1,Y1,Z1}, {X2,Y2,Z2}) -> - (((X1-X2) * 1000000) + Y1 - Y2) * 1000000 + Z1 - Z2. +%% -------------------------------------------------------------------- %% +%% +%% handle_call +%% +%% -------------------------------------------------------------------- %% -update_call_statistics(Tab, Key, Time) -> - try ets:update_counter(Tab, Key, Time) of - NewTime when is_integer(NewTime) -> - ets:update_counter(Tab, Key, get(three_one)) - catch - error:badarg -> - ets:insert(Tab, {Key,Time,1}) - end. +%% analyze -update_other_statistics(Tab, Key, Time) -> - try - ets:update_counter(Tab, Key, Time) - catch - error:badarg -> - ets:insert(Tab, {Key,Time,0}) - end. +handle_call({analyze, _, _}, _, #state{ bpd = #bpd{ p = {0,nil}, us = 0, n = 0} = Bpd } = S) when is_record(Bpd, bpd) -> + {reply, nothing_to_analyze, S}; -do_messages({trace_ts,From,Op,Mfa,Time}, Tab, undefined,_PrevOp0,_PrevTime0) -> - PrevFunc = [From|Mfa], - receive - {trace_ts,_,_,_,_}=Ts -> do_messages(Ts, Tab, PrevFunc, Op, Time) - after 0 -> - {PrevFunc,Op,Time} - end; -do_messages({trace_ts,From,Op,Mfa,Time}, Tab, PrevFunc0, call, PrevTime0) -> - update_call_statistics(Tab, PrevFunc0, subtr(Time, PrevTime0)), - PrevFunc = case Op of - exit -> undefined; - out -> undefined; - _ -> [From|Mfa] - end, - receive - {trace_ts,_,_,_,_}=Ts -> do_messages(Ts, Tab, PrevFunc, Op, Time) - after 0 -> - {PrevFunc,Op,Time} - end; -do_messages({trace_ts,From,Op,Mfa,Time}, Tab, PrevFunc0, _PrevOp0, PrevTime0) -> - update_other_statistics(Tab, PrevFunc0, subtr(Time, PrevTime0)), - PrevFunc = case Op of - exit -> undefined; - out -> undefined; - _ -> [From|Mfa] - end, - receive - {trace_ts,_,_,_,_}=Ts -> do_messages(Ts, Tab, PrevFunc, Op, Time) - after 0 -> - {PrevFunc,Op,Time} - end. +handle_call({analyze, procs, Opts}, _, #state{ bpd = #bpd{ p = Ps, us = Tus} = Bpd, fd = Fd} = S) when is_record(Bpd, bpd) -> + lists:foreach(fun + ({Pid, Mfas}) -> + {Pn, Pus} = sum_bp_total_n_us(Mfas), + format(Fd, "~n****** Process ~w -- ~s % of profiled time *** ~n", [Pid, s("~.2f", [100.0*(Pus/Tus)])]), + print_bp_mfa(Mfas, {Pn,Pus}, Fd, Opts), + ok + end, gb_trees:to_list(Ps)), + {reply, ok, S}; -%%%%%%%%%%%%%%%%%% +handle_call({analyze, total, Opts}, _, #state{ bpd = #bpd{ mfa = Mfas, n = Tn, us = Tus} = Bpd, fd = Fd} = S) when is_record(Bpd, bpd) -> + print_bp_mfa(Mfas, {Tn, Tus}, Fd, Opts), + {reply, ok, S}; -handle_cast(_Req, S) -> {noreply, S}. +handle_call({analyze, Type, _Opts}, _, S) -> + {reply, {error, {undefined, Type}}, S}; -terminate(_Reason,_S) -> - call_trace_for_all(false), - normal. +%% profile -%%%%%%%%%%%%%%%%%% +handle_call({profile, _Rootset, _Pattern, _M,_F,_A}, _From, #state{ profiling = true } = S) -> + {reply, {error, already_profiling}, S}; -handle_call({logfile, F}, _FromTag, Status) -> - case file:open(F, [write]) of - {ok, Fd} -> - case get(fd) of - undefined -> ok; - FdOld -> file:close(FdOld) - end, - put(fd, Fd), - {reply, ok, Status}; - {error, _} -> - {reply, error, Status} - end; +handle_call({profile, Rootset, Pattern, M,F,A}, From, #state{fd = Fd } = S) -> -handle_call({profile, Rootset}, {From, _Tag}, S) -> - link(From), - maybe_delete(S#state.table), - io:format("eprof: Starting profiling ..... ~n",[]), - ptrac(S#state.rootset, false, all()), - flush_receive(), - Tab = ets:new(eprof, [set, public]), - case ptrac(Rootset, true, all()) of - false -> - {reply, error, #state{}}; + set_pattern_trace(false, S#state.pattern), + set_process_trace(false, S#state.rootset), + + Pid = setup_profiling(M,F,A), + case set_process_trace(true, [Pid|Rootset]) of true -> - uni_schedule(), - call_trace_for_all(true), - erase(replyto), - {reply, profiling, #state{table = Tab, - proc = From, - profiling = true, - rootset = Rootset}} + set_pattern_trace(true, Pattern), + T0 = now(), + execute_profiling(Pid), + {noreply, #state{ + profiling = true, + rootset = [Pid|Rootset], + start_ts = T0, + reply = From, + fd = Fd, + pattern = Pattern + }}; + false -> + exit(Pid, eprof_kill), + {reply, error, #state{ fd = Fd}} end; -handle_call(stop_profiling, _FromTag, S) when S#state.profiling -> - ptrac(S#state.rootset, false, all()), - call_trace_for_all(false), - multi_schedule(), - io:format("eprof: Stop profiling~n",[]), - ets:delete(S#state.table, nofunc), - {reply, profiling_stopped, S#state{profiling = false}}; +handle_call({profile, _Rootset, _Pattern}, _From, #state{ profiling = true } = S) -> + {reply, {error, already_profiling}, S}; -handle_call(stop_profiling, _FromTag, S) -> - {reply, profiling_already_stopped, S}; +handle_call({profile, Rootset, Pattern}, From, #state{ fd = Fd } = S) -> + + set_pattern_trace(false, S#state.pattern), + set_process_trace(false, S#state.rootset), -handle_call({profile, Rootset, M, F, A}, FromTag, S) -> - io:format("eprof: Starting profiling..... ~n", []), - maybe_delete(S#state.table), - ptrac(S#state.rootset, false, all()), - flush_receive(), - put(replyto, FromTag), - Tab = ets:new(eprof, [set, public]), - P = spawn_link(eprof, call, [self(), M, F, A]), - case ptrac([P|Rootset], true, all()) of + case set_process_trace(true, Rootset) of true -> - uni_schedule(), - call_trace_for_all(true), - P ! {self(),go}, - {noreply, #state{table = Tab, - profiling = true, - rootset = [P|Rootset]}}; + T0 = now(), + set_pattern_trace(true, Pattern), + {reply, profiling, #state{ + profiling = true, + rootset = Rootset, + start_ts = T0, + reply = From, + fd = Fd, + pattern = Pattern + }}; false -> - exit(P, kill), - erase(replyto), - {reply, error, #state{}} + {reply, error, #state{ fd = Fd }} end; -handle_call(dump, _FromTag, S) -> - {reply, dump(S#state.table), S}; - -handle_call(analyse, _FromTag, S) -> - {reply, analyse(S), S}; +handle_call(stop_profiling, _From, #state{ profiling = false } = S) -> + {reply, profiling_already_stopped, S}; -handle_call(total_analyse, _FromTag, S) -> - {reply, total_analyse(S), S}; +handle_call(stop_profiling, _From, #state{ profiling = true } = S) -> -handle_call(stop, _FromTag, S) -> - multi_schedule(), - {stop, normal, stopped, S}. + set_pattern_trace(pause, S#state.pattern), -%%%%%%%%%%%%%%%%%%% + Bpd = collect_bpd(), -handle_info({trace_ts,_From,_Op,_Func,_Time}=M, S0) when S0#state.profiling -> - Start = erlang:now(), - #state{table=Tab,pop=PrevOp0,ptime=PrevTime0,pfunc=PrevFunc0, - overhead=Overhead0} = S0, - {PrevFunc,PrevOp,PrevTime} = do_messages(M, Tab, PrevFunc0, PrevOp0, PrevTime0), - Overhead = Overhead0 + subtr(erlang:now(), Start), - S = S0#state{overhead=Overhead,pfunc=PrevFunc,pop=PrevOp,ptime=PrevTime}, - {noreply,S}; + set_process_trace(false, S#state.rootset), + set_pattern_trace(false, S#state.pattern), -handle_info({trace_ts, From, _, _, _}, S) when not S#state.profiling -> - ptrac([From], false, all()), - {noreply, S}; + {reply, profiling_stopped, S#state{ + profiling = false, + rootset = [], + pattern = {'_','_','_'}, + bpd = Bpd + }}; -handle_info({_P, {answer, A}}, S) -> - ptrac(S#state.rootset, false, all()), - io:format("eprof: Stop profiling~n",[]), - {From,_Tag} = get(replyto), - catch unlink(From), - ets:delete(S#state.table, nofunc), - gen_server:reply(erase(replyto), {ok, A}), - multi_schedule(), - {noreply, S#state{profiling = false, - rootset = []}}; - -handle_info({'EXIT', P, Reason}, - #state{profiling=true,proc=P,table=T,rootset=RootSet}) -> - maybe_delete(T), - ptrac(RootSet, false, all()), - multi_schedule(), - io:format("eprof: Profiling failed\n",[]), - case erase(replyto) of - undefined -> - {noreply, #state{}}; - FromTag -> - gen_server:reply(FromTag, {error, Reason}), - {noreply, #state{}} +%% logfile +handle_call({logfile, File}, _From, #state{ fd = OldFd } = S) -> + case file:open(File, [write]) of + {ok, Fd} -> + case OldFd of + undefined -> ok; + OldFd -> file:close(OldFd) + end, + {reply, ok, S#state{ fd = Fd}}; + Error -> + {reply, Error, S} end; -handle_info({'EXIT',_P,_Reason}, S) -> - {noreply, S}. +handle_call(dump, _From, #state{ bpd = Bpd } = S) when is_record(Bpd, bpd) -> + {reply, gb_trees:to_list(Bpd#bpd.p), S}; -uni_schedule() -> - erlang:system_flag(multi_scheduling, block). +handle_call(stop, _FromTag, S) -> + {stop, normal, stopped, S}. -multi_schedule() -> - erlang:system_flag(multi_scheduling, unblock). +%% -------------------------------------------------------------------- %% +%% +%% handle_cast +%% +%% -------------------------------------------------------------------- %% -%%%%%%%%%%%%%%%%%% +handle_cast(_Msg, State) -> + {noreply, State}. -call(Top, M, F, A) -> - receive - {Top,go} -> - Top ! {self(), {answer, apply(M,F,A)}} - end. +%% -------------------------------------------------------------------- %% +%% +%% handle_info +%% +%% -------------------------------------------------------------------- %% -call_trace_for_all(Flag) -> - erlang:trace_pattern(on_load, Flag, [local]), - erlang:trace_pattern({'_','_','_'}, Flag, [local]). +handle_info({'EXIT', _, normal}, S) -> + {noreply, S}; +handle_info({'EXIT', _, eprof_kill}, S) -> + {noreply, S}; +handle_info({'EXIT', _, Reason}, #state{ reply = FromTag } = S) -> -ptrac([P|T], How, Flags) when is_pid(P) -> - case dotrace(P, How, Flags) of - true -> - ptrac(T, How, Flags); - false when How -> - false; - false -> - ptrac(T, How, Flags) - end; + set_process_trace(false, S#state.rootset), + set_pattern_trace(false, S#state.pattern), -ptrac([P|T], How, Flags) when is_atom(P) -> - case whereis(P) of - undefined when How -> - false; - undefined when not How -> - ptrac(T, How, Flags); - Pid -> - ptrac([Pid|T], How, Flags) - end; + gen_server:reply(FromTag, {error, Reason}), + {noreply, S#state{ + profiling = false, + rootset = [], + pattern = {'_','_','_'} + }}; -ptrac([H|_],_How,_Flags) -> - io:format("** eprof bad process ~w~n",[H]), - false; +% check if Pid is spawned process? +handle_info({_Pid, {answer, Result}}, #state{ reply = {From,_} = FromTag} = S) -> -ptrac([],_,_) -> true. + set_pattern_trace(pause, S#state.pattern), -dotrace(P, How, What) -> - case (catch erlang:trace(P, How, What)) of - 1 -> - true; - _Other when not How -> - true; - _Other -> - io:format("** eprof: bad process: ~p,~p,~p~n", [P,How,What]), - false - end. + Bpd = collect_bpd(), -all() -> [call,arity,return_to,running,timestamp,set_on_spawn]. - -total_analyse(#state{table=notable}) -> - nothing_to_analyse; -total_analyse(S) -> - #state{table = T, overhead = Overhead} = S, - QH = qlc:q([{{From,Mfa},Time,Count} || - {[From|Mfa],Time,Count} <- ets:table(T)]), - Pcalls = reverse(keysort(2, replicas(qlc:eval(QH)))), - Time = collect_times(Pcalls), - format("FUNCTION~44s TIME ~n", ["CALLS"]), - printit(Pcalls, Time), - format("\nTotal time: ~.2f\n", [Time / 1000000]), - format("Measurement overhead: ~.2f\n", [Overhead / 1000000]). - -analyse(#state{table=notable}) -> - nothing_to_analyse; -analyse(S) -> - #state{table = T, overhead = Overhead} = S, - Pids = ordsets:from_list(flatten(ets:match(T, {['$1'|'_'],'_', '_'}))), - Times = sum(ets:match(T, {'_','$1', '_'})), - format("FUNCTION~44s TIME ~n", ["CALLS"]), - do_pids(Pids, T, 0, Times), - format("\nTotal time: ~.2f\n", [Times / 1000000]), - format("Measurement overhead: ~.2f\n", [Overhead / 1000000]). - -do_pids([Pid|Tail], T, AckTime, Total) -> - Pcalls = - reverse(keysort(2, to_tups(ets:match(T, {[Pid|'$1'], '$2','$3'})))), - Time = collect_times(Pcalls), - PercentTotal = 100 * (divide(Time, Total)), - format("~n****** Process ~w -- ~s % of profiled time *** ~n", - [Pid, fpf(PercentTotal)]), - printit(Pcalls, Time), - do_pids(Tail, T, AckTime + Time, Total); -do_pids([], _, _, _) -> - ok. + set_process_trace(false, S#state.rootset), + set_pattern_trace(false, S#state.pattern), -printit([],_) -> ok; -printit([{{Mod,Fun,Arity}, Time, Calls} |Tail], ProcTime) -> - format("~s ~s ~s % ~n", [ff(Mod,Fun,Arity), fint(Calls), - fpf(100*(divide(Time,ProcTime)))]), - printit(Tail, ProcTime); -printit([{{_,{Mod,Fun,Arity}}, Time, Calls} |Tail], ProcTime) -> - format("~s ~s ~s % ~n", [ff(Mod,Fun,Arity), fint(Calls), - fpf(100*(divide(Time,ProcTime)))]), - printit(Tail, ProcTime); -printit([_|T], Time) -> - printit(T, Time). - -ff(Mod,Fun,Arity) -> - pad(flatten(io_lib:format("~w:~w/~w", [Mod,Fun, Arity])),45). - -pad(Str, Len) -> - Strlen = length(Str), - if - Strlen > Len -> strip_tail(Str, 45); - true -> lists:append(Str, mklist(Len-Strlen)) - end. + catch unlink(From), + gen_server:reply(FromTag, {ok, Result}), + {noreply, S#state{ + profiling = false, + rootset = [], + pattern = {'_','_','_'}, + bpd = Bpd + }}. + +%% -------------------------------------------------------------------- %% +%% +%% termination +%% +%% -------------------------------------------------------------------- %% + +terminate(_Reason, #state{ fd = undefined }) -> + set_pattern_trace(false, {'_','_','_'}), + ok; +terminate(_Reason, #state{ fd = Fd }) -> + file:close(Fd), + set_pattern_trace(false, {'_','_','_'}), + ok. -strip_tail([_|_], 0) ->[]; -strip_tail([H|T], I) -> [H|strip_tail(T, I-1)]; -strip_tail([],_I) -> []. +%% -------------------------------------------------------------------- %% +%% +%% code_change +%% +%% -------------------------------------------------------------------- %% -fpf(F) -> strip_tail(flatten(io_lib:format("~w", [round(F)])), 5). -fint(Int) -> pad(flatten(io_lib:format("~w",[Int])), 10). +code_change(_OldVsn, State, _Extra) -> + {ok, State}. -mklist(0) -> []; -mklist(I) -> [$ |mklist(I-1)]. -to_tups(L) -> lists:map(fun(List) -> erlang:list_to_tuple(List) end, L). +%% -------------------------------------------------------------------- %% +%% +%% AUX Functions +%% +%% -------------------------------------------------------------------- %% -divide(X,Y) -> X / Y. +setup_profiling(M,F,A) -> + spawn_link(fun() -> spin_profile(M,F,A) end). -collect_times([]) -> 0; -collect_times([Tup|Tail]) -> element(2, Tup) + collect_times(Tail). +spin_profile(M, F, A) -> + receive + {Pid, execute} -> + Pid ! {self(), {answer, erlang:apply(M,F,A)}} + end. -dump(T) -> - L = ets:tab2list(T), - format(L). +execute_profiling(Pid) -> + Pid ! {self(), execute}. -format([H|T]) -> - format("~p~n", [H]), format(T); -format([]) -> ok. +set_pattern_trace(Flag, Pattern) -> + erlang:system_flag(multi_scheduling, block), + erlang:trace_pattern(on_load, Flag, [call_time]), + erlang:trace_pattern(Pattern, Flag, [call_time]), + erlang:system_flag(multi_scheduling, unblock), + ok. -format(F, A) -> - io:format(F,A), - case get(fd) of - undefined -> ok; - Fd -> io:format(Fd, F,A) +set_process_trace(Flag, Pids) -> + % do we need procs for meta info? + % could be useful + set_process_trace(Flag, Pids, [call, set_on_spawn]). +set_process_trace(_, [], _) -> true; +set_process_trace(Flag, [Pid|Pids], Options) when is_pid(Pid) -> + try + erlang:trace(Pid, Flag, Options), + set_process_trace(Flag, Pids, Options) + catch + _:_ -> + false + end; +set_process_trace(Flag, [Name|Pids], Options) when is_atom(Name) -> + case whereis(Name) of + undefined -> + set_process_trace(Flag, Pids, Options); + Pid -> + set_process_trace(Flag, [Pid|Pids], Options) end. -maybe_delete(T) -> - catch ets:delete(T). - -sum([[H]|T]) -> H + sum(T); -sum([]) -> 0. +collect_bpd() -> + collect_bpd([M || M <- [element(1, Mi) || Mi <- code:all_loaded()], M =/= ?MODULE]). + +collect_bpd(Ms) when is_list(Ms) -> + collect_bpdf(collect_mfas(Ms) ++ erlang:system_info(snifs)). + +collect_mfas(Ms) -> + lists:foldl(fun + (M, Mfas) -> + Mfas ++ [{M, F, A} || {F, A} <- M:module_info(functions)] + end, [], Ms). + +collect_bpdf(Mfas) -> + collect_bpdf(Mfas, #bpd{}). +collect_bpdf([], Bpd) -> + Bpd; +collect_bpdf([Mfa|Mfas], #bpd{n = N, us = Us, p = Tree, mfa = Code } = Bpd) -> + case erlang:trace_info(Mfa, call_time) of + {call_time, []} -> + collect_bpdf(Mfas, Bpd); + {call_time, Data} when is_list(Data) -> + {CTn, CTus, CTree} = collect_bpdfp(Mfa, Tree, Data), + collect_bpdf(Mfas, Bpd#bpd{ + n = CTn + N, + us = CTus + Us, + p = CTree, + mfa = [{Mfa, {CTn, CTus}}|Code] + }); + {call_time, false} -> + collect_bpdf(Mfas, Bpd); + {call_time, _Other} -> + collect_bpdf(Mfas, Bpd) + end. -replicas(L) -> - replicas(L, []). +collect_bpdfp(Mfa, Tree, Data) -> + lists:foldl(fun + ({Pid, Ni, Si, Usi}, {PTno, PTuso, To}) -> + Time = Si * 1000000 + Usi, + Ti1 = case gb_trees:lookup(Pid, To) of + none -> + gb_trees:enter(Pid, [{Mfa, {Ni, Time}}], To); + {value, Pmfas} -> + gb_trees:enter(Pid, [{Mfa, {Ni, Time}}|Pmfas], To) + end, + {PTno + Ni, PTuso + Time, Ti1} + end, {0,0, Tree}, Data). + +%% manipulators +sort_mfa(Bpfs, mfa) when is_list(Bpfs) -> + lists:sort(fun + ({A,_}, {B,_}) when A < B -> true; + (_, _) -> false + end, Bpfs); +sort_mfa(Bpfs, time) when is_list(Bpfs) -> + lists:sort(fun + ({_,{A,_}}, {_,{B,_}}) when A < B -> true; + (_, _) -> false + end, Bpfs); +sort_mfa(Bpfs, calls) when is_list(Bpfs) -> + lists:sort(fun + ({_,{_,A}}, {_,{_,B}}) when A < B -> true; + (_, _) -> false + end, Bpfs); +sort_mfa(Bpfs, _) when is_list(Bpfs) -> sort_mfa(Bpfs, calls). + +filter_mfa(Bpfs, Ts) when is_list(Ts) -> + filter_mfa(Bpfs, [], proplists:get_value(calls, Ts, 0), proplists:get_value(time, Ts, 0)); +filter_mfa(Bpfs, _) -> Bpfs. +filter_mfa([], Out, _, _) -> lists:reverse(Out); +filter_mfa([{_, {C, T}}=Bpf|Bpfs], Out, Ct, Tt) when C >= Ct, T >= Tt -> filter_mfa(Bpfs, [Bpf|Out], Ct, Tt); +filter_mfa([_|Bpfs], Out, Ct, Tt) -> filter_mfa(Bpfs, Out, Ct, Tt). + +sum_bp_total_n_us(Mfas) -> + lists:foldl(fun ({_, {Ci,Usi}}, {Co, Uso}) -> {Co + Ci, Uso + Usi} end, {0,0}, Mfas). + +%% strings and format + +string_bp_mfa(Mfas, Tus) -> string_bp_mfa(Mfas, Tus, {0,0,0,0,0}, []). +string_bp_mfa([], _, Ws, Strings) -> {Ws, lists:reverse(Strings)}; +string_bp_mfa([{Mfa, {Count, Time}}|Mfas], Tus, {MfaW, CountW, PercW, TimeW, TpCW}, Strings) -> + Smfa = s(Mfa), + Scount = s(Count), + Stime = s(Time), + Sperc = s("~.2f", [100*(Time/Tus)]), + Stpc = s("~.2f", [Time/Count]), + + string_bp_mfa(Mfas, Tus, { + erlang:max(MfaW, length(Smfa)), + erlang:max(CountW,length(Scount)), + erlang:max(PercW, length(Sperc)), + erlang:max(TimeW, length(Stime)), + erlang:max(TpCW, length(Stpc)) + }, [[Smfa, Scount, Sperc, Stime, Stpc] | Strings]). + +print_bp_mfa(Mfas, {_Tn, Tus}, Fd, Opts) -> + Fmfas = filter_mfa(sort_mfa(Mfas, proplists:get_value(sort, Opts)), proplists:get_value(filter, Opts)), + {{MfaW, CountW, PercW, TimeW, TpCW}, Strs} = string_bp_mfa(Fmfas, Tus), + Ws = { + erlang:max(length("FUNCTION"), MfaW), + erlang:max(length("CALLS"), CountW), + erlang:max(length(" %"), PercW), + erlang:max(length("TIME"), TimeW), + erlang:max(length("uS / CALLS"), TpCW) + }, + format(Fd, Ws, ["FUNCTION", "CALLS", " %", "TIME", "uS / CALLS"]), + format(Fd, Ws, ["--------", "-----", "---", "----", "----------"]), + + lists:foreach(fun (String) -> format(Fd, Ws, String) end, Strs), + ok. -replicas([{{Pid, {Mod,Fun,Arity}}, Ack,Calls} |Tail], Result) -> - case search({Mod,Fun,Arity},Result) of - false -> - replicas(Tail, [{{Pid, {Mod,Fun,Arity}}, Ack,Calls} |Result]); - {Ack2, Calls2} -> - Result2 = del({Mod,Fun,Arity}, Result), - replicas(Tail, [{{Pid, {Mod,Fun,Arity}}, - Ack+Ack2,Calls+Calls2} |Result2]) - end; +s({M,F,A}) -> s("~w:~w/~w",[M,F,A]); +s(Term) -> s("~p", [Term]). +s(Format, Terms) -> lists:flatten(io_lib:format(Format, Terms)). -replicas([_|T], Ack) -> %% Whimpy - replicas(T, Ack); - -replicas([], Res) -> Res. - -search(Key, [{{_,Key}, Ack, Calls}|_]) -> - {Ack, Calls}; -search(Key, [_|T]) -> - search(Key, T); -search(_Key,[]) -> false. - -del(Key, [{{_,Key},_Ack,_Calls}|T]) -> - T; -del(Key, [H | Tail]) -> - [H|del(Key, Tail)]; -del(_Key,[]) -> []. - -flush_receive() -> - receive - {trace_ts, From, _, _, _} when is_pid(From) -> - ptrac([From], false, all()), - flush_receive(); - _ -> - flush_receive() - after 0 -> - ok - end. -code_change(_OldVsn, State, _Extra) -> - {ok,State}. +format(Fd, {MfaW, CountW, PercW, TimeW, TpCW}, Strings) -> + format(Fd, s("~~.~ps ~~~ps ~~~ps ~~~ps [~~~ps]~~n", [MfaW, CountW, PercW, TimeW, TpCW]), Strings); +format(undefined, Format, Strings) -> + io:format(Format, Strings), + ok; +format(Fd, Format, Strings) -> + io:format(Fd, Format, Strings), + io:format(Format, Strings), + ok. diff --git a/lib/tools/test/eprof_SUITE.erl b/lib/tools/test/eprof_SUITE.erl index 028fea8fe1..67607c6cf2 100644 --- a/lib/tools/test/eprof_SUITE.erl +++ b/lib/tools/test/eprof_SUITE.erl @@ -20,10 +20,89 @@ -include("test_server.hrl"). --export([all/1,tiny/1,eed/1]). +-export([all/1,tiny/1,eed/1,basic/1]). -all(suite) -> [tiny,eed]. +all(suite) -> [basic,tiny,eed]. +basic(suite) -> []; +basic(Config) when is_list(Config) -> + + %% load eprof_test and change directory + + ?line {ok, OldCurDir} = file:get_cwd(), + Datadir = ?config(data_dir, Config), + Privdir = ?config(priv_dir, Config), + ?line {ok,eprof_test} = compile:file(filename:join(Datadir, "eprof_test"), + [trace,{outdir, Privdir}]), + ?line ok = file:set_cwd(Privdir), + ?line code:purge(eprof_test), + ?line {module,eprof_test} = code:load_file(eprof_test), + + %% rootset profiling + + ?line ensure_eprof_stopped(), + ?line profiling = eprof:profile([self()]), + ?line {error, already_profiling} = eprof:profile([self()]), + ?line profiling_stopped = eprof:stop_profiling(), + ?line profiling_already_stopped = eprof:stop_profiling(), + ?line profiling = eprof:start_profiling([self(),self(),self()]), + ?line profiling_stopped = eprof:stop_profiling(), + + %% with patterns + + ?line profiling = eprof:start_profiling([self()], {?MODULE, '_', '_'}), + ?line {error, already_profiling} = eprof:start_profiling([self()], {?MODULE, '_', '_'}), + ?line profiling_stopped = eprof:stop_profiling(), + ?line profiling = eprof:start_profiling([self()], {?MODULE, start_stop, '_'}), + ?line profiling_stopped = eprof:stop_profiling(), + ?line profiling = eprof:start_profiling([self()], {?MODULE, start_stop, 1}), + ?line profiling_stopped = eprof:stop_profiling(), + + %% with fun + + ?line {ok, _} = eprof:profile(fun() -> eprof_test:go(10) end), + ?line profiling = eprof:profile([self()]), + ?line {error, already_profiling} = eprof:profile(fun() -> eprof_test:go(10) end), + ?line profiling_stopped = eprof:stop_profiling(), + ?line {ok, _} = eprof:profile(fun() -> eprof_test:go(10) end), + ?line {ok, _} = eprof:profile([], fun() -> eprof_test:go(10) end), + ?line Pid = whereis(eprof), + ?line {ok, _} = eprof:profile(erlang:processes() -- [Pid], fun() -> eprof_test:go(10) end), + ?line {ok, _} = eprof:profile([], fun() -> eprof_test:go(10) end, {eprof_test, '_', '_'}), + ?line {ok, _} = eprof:profile([], fun() -> eprof_test:go(10) end, {eprof_test, go, '_'}), + ?line {ok, _} = eprof:profile([], fun() -> eprof_test:go(10) end, {eprof_test, go, 1}), + ?line {ok, _} = eprof:profile([], fun() -> eprof_test:go(10) end, {eprof_test, dec, 1}), + + %% error case + + ?line error = eprof:profile([Pid], fun() -> eprof_test:go(10) end), + ?line Pid = whereis(eprof), + ?line error = eprof:profile([Pid], fun() -> eprof_test:go(10) end), + ?line A = spawn(fun() -> receive _ -> ok end end), + ?line profiling = eprof:profile([A]), + ?line true = exit(A, kill_it), + ?line profiling_stopped = eprof:stop_profiling(), + ?line {error,_} = eprof:profile(fun() -> a = b end), + + %% with mfa + + ?line {ok, _} = eprof:profile([], eprof_test, go, [10]), + ?line {ok, _} = eprof:profile([], eprof_test, go, [10], {eprof_test, dec, 1}), + + %% dump + + ?line {ok, _} = eprof:profile([], fun() -> eprof_test:go(10) end, {eprof_test, '_', '_'}), + ?line [{_, Mfas}] = eprof:dump(), + ?line Dec_mfa = {eprof_test, dec, 1}, + ?line Go_mfa = {eprof_test, go, 1}, + ?line {value, {Go_mfa, { 1, _Time1}}} = lists:keysearch(Go_mfa, 1, Mfas), + ?line {value, {Dec_mfa, {11, _Time2}}} = lists:keysearch(Dec_mfa, 1, Mfas), + + %% change current working directory + + ?line ok = file:set_cwd(OldCurDir), + ?line stopped = eprof:stop(), + ok. tiny(suite) -> []; tiny(Config) when is_list(Config) -> @@ -40,11 +119,11 @@ tiny(Config) when is_list(Config) -> ?line code:purge(eprof_suite_test), ?line {module,eprof_suite_test} = code:load_file(eprof_suite_test), ?line {ok,_Pid} = eprof:start(), - ?line nothing_to_analyse = eprof:analyse(), - ?line nothing_to_analyse = eprof:total_analyse(), + ?line nothing_to_analyze = eprof:analyze(), + ?line nothing_to_analyze = eprof:analyze(total), ?line eprof:profile([], eprof_suite_test, test, [Config]), - ?line ok = eprof:analyse(), - ?line ok = eprof:total_analyse(), + ?line ok = eprof:analyze(), + ?line ok = eprof:analyze(total), ?line ok = eprof:log("eprof_SUITE_logfile"), ?line stopped = eprof:stop(), ?line ?t:timetrap_cancel(TTrap), @@ -79,8 +158,8 @@ eed(Config) when is_list(Config) -> ?line {ok,ok} = eprof:profile([], eed, file, [Script]), ?line {T3,_} = statistics(runtime), ?line profiling_already_stopped = eprof:stop_profiling(), - ?line ok = eprof:analyse(), - ?line ok = eprof:total_analyse(), + ?line ok = eprof:analyze(), + ?line ok = eprof:analyze(total), ?line ok = eprof:log("eprof_SUITE_logfile"), ?line stopped = eprof:stop(), ?line ?t:timetrap_cancel(TTrap), diff --git a/lib/tools/test/eprof_SUITE_data/eprof_test.erl b/lib/tools/test/eprof_SUITE_data/eprof_test.erl new file mode 100644 index 0000000000..33c428e893 --- /dev/null +++ b/lib/tools/test/eprof_SUITE_data/eprof_test.erl @@ -0,0 +1,9 @@ +-module(eprof_test). +-export([go/1]). + +go(N) -> + 0 = dec(N), + ok. + +dec(0) -> 0; +dec(N) -> dec(N - 1). diff --git a/lib/tools/test/fprof_SUITE.erl b/lib/tools/test/fprof_SUITE.erl index e437007e76..1cd9ac7824 100644 --- a/lib/tools/test/fprof_SUITE.erl +++ b/lib/tools/test/fprof_SUITE.erl @@ -356,7 +356,7 @@ imm_tail_seq(Config) when is_list(Config) -> ?line profiling_stopped = eprof:stop_profiling(), ?line R2 = R0, %% - ?line eprof:analyse(), + ?line eprof:analyze(), ?line stopped = eprof:stop(), %% ?line {ok, Tracer} = fprof:profile(start), @@ -471,7 +471,7 @@ imm_compile(Config) when is_list(Config) -> ?line TS3 = erlang:now(), ?line profiling_stopped = eprof:stop_profiling(), %% - ?line eprof:analyse(), + ?line eprof:analyze(), ?line stopped = eprof:stop(), %% ?line {ok, Tracer} = fprof:profile(start), diff --git a/lib/tools/vsn.mk b/lib/tools/vsn.mk index 13cf5af9f5..abe9a804f0 100644 --- a/lib/tools/vsn.mk +++ b/lib/tools/vsn.mk @@ -1,19 +1,19 @@ # This is an -*-makefile-*- file. # %CopyrightBegin% -# -# Copyright Ericsson AB 1997-2009. All Rights Reserved. -# +# +# Copyright Ericsson AB 1997-2010. All Rights Reserved. +# # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in # compliance with the License. You should have received a copy of the # Erlang Public License along with this software. If not, it can be # retrieved online at http://www.erlang.org/. -# +# # Software distributed under the License is distributed on an "AS IS" # basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See # the License for the specific language governing rights and limitations # under the License. -# +# # %CopyrightEnd% -TOOLS_VSN = 2.6.5.1 +TOOLS_VSN = 2.6.6 diff --git a/lib/tv/doc/src/notes.xml b/lib/tv/doc/src/notes.xml index 62fab2f0f1..43b650dd29 100644 --- a/lib/tv/doc/src/notes.xml +++ b/lib/tv/doc/src/notes.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2004</year><year>2009</year> + <year>2004</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>TV Release Notes</title> @@ -30,6 +30,21 @@ </header> <p>This document describes the changes made to the TV application.</p> +<section><title>TV 2.1.4.5</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Warnings due to new autoimported BIFs removed</p> + <p> + Own Id: OTP-8674 Aux Id: OTP-8579 </p> + </item> + </list> + </section> + +</section> + <section><title>TV 2.1.4.4</title> <section><title>Improvements and New Features</title> diff --git a/lib/tv/vsn.mk b/lib/tv/vsn.mk index be1981b755..93973489bc 100644 --- a/lib/tv/vsn.mk +++ b/lib/tv/vsn.mk @@ -1,20 +1,20 @@ # # %CopyrightBegin% -# -# Copyright Ericsson AB 1997-2009. All Rights Reserved. -# +# +# Copyright Ericsson AB 1997-2010. All Rights Reserved. +# # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in # compliance with the License. You should have received a copy of the # Erlang Public License along with this software. If not, it can be # retrieved online at http://www.erlang.org/. -# +# # Software distributed under the License is distributed on an "AS IS" # basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See # the License for the specific language governing rights and limitations # under the License. -# +# # %CopyrightEnd% # -TV_VSN = 2.1.4.4 +TV_VSN = 2.1.4.5 diff --git a/lib/webtool/doc/src/notes.xml b/lib/webtool/doc/src/notes.xml index dc325b5b61..5179f37db2 100644 --- a/lib/webtool/doc/src/notes.xml +++ b/lib/webtool/doc/src/notes.xml @@ -31,6 +31,23 @@ <p>This document describes the changes made to the Webtool application.</p> +<section><title>WebTool 0.8.7</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Up until now Netscape has been the default web browser on + Unix/Linux. Webtool has now been updated to start Firefox + as default browser instead.</p> + <p> + Own Id: OTP-8651 Aux Id: OTP-8650 </p> + </item> + </list> + </section> + +</section> + <section><title>WebTool 0.8.6</title> <section><title>Improvements and New Features</title> diff --git a/lib/webtool/doc/src/start_webtool.xml b/lib/webtool/doc/src/start_webtool.xml index 184285c631..b525b38845 100644 --- a/lib/webtool/doc/src/start_webtool.xml +++ b/lib/webtool/doc/src/start_webtool.xml @@ -4,7 +4,7 @@ <comref> <header> <copyright> - <year>2003</year><year>2009</year> + <year>2003</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>start_webtool</title> @@ -56,7 +56,7 @@ Or http://127.0.0.1:8888/ Usage: start_webtool application [ browser ] Available applications are: [orber,appmon,crashdump_viewer,webcover] -Default browser is 'iexplore' (Internet Explorer) on Windows or else 'netscape' </pre> +Default browser is 'iexplore' (Internet Explorer) on Windows or else 'firefox' </pre> <p>To start any of the listed applications, give the application name as the first argument, e.g.</p> <pre> @@ -68,7 +68,7 @@ Starting webcover... Sending URL to netscape...done </pre> <p>The WebTool application WebCover is then started and the default browser is used. The default browser is Internet - Explorer on Windows or else Netscape. + Explorer on Windows or else Firefox. </p> <p>To use another browser, give the browser's start command as the second argument, e.g.</p> diff --git a/lib/webtool/src/webtool.erl b/lib/webtool/src/webtool.erl index 51d821751c..1be903b827 100644 --- a/lib/webtool/src/webtool.erl +++ b/lib/webtool/src/webtool.erl @@ -1,19 +1,19 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2001-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% -module(webtool). @@ -139,7 +139,7 @@ script_start([App]) -> DefaultBrowser = case os:type() of {win32,_} -> iexplore; - _ -> netscape + _ -> firefox end, script_start([App,DefaultBrowser]); script_start([App,Browser]) -> @@ -159,6 +159,8 @@ script_start([App,Browser]) -> "http://localhost:" ++ PortStr ++ "/" ++ StartPage end, case Browser of + none -> + ok; iexplore when OSType == win32-> io:format("Starting internet explorer...\n"), {ok,R} = win32reg:open(""), @@ -170,7 +172,7 @@ script_start([App,Browser]) -> _ when OSType == win32 -> io:format("Starting ~w...\n",[Browser]), os:cmd("\"" ++ atom_to_list(Browser) ++ "\" " ++ Url); - B when B==netscape; B==mozilla -> + B when B==firefox; B==mozilla -> io:format("Sending URL to ~w...",[Browser]), BStr = atom_to_list(Browser), SendCmd = BStr ++ " -raise -remote \'openUrl(" ++ @@ -209,7 +211,7 @@ usage() -> "\nUsage: start_webtool application [ browser ]\n" "\nAvailable applications are: ~p\n" "Default browser is \'iexplore\' (Internet Explorer) on Windows " - "or else \'netscape\'\n", + "or else \'firefox\'\n", [Apps]), stop(). diff --git a/lib/webtool/vsn.mk b/lib/webtool/vsn.mk index 712e3abbaf..6b76883330 100644 --- a/lib/webtool/vsn.mk +++ b/lib/webtool/vsn.mk @@ -1 +1 @@ -WEBTOOL_VSN=0.8.6 +WEBTOOL_VSN=0.8.7 diff --git a/lib/xmerl/vsn.mk b/lib/xmerl/vsn.mk index 83b9d4826f..aee7546c3c 100644 --- a/lib/xmerl/vsn.mk +++ b/lib/xmerl/vsn.mk @@ -21,7 +21,8 @@ XMERL_VSN = 1.2.5 TICKETS = \ - OTP-8537 + OTP-8537 \ + OTP-8599 TICKETS_1.2.5 = \ OTP-8537 diff --git a/system/README b/system/README index 15ccfd2f84..317030373c 100644 --- a/system/README +++ b/system/README @@ -1,7 +1,7 @@ -Erlang/OTP February 18, 2010 +Erlang/OTP June 11, 2010 -LAST MINUTE INFORMATION -- Release of Erlang 5.7.5/OTP R13B04 +LAST MINUTE INFORMATION -- Release of Erlang 5.8/OTP R14A 1. GENERAL diff --git a/system/doc/top/highlights.xml b/system/doc/top/highlights.xml index a30742aed6..251bb1a47d 100644 --- a/system/doc/top/highlights.xml +++ b/system/doc/top/highlights.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2006</year><year>2009</year> + <year>2006</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Highlights</title> @@ -28,133 +28,53 @@ <rev>A</rev> <file>highlights.xml</file> </header> - <p>This document lists some highlights of Erlang 5.7/OTP R13A - (unpatched), compared to the previous version Erlang 5.6/OTP R12B, - with focus on things not already released as R12B patches.</p> - <p>Note: This document was compiled at the time when R13A was released - and does not list any features introduced in R13 patches.</p> + <p>This document lists some highlights of Erlang 5.8/OTP R14A + (unpatched), compared to the previous version Erlang 5.7.1/OTP R13B, + with focus on things not already released as R13B patches.</p> + <p>Note: This document was compiled at the time when R14A was released + and does not list any features introduced in R14 patches.</p> <section> <title>Erlang Language and Run-time System</title> <section> - <title>Multi-core and SMP performance improvements</title> - <p> - There SMP performance is significantly improved and will - allow most applications to scale much better on systems with many - cores or processors. Listed below are some of the most important improvements: - </p> - <list type="bulleted"> - <item> - <p> - The runtime system with SMP support now uses <em>multiple, - scheduler specific run queues</em>, instead of one globally shared - run queue. - </p> - <p> - The lock protecting the shared run queue was heavily - contended, and the shared run queue also caused Erlang - processes to randomly migrate between schedulers with - negative cache effects as a result. - </p> - <p> - With the current scheduler specific run queue solution, lock - contention due to run queue protection has been reduced, and - Erlang processes are only migrated when needed to balance the - load between the schedulers. The reduced amount of migration - also reduce lock contention on locks protecting the scheduler - specific instances of the erts internal memory allocators. - </p> - <p> - The scheduler specific run queues are also a necessity for a - lot of future planned NUMA (Non-Uniform Memory Access) - specific optimizations. - </p> - </item> - <item> - <p> - Message passing has been further optimized for parallell execution. - This makes parallell sending to one common receiver much more efficient. - </p> - </item> - <item> - <p> - Scheduler threads can now be bound to logical processors on newer - Linux ans Solaris systems. - </p> - </item> - </list> - </section> - <section> - <title>Unicode support</title> - <p> - Support for Unicode is implemented as described in EEP10. - Formatting and reading of unicode data both from terminals - and files is supported by the <c>io</c> and <c>io_lib</c> modules. - Files - can be opened in modes with automatic translation to and from - different unicode formats. The module 'unicode' contains - functions for conversion between external and internal - unicode formats and the re module has support for unicode - data. There is also language syntax for specifying string and - character data beyond the ISO-latin-1 range. - </p> - </section> + <title>New NIF features</title> + <list type="bulleted"> + <item><p>Send messages from a NIF, or from thread created by NIF, to + any local process (enif_send)</p></item> - <section> - <title>New BIF's</title> - <p> - The BIFs <c>atom_to_binary/2</c>, <c>binary_to_atom/2</c> and - <c>binary_to_existing_atom/2</c> have been added. - </p> - </section> - - <section> - <title>Independent Erlang clusters on the same host</title> - <p> - Nodes belonging to different independent clusters can now - co-exist on the same host with the help of a new - environment variable setting <c>ERL_EPMD_PORT</c>. - The environment variable is used by Erl_interface and J_interface - as well. - </p> - </section> - </section> + <item><p>Store terms between NIF calls (enif_alloc_env, + enif_make_copy)</p></item> - <section> - <title>New Applications</title> + <item><p>Create binary terms with user defined memory management + (enif_make_resource_binary)</p></item> + <item><p>Some incompatible changes made to the API. For more + information see the warning text in erl_nif(3).</p></item> + </list> + </section> <section> - <title>Reltool</title> - - <p> - Reltool is a release management tool. - It analyses a given Erlang/OTP installation and determines - various dependencies between applications. - The graphical frontend depicts the dependencies and enables - interactive customization of a target system. - The backend provides a batch interface for generation of - customized target systems. - The application is still somewhat limited and should be regarded - as experimental in this release. The intention is that this - application will be a valuable tool for making both traditional - Erlang target systems as well as standalone components in Erlang. - </p> + <title>Receive optimizations</title> + <p>Receive statements that can only read out a newly created + reference are now specially optimized so that it will execute + in constant time regardless of the number of messages in the + receive queue for the process. That optimization will benefit + calls to gen_server:call(). (See gen:do_call/4 for an example + of a receive statement that will be optimized.)</p> </section> - <section> - <title>WxErlang</title> - <p> - wxErlang is an Erlang binding to the WxWidgets GUI library which - provides support for cross platform GUI applications. - wxErlang is still in beta status and the intention is that it shall - replace GS in a later stage. The Erlang debugger is also shipped in - a wxErlang version. - </p> + <title>Improved eprof</title> + <p>eprof has been reimplemented with support in the Erlang + virtual machine and is now both faster (i.e. slows down the + code being measured less) and scales much better. In + measurements we saw speed-ups compared to the old eprof + ranging from 6 times (for sequential code that only uses one + scheduler/core) up to 84 times (for parallel code that uses 8 + cores).</p> + + <p>Note: The API for the eprof has been cleaned up and extended. + See the documentation.</p> </section> - - </section> - <section> <title>New features in Existing Applications</title> @@ -162,77 +82,29 @@ <title>Common_test</title> <p> - A support client module for SSH and SFTP, <c>ct_ssh</c>, has - been introduced. - </p> - <p> - Test case groups have been introduced. With this feature - it's possible to execute groups (possibly nested) - of test cases. + It is now possible for the user to provide specific callback + modules that handle test configuration data, so that data on + arbitray form can be accessed (e.g. by reading files or by + communicating with a configuration server process). Two + default callback modules have been introduced in Common Test: + ct_config_plain and ct_config_xml. The former is used to + handle the traditional Common Test configuration files (with + terms on key-value tuple form) and the latter to handle + configuration data on XML representation. </p> <p> - A group definition contains a name tag, a list of - properties and a list of test cases (including possible - nested group definitions). The properties make it possible - to execute test cases in parallel, in sequence and in - shuffled order. It is also possible to repeat test cases - according to different criteria. + The run_test script has been replaced by a program (with the + same name) which can be executed without explicit + installation. The start flags are the same as for the legacy + start script. </p> </section> - - <section> - <title>Dialyzer</title> - <p> - The analysis now accepts <em>opaque type declarations</em> and - detects - violations of opaqueness of terms of such types. Starting - with R13, many Erlang/OTP standard libraries (array, dict, - digraph, ets, gb_sets, gb_trees, queue, and sets) contain - opaque type declarations of their main data types. Dialyzer - will spit out warnings in code that explicitly depends on the - structure of these terms. - </p> - <p> - Added support for handling UTF segments in bitstreams and for - detecting obvious type errors in these segments. Warning: - This code is not terribly tested though since there are very - few Erlang programs which use Unicode-based binaries - not - surprising since this is a new language feature of R13. - </p> - <p> - Strengthened the discrepancy identification when testing for - equality and matching between terms of different types. This - detects more bugs in code. - </p> - <p> - See the Dialyzer documentation and release notes for even more - enhancements. - </p> - </section> - - <section> - <title>SSL</title> - <p> - The "new_ssl" implementation is significantly improved and should be - near product status now. - The new SSL is implemented in pure Erlang except for - the crypto routines that are implemented in the crypto driver which - is an interface to libcrypto from OpenSSL. - </p> - </section> - - <section> <title>STDLIB</title> - - <p>The Erlang scanner has been augmented as to return white-space, - comments and exact location of tokens. - This means that the scanner can easily be used in tools such as editors, - pretty printers, syntax highlighters etc. - where it is important to be able recreate the original source document. + <p> + The module binary from EEP31 (and EEP9) is implemented. </p> </section> - </section> </chapter> diff --git a/system/doc/top/incompatible.xml b/system/doc/top/incompatible.xml index ce9522725b..e09fa7272f 100644 --- a/system/doc/top/incompatible.xml +++ b/system/doc/top/incompatible.xml @@ -4,7 +4,7 @@ <chapter> <header> <copyright> - <year>2006</year><year>2009</year> + <year>2006</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -13,12 +13,12 @@ compliance with the License. You should have received a copy of the Erlang Public License along with this software. If not, it can be retrieved online at http://www.erlang.org/. - + Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. - + </legalnotice> <title>Potential Incompatibilities</title> @@ -29,353 +29,241 @@ <file>incompatible.xml</file> </header> <p>This document contains a list of potential incompatibilities - between Erlang 5.7/OTP R12A and Erl 5.6.5/OTP R12B-5, + between Erlang 5.8/OTP R14A and Erl 5.7.5/OTP R13B04, and is an extract from the release notes for the respective applications.</p> <section> - <title>compiler</title> + <title>Tools</title> <list type="bulleted"> <item> <p> - The undocumented, unsupported, and deprecated guard BIF - <c>is_constant/1</c> has been removed.</p> - <p> - *** INCOMPATIBILITY with R12B ***</p> - <p> - Own Id: OTP-7673</p> - </item> - <item> - <p>The short-circuit operators <c>andalso</c> and - <c>orelse</c> no longer guarantees that their second - argument is either <c>true</c> or <c>false</c>. As a - consequence, <c>andalso</c>/<c>orelse</c> are now - tail-recursive.</p> - <p> - *** POTENTIAL INCOMPATIBILITY ***</p> + Xref has been updated to use the re module instead of the + deprecated regexp module.</p> + <p>*** INCOMPATIBILITY with R12B ***</p> <p> - Own Id: OTP-7748</p> + Own Id: OTP-8472</p> </item> <item> - <p>The compiler will refuse to a compile file where the - module name in the file differs from the output file - name.</p> - <p>When compiling using <c>erlc</c>, the current working - directory will no be included in the code path (unless - explicitly added using "-pa .").</p> - <p> - *** POTENTIAL INCOMPATIBILITY ***</p> - <p> - Own Id: OTP-7793</p> - </item> - <item> - <p>If a module contains an exported function with the - same name as an auto-imported BIF (such as - <c>length/1</c>), any calls to the BIF must have an - explicit <c>erlang:</c> prefix, or there will be a - compilation error (such calls would only generate a - warning in previous releases).</p> - <p>(The reason for the change is to avoid breaking code - in a future major release, R14 or R15, in which we plan - to make calls without a module prefix always call the - local function in the same module even if there is an - auto-imported BIF with the same name.)</p> + <p>eprof has been reimplemented with support in the Erlang + virtual machine and is now both faster (i.e. slows down the + code being measured less) and scales much better. In + measurements we saw speed-ups compared to the old eprof + ranging from 6 times (for sequential code that only uses one + scheduler/core) up to 84 times (for parallel code that uses 8 + cores).</p> + + <p>Note: The API for the eprof has been cleaned up and extended. + See the documentation.</p> <p> *** POTENTIAL INCOMPATIBILITY ***</p> <p> - Own Id: OTP-7873</p> + Own Id: OTP-8706</p> </item> </list> </section> <section> - <title>Erts</title> + <title>Odbc</title> <list type="bulleted"> <item> - <p>Support for Unicode is implemented as described in - EEP10. Formatting and reading of unicode data both from - terminals and files is supported by the io and io_lib - modules. Files can be opened in modes with automatic - translation to and from different unicode formats. The - module 'unicode' contains functions for conversion - between external and internal unicode formats and the re - module has support for unicode data. There is also - language syntax for specifying string and character data - beyond the ISO-latin-1 range.</p> - <p>The interactive shell will support input and output of - unicode characters when the terminal and operating system - supports it.</p> - <p>Please see the EEP and the io/io_lib manual pages as - well as the stdlib users guide for details.</p> - <p><em>I/O-protocol incompatibilities:</em></p> - <p>The io_protocol between io_Server and client is - updated to handle protocol data in unicode formats. The - updated protocol is now documented. The specification - resides in the stdlib <em>users manual</em>, which is a - new part of the manual.</p> - <p><em>io module incompatibilities:</em></p> - <p>The io:put_chars, io:get_chars and io:get_line all - handle and return unicode data. In the case where - binaries can be provided (as to io:put_chars), they shall - be encoded in UTF-8. When binaries are returned (as by - io:get_line/get_chars when the io_server is set in - <em>binary mode</em>) the returned data is also - <em>always</em> encoded as UTF-8. The file module however - still returns byte-oriented data, why file:read can be - used instead of io:get_chars to read binary data in - ISO-latin-1.</p> - <p><em>io_lib module incompatibilities:</em></p> - <p>io_lib:format can, given new format directives (i.e - "~ts" and "~tc"), return lists containing integers larger - than 255. </p> - <p> - *** POTENTIAL INCOMPATIBILITY ***</p> - <p> - Own Id: OTP-7648 Aux Id: OTP-7580 OTP-7514 OTP-7494 - OTP-7443 OTP-7181 EEP10 EEP11 </p> - </item> - <item> - <p> - The undocumented, unsupported, and deprecated guard BIF - <c>is_constant/1</c> has been removed.</p> - <p> - *** INCOMPATIBILITY with R12B ***</p> - <p> - Own Id: OTP-7673</p> - </item> - <item> - <p>The short-circuit operators <c>andalso</c> and - <c>orelse</c> no longer guarantees that their second - argument is either <c>true</c> or <c>false</c>. As a - consequence, <c>andalso</c>/<c>orelse</c> are now - tail-recursive.</p> + <p> + Now supports SQL_TYPE_TIMESTAMP on the format {{YY, MM, DD}, + {HH, MM, SS}}. Thanks to Juhani Ränkimies. + </p> <p> *** POTENTIAL INCOMPATIBILITY ***</p> <p> - Own Id: OTP-7748</p> + Own Id: OTP-8511</p> </item> - <item> - <p>The compiler will refuse to a compile file where the - module name in the file differs from the output file - name.</p> - <p>When compiling using <c>erlc</c>, the current working - directory will no be included in the code path (unless - explicitly added using "-pa .").</p> + </list> + </section> + + <section> + <title>Ssh</title> + <list type="bulleted"> + <item> <p> - *** POTENTIAL INCOMPATIBILITY ***</p> + The configuration parameter ip_v6_disabled is now available, + which makes it possible for the user to alter the IP version + SSH shall use.</p> <p> - Own Id: OTP-7793</p> - </item> - <item> - <p>The deprecated functions <c>erlang:fault/1</c>, - <c>erlang:fault/2</c>, and <c>file:rawopen/2</c> have - been removed.</p> + *** POTENTIAL INCOMPATIBILITY ***</p> <p> - *** POTENTIAL INCOMPATIBILITY ***</p> + Own Id: OTP-8535</p> + </item> + <item> <p> - Own Id: OTP-7812</p> - </item> - <item> - <p>The escape sequences <c>\x</c> and <c>\{</c> have been - assigned new interpretations (they used to return the - ASCII code for <c>x</c> and <c>{</c> respectively). One - or more octal characters inside curly brackets after a - leading backslash is from now on an alternative to the - existing syntax <c>\NNN</c>, but can also be used for - codes greater than 255. In a similar fashion, one or more - hexadecimal characters can be put inside curly brackets - after a leading <c>\x</c>. Furthermore, the escape - sequences <c>\xH</c> and <c>\xHH</c>, where N is a - hexadecimal character, can be used for codes less than - 256.</p> - <p>NOTE: These new escape sequences are still considered - experimental and may be changed in the R13B release.</p> + Removed deprecated modules (ssh_ssh, ssh_sshd and ssh_cm) and + functions (ssh_sftp:connect and ssh_sftp:stop).</p> <p> - *** POTENTIAL INCOMPATIBILITY ***</p> + *** POTENTIAL INCOMPATIBILITY ***</p> <p> - Own Id: OTP-7855</p> - </item> - </list> - </section> - - <section> - <title>Inets</title> - <list type="bulleted"> + Own Id: OTP-8596</p> + </item> <item> <p> - [httpc] - The inets http client will now use persistent - connections without pipelining as default and if a - pipeline timeout is set it will pipeline the requests on - the persistent connections.</p> + Aligned error message with used version (SSH_FX_FAILURE vs + SSH_FX_NOT_A_DIRECTORY, the latter introduced in version 6).</p> <p> *** POTENTIAL INCOMPATIBILITY ***</p> <p> - Own Id: OTP-7463</p> - </item> + Own Id: OTP-8644</p> + </item> </list> </section> <section> - <title>Jinterface</title> + <title>ERTS</title> <list> <item> - <p> - A number of fixes and improvements from the ErlIDE group; - Vlad Dumitrescu and Jakob Cederlund: JDK 1.5 is now a - minimal requirement for building Jinterface. New method: - OtpEpmd.lookupNames. OtpErlangList is now iterable. - Non-proper lists are now allowed - you have to test if a - list is proper or not. Non-proper lists can also be - created. New methods: isProper, getHead, getTail and - getNthTail. The get tail methods creates a sublist object - that re-uses the original list. OtpErlangPid is now - Comparable. Empty atoms can now be constructed, a missing - feature pointed out by Sebastien Boisgerault on - erlang-questions.</p> + <p>Some incompatible changes made to the NIF API. For more + information see the warning text in erl_nif(3).</p> <p> *** POTENTIAL INCOMPATIBILITY ***</p> <p> - Own Id: OTP-7832</p> + Own Id: OTP-8555</p> </item> </list> </section> <section> - <title>Kernel</title> + <title>STDLIB</title> <list type="bulleted"> <item> - <p>The deprecated functions <c>erlang:fault/1</c>, - <c>erlang:fault/2</c>, and <c>file:rawopen/2</c> have - been removed.</p> + <p>When defining macros the closing right parenthesis before the + dot is now mandatory.</p> <p> *** POTENTIAL INCOMPATIBILITY ***</p> <p> - Own Id: OTP-7812</p> + Own Id: OTP-8562</p> </item> </list> </section> <section> - <title>SSH</title> + <title>Compiler and STDLIB</title> <list type="bulleted"> <item> <p> - Ssh timeouts will now behave as expected i.e. defaults to - infinity. Only the user of the ssh application can know of - a reasonable timeout value for their application.</p> + Local and imported functions now override the auto-imported + BIFs when the names clash. The pre R14 behaviour was that + auto-imported BIFs would override local functions. To avoid + that old programs change behaviour, the following will + generate an error:</p> + + <list type="bulleted"> + <item>Doing a call without explicit module name to a local function + having a name clashing with the name of an auto-imported BIF + that was present (and auto-imported) before OTP R14A</item> + + <item>Explicitly importing a function having a name clashing with + the name of an autoimported BIF that was present (and + autoimported) before OTP R14A</item> + + <item>Using any form of the old compiler directive nowarn_bif_clash</item> + </list> + + <p>If the BIF was added or auto-imported in OTP R14A or later, + overriding it with an import or a local function will only + result in a warning.</p> + + <p>To resolve clashes, you can either use the explicit module + name erlang to call the BIF, or you can remove the + auto-import of that specific BIF by using the new compiler + directive -compile({no_auto_import,[F/A]})., which makes all + calls to the local or imported function without explicit + module name pass without warnings or errors.</p> + + <p>The change makes it possible to add auto-imported BIFs + without breaking or silently changing old code in the future. + However some current code ingeniously utilizing the old + behaviour or the nowarn_bif_clash compiler directive, might + need changing to be accepted by the compiler.</p> <p> *** POTENTIAL INCOMPATIBILITY ***</p> <p> - Own Id: OTP-7807</p> + Own Id: OTP-8579</p> </item> <item> <p> - Added the message {ssh_channel_up, ChannelId, - ConnectionManager} that shall be handled by the channel - callback handle_msg/2. This makes the function - handle_msg/2 a mandatory function for ssh channels - implementations which it was not in ssh-1.1.</p> + The recently added BIFs erlang:min/2, erlang:max/2 and + erlang:port_command/3 are now auto-imported (as they were + originally intended to be). Due to the recent compiler change + (OTP-8579), the only impact on old code defining it's own + min/2, max/2 or port_command/3 functions will be a warning, + the local functions will still be used. The warning can be + removed by using + -compile({no_auto_import,[min/2,max/2,port_command/3]}). in + the source file.</p> <p> *** POTENTIAL INCOMPATIBILITY ***</p> <p> - Own Id: OTP-7828</p> + Own Id: OTP-8669</p> </item> - </list> - </section> - - <section> - <title>STDLIB</title> - <list type="bulleted"> <item> - <p>The functions <c>lists:seq/1,2</c> return the empty - list in a few cases when they used to generate an - exception, for example <c>lists:seq(1, 0)</c>. See - lists(3) for details. (Thanks to Richard O'Keefe.)</p> <p> - *** POTENTIAL INCOMPATIBILITY ***</p> + Now, binary_to_term/2 is auto-imported. This will cause a + compile warning if and only if a module has got a local + function with that name.</p> <p> - Own Id: OTP-7230</p> - </item> - <item> - <p>Support for Unicode is implemented as described in - EEP10. Formatting and reading of unicode data both from - terminals and files is supported by the io and io_lib - modules. Files can be opened in modes with automatic - translation to and from different unicode formats. The - module 'unicode' contains functions for conversion - between external and internal unicode formats and the re - module has support for unicode data. There is also - language syntax for specifying string and character data - beyond the ISO-latin-1 range.</p> - <p>The interactive shell will support input and output of - unicode characters when the terminal and operating system - supports it.</p> - <p>Please see the EEP and the io/io_lib manual pages as - well as the stdlib users guide for details.</p> - <p><em>I/O-protocol incompatibilities:</em></p> - <p>The io_protocol between io_Server and client is - updated to handle protocol data in unicode formats. The - updated protocol is now documented. The specification - resides in the stdlib <em>users manual</em>, which is a - new part of the manual.</p> - <p><em>io module incompatibilities:</em></p> - <p>The io:put_chars, io:get_chars and io:get_line all - handle and return unicode data. In the case where - binaries can be provided (as to io:put_chars), they shall - be encoded in UTF-8. When binaries are returned (as by - io:get_line/get_chars when the io_server is set in - <em>binary mode</em>) the returned data is also - <em>always</em> encoded as UTF-8. The file module however - still returns byte-oriented data, why file:read can be - used instead of io:get_chars to read binary data in - ISO-latin-1.</p> - <p><em>io_lib module incompatibilities:</em></p> - <p>io_lib:format can, given new format directives (i.e - "~ts" and "~tc"), return lists containing integers larger - than 255. </p> - <p> - *** POTENTIAL INCOMPATIBILITY ***</p> - <p> - Own Id: OTP-7648 Aux Id: OTP-7580 OTP-7514 OTP-7494 - OTP-7443 OTP-7181 EEP10 EEP11 </p> - </item> - <item> - <p> - <c>filelib:fold_files/5</c> now uses the <c>re</c> module - instead of the <c>regexp</c> module for regular - expression matching. In practice, this change will not be - a problem for most regular expressions used for - <c>filelib:fold_files/5</c>. (The major difference in - regular expression is that parenthesis and curly brackets - is treated as literal characters by <c>regexp</c> but as - special characters by <c>re</c>; fortunately, those - characters are rarely used in filenames.)</p> - <p> - *** POTENTIAL INCOMPATIBILITY ***</p> + *** POTENTIAL INCOMPATIBILITY ***</p> <p> - Own Id: OTP-7819</p> + Own Id: OTP-8671</p> </item> + </list> + </section> + + <section> + <title>Erl_interface</title> + <list type="bulleted"> <item> - <p> - <c>digraph:new(Type)</c> will now cause a <c>badarg</c> - exception if <c>Type</c> is not a valid type. Similarly, - <c>digraph_utils:subgraph/2,3</c> will now cause a - <c>badarg</c> if the arguments are invalid. (Those - functions used to return error tuples if something was - wrong.)</p> + <p>Compact IEEE 754 double encoding in external binary format + for ei</p> + + <p>Implement the compact IEEE 754 double encoding in external + binary format for ei. Encoding for ei now always produces the + NEW_FLOAT_EXT format. Decoding and term printing handle both + the old ERL_FLOAT_EXT encoding and the new NEW_FLOAT_EXT + encoding.</p> + + <p>Legacy erl_interface code also handles the new encoding, but + still produces the ERL_FLOAT_EXT encoding by default.</p> + + <p>Also enable the DFLAG_NEW_FLOATS distribution flag.</p> + + <p>ei_get_type() will return ERL_FLOAT_EXT regardless if the + external format is encoded with ERL_FLOAT_EXT or + NEW_FLOAT_EXT for doubles.</p> + + <p>Reduce the number of copies of the code for encoding and + decoding doubles throughout ei and erl_interface by instead + calling the ei encoding and decoding functions wherever + possible.</p> + + <p>Restore commented-out float tests in ei_decode_SUITE and + ei_encode_SUITE in lib/erl_interface/test. Modify them to + make them match the style of other tests in the same suites.</p> + + <p>These changes are based on an ei float patch from Serge + Aleynikov originally submitted against R12B-2 in July 2008 + and reworked by Steve Vinoski May 2010.</p> <p> *** POTENTIAL INCOMPATIBILITY ***</p> <p> - Own Id: OTP-7824</p> + Own Id: OTP-8684</p> </item> + </list> + </section> + <section> + <title>Debugger and STDLIB</title> + <list type="bulleted"> <item> - <p>The argument passed to <c>random:uniform/1</c> must - now be an integer (as stated in the documentation). In - previous releases, a floating point number was also - allowed.</p> + <p> The predefined builtin type tid() has been removed. Instead, + ets:tid() should be used.</p> <p> *** POTENTIAL INCOMPATIBILITY ***</p> <p> - Own Id: OTP-7827</p> + Own Id: OTP-8687</p> </item> </list> </section> |
