aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--erts/doc/src/crash_dump.xml5
-rw-r--r--erts/doc/src/erl.xml20
-rw-r--r--erts/doc/src/erl_driver.xml3
-rw-r--r--erts/doc/src/erlang.xml11
-rw-r--r--erts/emulator/beam/erl_alloc.types4
-rw-r--r--erts/emulator/beam/erl_bif_info.c36
-rw-r--r--erts/emulator/beam/erl_bif_port.c15
-rw-r--r--erts/emulator/beam/erl_init.c18
-rw-r--r--erts/emulator/beam/erl_port_task.c24
-rw-r--r--erts/emulator/beam/erl_port_task.h2
-rw-r--r--erts/emulator/beam/erl_process.c68
-rw-r--r--erts/emulator/beam/erl_process.h1
-rw-r--r--erts/emulator/beam/sys.h12
-rw-r--r--erts/emulator/beam/utils.c1
-rw-r--r--erts/emulator/sys/common/erl_check_io.c694
-rw-r--r--erts/emulator/sys/common/erl_check_io.h45
-rw-r--r--erts/emulator/sys/common/erl_sys_common_misc.c8
-rw-r--r--erts/emulator/sys/unix/erl_unix_sys.h3
-rw-r--r--erts/emulator/sys/unix/sys.c6
-rw-r--r--erts/emulator/sys/win32/erl_poll.c2
-rw-r--r--erts/emulator/sys/win32/erl_win_sys.h4
-rw-r--r--erts/emulator/test/a_SUITE.erl14
-rw-r--r--erts/emulator/test/driver_SUITE.erl62
-rw-r--r--erts/emulator/test/port_SUITE.erl51
-rw-r--r--erts/emulator/test/z_SUITE.erl28
-rw-r--r--erts/etc/common/erlexec.c1
-rw-r--r--erts/preloaded/ebin/erlang.beambin97908 -> 97972 bytes
-rw-r--r--erts/preloaded/src/erlang.erl1
-rw-r--r--lib/asn1/c_src/asn1_erl_nif.c2
-rw-r--r--lib/asn1/test/asn1_SUITE_data/Constructed.asn6
-rw-r--r--lib/asn1/test/ber_decode_error.erl4
-rw-r--r--lib/common_test/src/ct_logs.erl67
-rw-r--r--lib/debugger/src/dbg_icmd.erl6
-rw-r--r--lib/debugger/src/dbg_ieval.erl12
-rw-r--r--lib/debugger/src/int.erl5
-rw-r--r--lib/eldap/test/eldap_connections_SUITE.erl85
-rw-r--r--lib/inets/src/ftp/ftp.erl77
-rw-r--r--lib/inets/src/http_client/httpc_handler.erl4
-rw-r--r--lib/kernel/src/application_master.erl6
-rw-r--r--lib/observer/doc/src/observer_ug.xml5
-rw-r--r--lib/parsetools/include/leexinc.hrl12
-rw-r--r--lib/parsetools/test/leex_SUITE.erl51
-rw-r--r--lib/ssh/doc/src/notes.xml47
-rw-r--r--lib/ssh/doc/src/ssh_sftp.xml13
-rw-r--r--lib/ssh/src/Makefile1
-rw-r--r--lib/ssh/src/ssh.app.src1
-rw-r--r--lib/ssh/src/ssh.appup.src40
-rw-r--r--lib/ssh/src/ssh_acceptor.erl4
-rw-r--r--lib/ssh/src/ssh_auth.erl110
-rw-r--r--lib/ssh/src/ssh_channel.erl14
-rw-r--r--lib/ssh/src/ssh_connection.erl14
-rw-r--r--lib/ssh/src/ssh_connection_handler.erl99
-rw-r--r--lib/ssh/src/ssh_info.erl193
-rw-r--r--lib/ssh/src/ssh_io.erl6
-rw-r--r--lib/ssh/src/ssh_message.erl18
-rw-r--r--lib/ssh/src/ssh_sftp.erl35
-rw-r--r--lib/ssh/src/ssh_xfer.erl8
-rw-r--r--lib/ssh/test/property_test/ssh_eqc_client_server.erl63
-rw-r--r--lib/ssh/test/property_test/ssh_eqc_encode_decode.erl2
-rw-r--r--lib/ssh/test/ssh_connection_SUITE.erl27
-rw-r--r--lib/ssh/test/ssh_sftp_SUITE.erl24
-rw-r--r--lib/ssh/vsn.mk2
-rw-r--r--lib/ssl/doc/src/notes.xml18
-rw-r--r--lib/ssl/src/ssl.appup.src2
-rw-r--r--lib/ssl/vsn.mk2
-rw-r--r--lib/stdlib/src/dets.erl25
-rw-r--r--lib/stdlib/src/dets_server.erl18
-rw-r--r--lib/stdlib/test/dets_SUITE.erl138
-rw-r--r--otp_versions.table2
69 files changed, 1937 insertions, 470 deletions
diff --git a/erts/doc/src/crash_dump.xml b/erts/doc/src/crash_dump.xml
index d3de29b876..2b5fc877c3 100644
--- a/erts/doc/src/crash_dump.xml
+++ b/erts/doc/src/crash_dump.xml
@@ -115,8 +115,9 @@
sockets/pipes can be used simultaneously by Erlang (due to
limitations in the Unix <c><![CDATA[select]]></c> call). The number of
open regular files is not affected by this.</item>
- <item>"Received SIGUSR1" - The SIGUSR1 signal was sent to the
- Erlang machine (Unix only).</item>
+ <item>"Received SIGUSR1" - Sending the SIGUSR1 signal to a
+ Erlang machine (Unix only) forces a crash dump. This slogan reflects
+ that the Erlang machine crash-dumped due to receiving that signal.</item>
<item>"Kernel pid terminated (<em>Who</em>)
(<em>Exit-reason</em>)" - The kernel supervisor has detected
a failure, usually that the <c><![CDATA[application_controller]]></c>
diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml
index f856b9ab86..d11f6b0c6d 100644
--- a/erts/doc/src/erl.xml
+++ b/erts/doc/src/erl.xml
@@ -525,7 +525,8 @@
core dump and no crash dump if an internal error is detected.</p>
<p>Calling <c>erlang:halt/1</c> with a string argument will still
- produce a crash dump.</p>
+ produce a crash dump. On Unix systems, sending an emulator process
+ a SIGUSR1 signal will also force a crash dump.</p>
</item>
<tag><marker id="+e"><c><![CDATA[+e Number]]></c></marker></tag>
<item>
@@ -1141,6 +1142,23 @@
<p>For more information, see
<seealso marker="erlang#system_info_cpu_topology">erlang:system_info(cpu_topology)</seealso>.</p>
</item>
+ <tag><marker id="+secio"><c>+secio true|false</c></marker></tag>
+ <item>
+ <p>Enable or disable eager check I/O scheduling. The default
+ is currently <c>false</c>, but will most likely be changed
+ to <c>true</c> in OTP 18. The behaviour before this flag
+ was introduced corresponds to <c>+secio false</c>.</p>
+ <p>The flag effects when schedulers will check for I/O
+ operations possible to execute, and when such I/O operations
+ will execute. As the name of the parameter implies,
+ schedulers will be more eager to check for I/O when
+ <c>true</c> is passed. This however also implies that
+ execution of outstanding I/O operation will not be
+ prioritized to the same extent as when <c>false</c> is
+ passed.</p>
+ <p><seealso marker="erlang#system_info_eager_check_io"><c>erlang:system_info(eager_check_io)</c></seealso>
+ returns the value of this parameter used when starting the VM.</p>
+ </item>
<tag><marker id="+sfwi"><c>+sfwi Interval</c></marker></tag>
<item>
<p>Set scheduler forced wakeup interval. All run queues will
diff --git a/erts/doc/src/erl_driver.xml b/erts/doc/src/erl_driver.xml
index 4a1aab75c7..77fc906aca 100644
--- a/erts/doc/src/erl_driver.xml
+++ b/erts/doc/src/erl_driver.xml
@@ -2033,7 +2033,8 @@ ERL_DRV_MAP int sz
entry function is called. If <c>ready_async</c> is null in
the driver entry, the <c>async_free</c> function is called
instead.</p>
- <p>The return value is a handle to the asynchronous task.</p>
+ <p>The return value is -1 if the <c>driver_async</c> call
+ fails.</p>
<note>
<p>As of erts version 5.5.4.3 the default stack size for
threads in the async-thread pool is 16 kilowords,
diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml
index 84168397f6..111756407f 100644
--- a/erts/doc/src/erlang.xml
+++ b/erts/doc/src/erlang.xml
@@ -5775,6 +5775,7 @@ ok
<name name="system_info" arity="1" clause_i="52"/>
<name name="system_info" arity="1" clause_i="53"/>
<name name="system_info" arity="1" clause_i="54"/>
+ <name name="system_info" arity="1" clause_i="55"/>
<fsummary>Information about the system</fsummary>
<desc>
<p>Returns various information about the current system
@@ -5970,6 +5971,16 @@ ok
The return value will always be <c>false</c> since
the elib_malloc allocator has been removed.</p>
</item>
+ <tag><marker id="system_info_eager_check_io"><c>eager_check_io</c></marker></tag>
+ <item>
+ <p>
+ Returns the value of the <c>erl</c>
+ <seealso marker="erl#+secio">+secio</seealso> command line
+ flag which is either <c>true</c> or <c>false</c>. See the
+ documentation of the command line flag for information about
+ the different values.
+ </p>
+ </item>
<tag><c>ets_limit</c></tag>
<item>
<p>Returns the maximum number of ETS tables allowed. This limit
diff --git a/erts/emulator/beam/erl_alloc.types b/erts/emulator/beam/erl_alloc.types
index 37354b7f8d..21434eb117 100644
--- a/erts/emulator/beam/erl_alloc.types
+++ b/erts/emulator/beam/erl_alloc.types
@@ -267,7 +267,6 @@ type CODE_IX_LOCK_Q SHORT_LIVED SYSTEM code_ix_lock_q
type PROC_INTERVAL LONG_LIVED SYSTEM process_interval
type BUSY_CALLER_TAB SHORT_LIVED SYSTEM busy_caller_table
type BUSY_CALLER SHORT_LIVED SYSTEM busy_caller
-type PORT_DATA_HEAP STANDARD SYSTEM port_data_heap
type PROC_SYS_TSK SHORT_LIVED PROCESSES proc_sys_task
type PROC_SYS_TSK_QS SHORT_LIVED PROCESSES proc_sys_task_queues
@@ -364,6 +363,7 @@ type NLINK_SH STANDARD_LOW PROCESSES nlink_sh
type AINFO_REQ STANDARD_LOW SYSTEM alloc_info_request
type SCHED_WTIME_REQ STANDARD_LOW SYSTEM sched_wall_time_request
type GC_INFO_REQ STANDARD_LOW SYSTEM gc_info_request
+type PORT_DATA_HEAP STANDARD_LOW SYSTEM port_data_heap
+else # "fullword"
@@ -383,6 +383,7 @@ type NLINK_SH FIXED_SIZE PROCESSES nlink_sh
type AINFO_REQ SHORT_LIVED SYSTEM alloc_info_request
type SCHED_WTIME_REQ SHORT_LIVED SYSTEM sched_wall_time_request
type GC_INFO_REQ SHORT_LIVED SYSTEM gc_info_request
+type PORT_DATA_HEAP STANDARD SYSTEM port_data_heap
+endif
@@ -397,6 +398,7 @@ type DRV_EV_STATE LONG_LIVED SYSTEM driver_event_state
type DRV_EV_D_STATE FIXED_SIZE SYSTEM driver_event_data_state
type DRV_SEL_D_STATE FIXED_SIZE SYSTEM driver_select_data_state
type FD_LIST SHORT_LIVED SYSTEM fd_list
+type ACTIVE_FD_ARR SHORT_LIVED SYSTEM active_fd_array
type POLLSET LONG_LIVED SYSTEM pollset
type POLLSET_UPDREQ SHORT_LIVED SYSTEM pollset_update_req
type POLL_FDS LONG_LIVED SYSTEM poll_fds
diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c
index 6efe9d9550..61e4469600 100644
--- a/erts/emulator/beam/erl_bif_info.c
+++ b/erts/emulator/beam/erl_bif_info.c
@@ -2696,6 +2696,9 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1)
? am_disabled
: am_enabled);
}
+ else if (ERTS_IS_ATOM_STR("eager_check_io",BIF_ARG_1)) {
+ BIF_RET(erts_eager_check_io ? am_true : am_false);
+ }
BIF_ERROR(BIF_P, BADARG);
}
@@ -3304,17 +3307,38 @@ BIF_RETTYPE erts_debug_get_internal_state_1(BIF_ALIST_1)
BIF_RET(make_small((Uint) words));
}
else if (ERTS_IS_ATOM_STR("check_io_debug", BIF_ARG_1)) {
- /* Used by (emulator) */
- int res;
+ /* Used by driver_SUITE (emulator) */
+ Uint sz, *szp;
+ Eterm res, *hp, **hpp;
+ int no_errors;
+ ErtsCheckIoDebugInfo ciodi = {0};
#ifdef HAVE_ERTS_CHECK_IO_DEBUG
erts_smp_proc_unlock(BIF_P,ERTS_PROC_LOCK_MAIN);
- res = erts_check_io_debug();
+ no_errors = erts_check_io_debug(&ciodi);
erts_smp_proc_lock(BIF_P,ERTS_PROC_LOCK_MAIN);
#else
- res = 0;
+ no_errors = 0;
#endif
- ASSERT(res >= 0);
- BIF_RET(erts_make_integer((Uint) res, BIF_P));
+ sz = 0;
+ szp = &sz;
+ hpp = NULL;
+ while (1) {
+ res = erts_bld_tuple(hpp, szp, 4,
+ erts_bld_uint(hpp, szp,
+ (Uint) no_errors),
+ erts_bld_uint(hpp, szp,
+ (Uint) ciodi.no_used_fds),
+ erts_bld_uint(hpp, szp,
+ (Uint) ciodi.no_driver_select_structs),
+ erts_bld_uint(hpp, szp,
+ (Uint) ciodi.no_driver_event_structs));
+ if (hpp)
+ break;
+ hp = HAlloc(BIF_P, sz);
+ szp = NULL;
+ hpp = &hp;
+ }
+ BIF_RET(res);
}
else if (ERTS_IS_ATOM_STR("process_info_args", BIF_ARG_1)) {
/* Used by process_SUITE (emulator) */
diff --git a/erts/emulator/beam/erl_bif_port.c b/erts/emulator/beam/erl_bif_port.c
index afb33c1cdb..64bd598ba6 100644
--- a/erts/emulator/beam/erl_bif_port.c
+++ b/erts/emulator/beam/erl_bif_port.c
@@ -493,8 +493,8 @@ void
erts_cleanup_port_data(Port *prt)
{
ASSERT(erts_atomic32_read_nob(&prt->state) & ERTS_PORT_SFLGS_INVALID_LOOKUP);
- cleanup_old_port_data(erts_smp_atomic_read_nob(&prt->data));
- erts_smp_atomic_set_nob(&prt->data, (erts_aint_t) THE_NON_VALUE);
+ cleanup_old_port_data(erts_smp_atomic_xchg_nob(&prt->data,
+ (erts_aint_t) NULL));
}
Uint
@@ -554,6 +554,7 @@ BIF_RETTYPE port_set_data_2(BIF_ALIST_2)
hp = &pdhp->heap[0];
pdhp->off_heap.first = NULL;
pdhp->off_heap.overhead = 0;
+ pdhp->hsize = hsize;
pdhp->data = copy_struct(BIF_ARG_2, hsize, &hp, &pdhp->off_heap);
data = (erts_aint_t) pdhp;
ASSERT((data & 0x3) == 0);
@@ -561,8 +562,14 @@ BIF_RETTYPE port_set_data_2(BIF_ALIST_2)
data = erts_smp_atomic_xchg_wb(&prt->data, data);
+ if (data == (erts_aint_t)NULL) {
+ /* Port terminated by racing thread */
+ data = erts_smp_atomic_xchg_wb(&prt->data, data);
+ ASSERT(data != (erts_aint_t)NULL);
+ cleanup_old_port_data(data);
+ BIF_ERROR(BIF_P, BADARG);
+ }
cleanup_old_port_data(data);
-
BIF_RET(am_true);
}
@@ -581,6 +588,8 @@ BIF_RETTYPE port_get_data_1(BIF_ALIST_1)
BIF_ERROR(BIF_P, BADARG);
data = erts_smp_atomic_read_ddrb(&prt->data);
+ if (data == (erts_aint_t)NULL)
+ BIF_ERROR(BIF_P, BADARG); /* Port terminated by racing thread */
if ((data & 0x3) != 0) {
res = (Eterm) (UWord) data;
diff --git a/erts/emulator/beam/erl_init.c b/erts/emulator/beam/erl_init.c
index 88c4006934..61f8385efc 100644
--- a/erts/emulator/beam/erl_init.c
+++ b/erts/emulator/beam/erl_init.c
@@ -548,6 +548,8 @@ void erts_usage(void)
erts_fprintf(stderr, " see the erl(1) documentation for more info.\n");
erts_fprintf(stderr, "-sct cput set cpu topology,\n");
erts_fprintf(stderr, " see the erl(1) documentation for more info.\n");
+ erts_fprintf(stderr, "-secio bool enable/disable eager check I/O scheduling,\n");
+ erts_fprintf(stderr, " see the erl(1) documentation for more info.\n");
#if ERTS_HAVE_SCHED_UTIL_BALANCING_SUPPORT_OPT
erts_fprintf(stderr, "-sub bool enable/disable scheduler utilization balancing,\n");
#else
@@ -1674,6 +1676,22 @@ erl_start(int argc, char **argv)
erts_usage();
}
}
+ else if (has_prefix("ecio", sub_param)) {
+ arg = get_arg(sub_param+4, argv[i+1], &i);
+#ifndef __OSE__
+ if (sys_strcmp("true", arg) == 0)
+ erts_eager_check_io = 1;
+ else
+#endif
+ if (sys_strcmp("false", arg) == 0)
+ erts_eager_check_io = 0;
+ else {
+ erts_fprintf(stderr,
+ "bad schedule eager check I/O value '%s'\n",
+ arg);
+ erts_usage();
+ }
+ }
else if (has_prefix("pp", sub_param)) {
arg = get_arg(sub_param+2, argv[i+1], &i);
if (sys_strcmp(arg, "true") == 0)
diff --git a/erts/emulator/beam/erl_port_task.c b/erts/emulator/beam/erl_port_task.c
index 682f6f8f4b..2aa0a27197 100644
--- a/erts/emulator/beam/erl_port_task.c
+++ b/erts/emulator/beam/erl_port_task.c
@@ -32,6 +32,7 @@
#include "global.h"
#include "erl_port_task.h"
#include "dist.h"
+#include "erl_check_io.h"
#include "dtrace-wrapper.h"
#include <stdarg.h>
@@ -550,6 +551,16 @@ reset_handle(ErtsPortTask *ptp)
}
static ERTS_INLINE void
+reset_executed_io_task_handle(ErtsPortTask *ptp)
+{
+ if (ptp->u.alive.handle) {
+ ASSERT(ptp == handle2task(ptp->u.alive.handle));
+ erts_io_notify_port_task_executed(ptp->u.alive.handle);
+ reset_port_task_handle(ptp->u.alive.handle);
+ }
+}
+
+static ERTS_INLINE void
set_handle(ErtsPortTask *ptp, ErtsPortTaskHandle *pthp)
{
ptp->u.alive.handle = pthp;
@@ -1396,10 +1407,7 @@ erts_port_task_schedule(Eterm id,
erts_aint32_t act, add_flags;
unsigned int prof_runnable_ports;
- if (pthp && erts_port_task_is_scheduled(pthp)) {
- ASSERT(0);
- erts_port_task_abort(pthp);
- }
+ ERTS_LC_ASSERT(!pthp || !erts_port_task_is_scheduled(pthp));
ASSERT(is_internal_port(id));
@@ -1699,8 +1707,6 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp)
goto aborted_port_task;
}
- reset_handle(ptp);
-
if (erts_system_monitor_long_schedule != 0) {
start_time = erts_timestamp_millis();
}
@@ -1711,6 +1717,7 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp)
switch (ptp->type) {
case ERTS_PORT_TASK_TIMEOUT:
+ reset_handle(ptp);
reds = ERTS_PORT_REDS_TIMEOUT;
if (!(state & ERTS_PORT_SFLGS_DEAD)) {
DTRACE_DRIVER(driver_timeout, pp);
@@ -1725,6 +1732,7 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp)
for input and output */
(*pp->drv_ptr->ready_input)((ErlDrvData) pp->drv_data,
ptp->u.alive.td.io.event);
+ reset_executed_io_task_handle(ptp);
io_tasks_executed++;
break;
case ERTS_PORT_TASK_OUTPUT:
@@ -1733,6 +1741,7 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp)
DTRACE_DRIVER(driver_ready_output, pp);
(*pp->drv_ptr->ready_output)((ErlDrvData) pp->drv_data,
ptp->u.alive.td.io.event);
+ reset_executed_io_task_handle(ptp);
io_tasks_executed++;
break;
case ERTS_PORT_TASK_EVENT:
@@ -1742,10 +1751,12 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp)
(*pp->drv_ptr->event)((ErlDrvData) pp->drv_data,
ptp->u.alive.td.io.event,
ptp->u.alive.td.io.event_data);
+ reset_executed_io_task_handle(ptp);
io_tasks_executed++;
break;
case ERTS_PORT_TASK_PROC_SIG: {
ErtsProc2PortSigData *sigdp = &ptp->u.alive.td.psig.data;
+ reset_handle(ptp);
ASSERT((state & ERTS_PORT_SFLGS_DEAD) == 0);
if (!pp->sched.taskq.bpq)
reds = ptp->u.alive.td.psig.callback(pp,
@@ -1763,6 +1774,7 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp)
break;
}
case ERTS_PORT_TASK_DIST_CMD:
+ reset_handle(ptp);
reds = erts_dist_command(pp, CONTEXT_REDS - pp->reds);
break;
default:
diff --git a/erts/emulator/beam/erl_port_task.h b/erts/emulator/beam/erl_port_task.h
index 9ef0cfcedc..406cd3c492 100644
--- a/erts/emulator/beam/erl_port_task.h
+++ b/erts/emulator/beam/erl_port_task.h
@@ -156,7 +156,7 @@ erts_port_task_handle_init(ErtsPortTaskHandle *pthp)
ERTS_GLB_INLINE int
erts_port_task_is_scheduled(ErtsPortTaskHandle *pthp)
{
- return ((void *) erts_smp_atomic_read_nob(pthp)) != NULL;
+ return ((void *) erts_smp_atomic_read_acqb(pthp)) != NULL;
}
ERTS_GLB_INLINE void erts_port_task_pre_init_sched(ErtsPortTaskSched *ptsp,
diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c
index 20a88ec581..e5bb1203c8 100644
--- a/erts/emulator/beam/erl_process.c
+++ b/erts/emulator/beam/erl_process.c
@@ -148,6 +148,12 @@ extern BeamInstr beam_apply[];
extern BeamInstr beam_exit[];
extern BeamInstr beam_continue_exit[];
+#ifdef __OSE__
+/* Eager check I/O not supported on OSE yet. */
+int erts_eager_check_io = 0;
+#else
+int erts_eager_check_io = 0;
+#endif
int erts_sched_compact_load;
int erts_sched_balance_util = 0;
Uint erts_no_schedulers;
@@ -2381,29 +2387,47 @@ try_set_sys_scheduling(void)
#endif
static ERTS_INLINE int
-prepare_for_sys_schedule(ErtsSchedulerData *esdp)
+prepare_for_sys_schedule(ErtsSchedulerData *esdp, int non_blocking)
{
+ if (non_blocking && erts_eager_check_io) {
#ifdef ERTS_SMP
- while (!erts_port_task_have_outstanding_io_tasks()
- && try_set_sys_scheduling()) {
#ifdef ERTS_SCHED_ONLY_POLL_SCHED_1
- if (esdp->no != 1) {
- /* If we are not scheduler 1 and ERTS_SCHED_ONLY_POLL_SCHED_1 is used
- then we make sure to wake scheduler 1 */
- ErtsRunQueue *rq = ERTS_RUNQ_IX(0);
- clear_sys_scheduling();
- wake_scheduler(rq);
- return 0;
- }
+ if (esdp->no != 1) {
+ /* If we are not scheduler 1 and ERTS_SCHED_ONLY_POLL_SCHED_1 is used
+ then we make sure to wake scheduler 1 */
+ ErtsRunQueue *rq = ERTS_RUNQ_IX(0);
+ wake_scheduler(rq);
+ return 0;
+ }
#endif
- if (!erts_port_task_have_outstanding_io_tasks())
+ return try_set_sys_scheduling();
+#else
return 1;
- clear_sys_scheduling();
+#endif
}
- return 0;
+ else {
+#ifdef ERTS_SMP
+ while (!erts_port_task_have_outstanding_io_tasks()
+ && try_set_sys_scheduling()) {
+#ifdef ERTS_SCHED_ONLY_POLL_SCHED_1
+ if (esdp->no != 1) {
+ /* If we are not scheduler 1 and ERTS_SCHED_ONLY_POLL_SCHED_1 is used
+ then we make sure to wake scheduler 1 */
+ ErtsRunQueue *rq = ERTS_RUNQ_IX(0);
+ clear_sys_scheduling();
+ wake_scheduler(rq);
+ return 0;
+ }
+#endif
+ if (!erts_port_task_have_outstanding_io_tasks())
+ return 1;
+ clear_sys_scheduling();
+ }
+ return 0;
#else
- return !erts_port_task_have_outstanding_io_tasks();
+ return !erts_port_task_have_outstanding_io_tasks();
#endif
+ }
}
#ifdef ERTS_SMP
@@ -2780,7 +2804,7 @@ scheduler_wait(int *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq)
* be waiting in erl_sys_schedule()
*/
- if (ERTS_SCHEDULER_IS_DIRTY(esdp) || !prepare_for_sys_schedule(esdp)) {
+ if (ERTS_SCHEDULER_IS_DIRTY(esdp) || !prepare_for_sys_schedule(esdp, 0)) {
sched_waiting(esdp->no, rq);
@@ -2944,7 +2968,7 @@ scheduler_wait(int *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq)
* Got to check that we still got I/O tasks; otherwise
* we have to continue checking for I/O...
*/
- if (!prepare_for_sys_schedule(esdp)) {
+ if (!prepare_for_sys_schedule(esdp, 0)) {
spincount *= ERTS_SCHED_TSE_SLEEP_SPINCOUNT_FACT;
goto tse_wait;
}
@@ -2966,7 +2990,7 @@ scheduler_wait(int *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq)
* Got to check that we still got I/O tasks; otherwise
* we have to wait in erl_sys_schedule() after all...
*/
- if (!prepare_for_sys_schedule(esdp)) {
+ if (!prepare_for_sys_schedule(esdp, 0)) {
/*
* Not allowed to wait in erl_sys_schedule;
* do tse wait instead...
@@ -9200,7 +9224,7 @@ Process *schedule(Process *p, int calls)
}
else if (!ERTS_SCHEDULER_IS_DIRTY(esdp) &&
(fcalls > input_reductions &&
- prepare_for_sys_schedule(esdp))) {
+ prepare_for_sys_schedule(esdp, !0))) {
/*
* Schedule system-level activities.
*/
@@ -9208,8 +9232,6 @@ Process *schedule(Process *p, int calls)
erts_smp_atomic32_set_relb(&function_calls, 0);
fcalls = 0;
- ASSERT(!erts_port_task_have_outstanding_io_tasks());
-
#if 0 /* Not needed since we wont wait in sys schedule */
erts_sys_schedule_interrupt(0);
#endif
@@ -9241,7 +9263,9 @@ Process *schedule(Process *p, int calls)
if (RUNQ_READ_LEN(&rq->ports.info.len)) {
int have_outstanding_io;
have_outstanding_io = erts_port_task_execute(rq, &esdp->current_port);
- if ((have_outstanding_io && fcalls > 2*input_reductions)
+ if ((!erts_eager_check_io
+ && have_outstanding_io
+ && fcalls > 2*input_reductions)
|| rq->halt_in_progress) {
/*
* If we have performed more than 2*INPUT_REDUCTIONS since
diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h
index 3b0798207e..27a3a3553b 100644
--- a/erts/emulator/beam/erl_process.h
+++ b/erts/emulator/beam/erl_process.h
@@ -104,6 +104,7 @@ struct saved_calls {
};
extern Export exp_send, exp_receive, exp_timeout;
+extern int erts_eager_check_io;
extern int erts_sched_compact_load;
extern int erts_sched_balance_util;
extern Uint erts_no_schedulers;
diff --git a/erts/emulator/beam/sys.h b/erts/emulator/beam/sys.h
index 3d8dd9c6d0..1bdb1579f5 100644
--- a/erts/emulator/beam/sys.h
+++ b/erts/emulator/beam/sys.h
@@ -66,8 +66,12 @@
*/
#ifndef ERTS_SYS_FD_TYPE
+#define ERTS_SYS_FD_INVALID ((ErtsSysFdType) -1)
typedef int ErtsSysFdType;
#else
+#ifndef ERTS_SYS_FD_INVALID
+# error missing ERTS_SYS_FD_INVALID
+#endif
typedef ERTS_SYS_FD_TYPE ErtsSysFdType;
#endif
@@ -744,6 +748,14 @@ void init_getenv_state(GETENV_STATE *);
char * getenv_string(GETENV_STATE *);
void fini_getenv_state(GETENV_STATE *);
+#define HAVE_ERTS_CHECK_IO_DEBUG
+typedef struct {
+ int no_used_fds;
+ int no_driver_select_structs;
+ int no_driver_event_structs;
+} ErtsCheckIoDebugInfo;
+int erts_check_io_debug(ErtsCheckIoDebugInfo *ip);
+
/* xxxP */
#define SYS_DEFAULT_FLOAT_DECIMALS 20
void init_sys_float(void);
diff --git a/erts/emulator/beam/utils.c b/erts/emulator/beam/utils.c
index 55f9e68e78..f20e6e5665 100644
--- a/erts/emulator/beam/utils.c
+++ b/erts/emulator/beam/utils.c
@@ -48,6 +48,7 @@
#include "erl_sched_spec_pre_alloc.h"
#include "beam_bp.h"
#include "erl_ptab.h"
+#include "erl_check_io.h"
#undef M_TRIM_THRESHOLD
#undef M_TOP_PAD
diff --git a/erts/emulator/sys/common/erl_check_io.c b/erts/emulator/sys/common/erl_check_io.c
index 1db673e7f3..81cb5dc4bb 100644
--- a/erts/emulator/sys/common/erl_check_io.c
+++ b/erts/emulator/sys/common/erl_check_io.c
@@ -52,8 +52,17 @@ typedef char EventStateType;
#define ERTS_EV_TYPE_STOP_USE ((EventStateType) 3) /* pending stop_select */
typedef char EventStateFlags;
-#define ERTS_EV_FLAG_USED ((EventStateFlags) 1) /* ERL_DRV_USE has been turned on */
+#define ERTS_EV_FLAG_USED ((EventStateFlags) 1) /* ERL_DRV_USE has been turned on */
+#define ERTS_EV_FLAG_DEFER_IN_EV ((EventStateFlags) 2)
+#define ERTS_EV_FLAG_DEFER_OUT_EV ((EventStateFlags) 4)
+#ifdef DEBUG
+# define ERTS_ACTIVE_FD_INC 2
+#else
+# define ERTS_ACTIVE_FD_INC 128
+#endif
+
+#define ERTS_CHECK_IO_POLL_RES_LEN 512
#if defined(ERTS_KERNEL_POLL_VERSION)
# define ERTS_CIO_EXPORT(FUNC) FUNC ## _kp
@@ -67,6 +76,7 @@ typedef char EventStateFlags;
(ERTS_POLL_USE_POLL && !ERTS_POLL_USE_KERNEL_POLL)
#define ERTS_CIO_POLL_CTL ERTS_POLL_EXPORT(erts_poll_control)
+#define ERTS_CIO_POLL_CTLV ERTS_POLL_EXPORT(erts_poll_controlv)
#define ERTS_CIO_POLL_WAIT ERTS_POLL_EXPORT(erts_poll_wait)
#ifdef ERTS_POLL_NEED_ASYNC_INTERRUPT_SUPPORT
#define ERTS_CIO_POLL_AS_INTR ERTS_POLL_EXPORT(erts_poll_async_sig_interrupt)
@@ -85,6 +95,13 @@ static struct pollset_info
{
ErtsPollSet ps;
erts_smp_atomic_t in_poll_wait; /* set while doing poll */
+ struct {
+ int six; /* start index */
+ int eix; /* end index */
+ erts_smp_atomic32_t no;
+ int size;
+ ErtsSysFdType *array;
+ } active_fd;
#ifdef ERTS_SMP
struct removed_fd* removed_list; /* list of deselected fd's*/
erts_smp_spinlock_t removed_list_lock;
@@ -97,9 +114,11 @@ typedef struct {
SafeHashBucket hb;
#endif
ErtsSysFdType fd;
- union {
- ErtsDrvEventDataState *event; /* ERTS_EV_TYPE_DRV_EV */
+ struct {
ErtsDrvSelectDataState *select; /* ERTS_EV_TYPE_DRV_SEL */
+#if ERTS_CIO_HAVE_DRV_EVENT
+ ErtsDrvEventDataState *event; /* ERTS_EV_TYPE_DRV_EV */
+#endif
erts_driver_t* drv_ptr; /* ERTS_EV_TYPE_STOP_USE */
} driver;
ErtsPollEvents events;
@@ -169,6 +188,10 @@ static ERTS_INLINE ErtsDrvEventState* hash_new_drv_ev_state(ErtsSysFdType fd)
ErtsDrvEventState tmpl;
tmpl.fd = fd;
tmpl.driver.select = NULL;
+#if ERTS_CIO_HAVE_DRV_EVENT
+ tmpl.driver.event = NULL;
+#endif
+ tmpl.driver.drv_ptr = NULL;
tmpl.events = 0;
tmpl.remove_cnt = 0;
tmpl.type = ERTS_EV_TYPE_NONE;
@@ -209,6 +232,65 @@ ERTS_SCHED_PREF_QUICK_ALLOC_IMPL(removed_fd, struct removed_fd, 64, ERTS_ALC_T_F
#endif
static ERTS_INLINE void
+init_iotask(ErtsIoTask *io_task)
+{
+ erts_port_task_handle_init(&io_task->task);
+ erts_smp_atomic_init_nob(&io_task->executed_time, ~((erts_aint_t) 0));
+}
+
+static ERTS_INLINE int
+is_iotask_active(ErtsIoTask *io_task, erts_aint_t current_cio_time)
+{
+ if (erts_port_task_is_scheduled(&io_task->task))
+ return 1;
+ if (erts_smp_atomic_read_nob(&io_task->executed_time) == current_cio_time)
+ return 1;
+ return 0;
+}
+
+static ERTS_INLINE ErtsDrvSelectDataState *
+alloc_drv_select_data(void)
+{
+ ErtsDrvSelectDataState *dsp = erts_alloc(ERTS_ALC_T_DRV_SEL_D_STATE,
+ sizeof(ErtsDrvSelectDataState));
+ dsp->inport = NIL;
+ dsp->outport = NIL;
+ init_iotask(&dsp->iniotask);
+ init_iotask(&dsp->outiotask);
+ return dsp;
+}
+
+static ERTS_INLINE void
+free_drv_select_data(ErtsDrvSelectDataState *dsp)
+{
+ ASSERT(!erts_port_task_is_scheduled(&dsp->iniotask.task));
+ ASSERT(!erts_port_task_is_scheduled(&dsp->outiotask.task));
+ erts_free(ERTS_ALC_T_DRV_SEL_D_STATE, dsp);
+}
+
+static ERTS_INLINE ErtsDrvEventDataState *
+alloc_drv_event_data(void)
+{
+ ErtsDrvEventDataState *dep = erts_alloc(ERTS_ALC_T_DRV_EV_D_STATE,
+ sizeof(ErtsDrvEventDataState));
+ dep->port = NIL;
+ dep->data = NULL;
+ dep->removed_events = 0;
+#if ERTS_CIO_DEFER_ACTIVE_EVENTS
+ dep->deferred_events = 0;
+#endif
+ init_iotask(&dep->iotask);
+ return dep;
+}
+
+static ERTS_INLINE void
+free_drv_event_data(ErtsDrvEventDataState *dep)
+{
+ ASSERT(!erts_port_task_is_scheduled(&dep->iotask.task));
+ erts_free(ERTS_ALC_T_DRV_EV_D_STATE, dep);
+}
+
+static ERTS_INLINE void
remember_removed(ErtsDrvEventState *state, struct pollset_info* psi)
{
#ifdef ERTS_SMP
@@ -288,7 +370,7 @@ forget_removed(struct pollset_info* psi)
drv_ptr = state->driver.drv_ptr;
ASSERT(drv_ptr);
state->type = ERTS_EV_TYPE_NONE;
- state->flags = 0;
+ state->flags &= ~ERTS_EV_FLAG_USED;
state->driver.drv_ptr = NULL;
/* Fall through */
case ERTS_EV_TYPE_NONE:
@@ -345,6 +427,10 @@ grow_drv_ev_state(int min_ix)
for (i = erts_smp_atomic_read_nob(&drv_ev_state_len); i < new_len; i++) {
drv_ev_state[i].fd = (ErtsSysFdType) i;
drv_ev_state[i].driver.select = NULL;
+#if ERTS_CIO_HAVE_DRV_EVENT
+ drv_ev_state[i].driver.event = NULL;
+#endif
+ drv_ev_state[i].driver.drv_ptr = NULL;
drv_ev_state[i].events = 0;
drv_ev_state[i].remove_cnt = 0;
drv_ev_state[i].type = ERTS_EV_TYPE_NONE;
@@ -365,11 +451,7 @@ grow_drv_ev_state(int min_ix)
static ERTS_INLINE void
abort_task(Eterm id, ErtsPortTaskHandle *pthp, EventStateType type)
{
- if (is_nil(id)) {
- ASSERT(type == ERTS_EV_TYPE_NONE
- || !erts_port_task_is_scheduled(pthp));
- }
- else if (erts_port_task_is_scheduled(pthp)) {
+ if (is_not_nil(id) && erts_port_task_is_scheduled(pthp)) {
erts_port_task_abort(pthp);
ASSERT(erts_is_port_alive(id));
}
@@ -384,7 +466,7 @@ abort_tasks(ErtsDrvEventState *state, int mode)
#if ERTS_CIO_HAVE_DRV_EVENT
case ERTS_EV_TYPE_DRV_EV:
abort_task(state->driver.event->port,
- &state->driver.event->task,
+ &state->driver.event->iotask.task,
ERTS_EV_TYPE_DRV_EV);
return;
#endif
@@ -398,14 +480,14 @@ abort_tasks(ErtsDrvEventState *state, int mode)
case ERL_DRV_WRITE:
ASSERT(state->type == ERTS_EV_TYPE_DRV_SEL);
abort_task(state->driver.select->outport,
- &state->driver.select->outtask,
+ &state->driver.select->outiotask.task,
state->type);
if (mode == ERL_DRV_WRITE)
break;
case ERL_DRV_READ:
ASSERT(state->type == ERTS_EV_TYPE_DRV_SEL);
abort_task(state->driver.select->inport,
- &state->driver.select->intask,
+ &state->driver.select->iniotask.task,
state->type);
break;
default:
@@ -443,16 +525,14 @@ deselect(ErtsDrvEventState *state, int mode)
if (!(state->events)) {
switch (state->type) {
case ERTS_EV_TYPE_DRV_SEL:
- ASSERT(!erts_port_task_is_scheduled(&state->driver.select->intask));
- ASSERT(!erts_port_task_is_scheduled(&state->driver.select->outtask));
- erts_free(ERTS_ALC_T_DRV_SEL_D_STATE,
- state->driver.select);
+ state->driver.select->inport = NIL;
+ state->driver.select->outport = NIL;
break;
#if ERTS_CIO_HAVE_DRV_EVENT
case ERTS_EV_TYPE_DRV_EV:
- ASSERT(!erts_port_task_is_scheduled(&state->driver.event->task));
- erts_free(ERTS_ALC_T_DRV_EV_D_STATE,
- state->driver.event);
+ state->driver.event->port = NIL;
+ state->driver.event->data = NULL;
+ state->driver.event->removed_events = (ErtsPollEvents) 0;
break;
#endif
case ERTS_EV_TYPE_NONE:
@@ -462,20 +542,297 @@ deselect(ErtsDrvEventState *state, int mode)
break;
}
- state->driver.select = NULL;
state->type = ERTS_EV_TYPE_NONE;
- state->flags = 0;
+ state->flags &= ~ERTS_EV_FLAG_USED;
remember_removed(state, &pollset);
}
}
-
#ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS
# define IS_FD_UNKNOWN(state) ((state)->type == ERTS_EV_TYPE_NONE && (state)->remove_cnt == 0)
#else
# define IS_FD_UNKNOWN(state) ((state) == NULL)
#endif
+static ERTS_INLINE void
+check_fd_cleanup(ErtsDrvEventState *state,
+#if ERTS_CIO_HAVE_DRV_EVENT
+ ErtsDrvEventDataState **free_event,
+#endif
+ ErtsDrvSelectDataState **free_select)
+{
+ erts_aint_t current_cio_time;
+
+ ERTS_SMP_LC_ASSERT(erts_smp_lc_mtx_is_locked(fd_mtx(state->fd)));
+
+ current_cio_time = erts_smp_atomic_read_acqb(&erts_check_io_time);
+ *free_select = NULL;
+ if (state->driver.select
+ && (state->type != ERTS_EV_TYPE_DRV_SEL)
+ && !is_iotask_active(&state->driver.select->iniotask, current_cio_time)
+ && !is_iotask_active(&state->driver.select->outiotask, current_cio_time)) {
+
+ *free_select = state->driver.select;
+ state->driver.select = NULL;
+ }
+
+#if ERTS_CIO_HAVE_DRV_EVENT
+ *free_event = NULL;
+ if (state->driver.event
+ && (state->type != ERTS_EV_TYPE_DRV_EV)
+ && !is_iotask_active(&state->driver.event->iotask, current_cio_time)) {
+
+ *free_event = state->driver.event;
+ state->driver.event = NULL;
+ }
+#endif
+
+#ifndef ERTS_SYS_CONTINOUS_FD_NUMBERS
+ if (((state->type != ERTS_EV_TYPE_NONE)
+ | state->remove_cnt
+#if ERTS_CIO_HAVE_DRV_EVENT
+ | (state->driver.event != NULL)
+#endif
+ | (state->driver.select != NULL)) == 0) {
+
+ hash_erase_drv_ev_state(state);
+
+ }
+#endif
+}
+
+static ERTS_INLINE int
+check_cleanup_active_fd(ErtsSysFdType fd,
+#if ERTS_CIO_DEFER_ACTIVE_EVENTS
+ ErtsPollControlEntry *pce,
+ int *pce_ix,
+#endif
+ erts_aint_t current_cio_time)
+{
+ ErtsDrvEventState *state;
+ int active = 0;
+ erts_smp_mtx_t *mtx = fd_mtx(fd);
+ void *free_select = NULL;
+#if ERTS_CIO_HAVE_DRV_EVENT
+ void *free_event = NULL;
+#endif
+#if ERTS_CIO_DEFER_ACTIVE_EVENTS
+ ErtsPollEvents evon = 0, evoff = 0;
+#endif
+
+ erts_smp_mtx_lock(mtx);
+
+#ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS
+ state = &drv_ev_state[(int) fd];
+#else
+ state = hash_get_drv_ev_state(fd); /* may be NULL! */
+ if (state)
+#endif
+ {
+ if (state->driver.select) {
+#if ERTS_CIO_DEFER_ACTIVE_EVENTS
+ if (is_iotask_active(&state->driver.select->iniotask, current_cio_time)) {
+ active = 1;
+ if ((state->events & ERTS_POLL_EV_IN)
+ && !(state->flags & ERTS_EV_FLAG_DEFER_IN_EV)) {
+ evoff |= ERTS_POLL_EV_IN;
+ state->flags |= ERTS_EV_FLAG_DEFER_IN_EV;
+ }
+ }
+ else if (state->flags & ERTS_EV_FLAG_DEFER_IN_EV) {
+ if (state->events & ERTS_POLL_EV_IN)
+ evon |= ERTS_POLL_EV_IN;
+ state->flags &= ~ERTS_EV_FLAG_DEFER_IN_EV;
+ }
+ if (is_iotask_active(&state->driver.select->outiotask, current_cio_time)) {
+ active = 1;
+ if ((state->events & ERTS_POLL_EV_OUT)
+ && !(state->flags & ERTS_EV_FLAG_DEFER_OUT_EV)) {
+ evoff |= ERTS_POLL_EV_OUT;
+ state->flags |= ERTS_EV_FLAG_DEFER_OUT_EV;
+ }
+ }
+ else if (state->flags & ERTS_EV_FLAG_DEFER_OUT_EV) {
+ if (state->events & ERTS_POLL_EV_OUT)
+ evon |= ERTS_POLL_EV_OUT;
+ state->flags &= ~ERTS_EV_FLAG_DEFER_OUT_EV;
+ }
+ if (active)
+ (void) 0;
+ else
+#else
+ if (is_iotask_active(&state->driver.select->iniotask, current_cio_time)
+ || is_iotask_active(&state->driver.select->outiotask, current_cio_time))
+ active = 1;
+ else
+#endif
+ if (state->type != ERTS_EV_TYPE_DRV_SEL) {
+ free_select = state->driver.select;
+ state->driver.select = NULL;
+ }
+ }
+
+#if ERTS_CIO_HAVE_DRV_EVENT
+ if (state->driver.event) {
+ if (is_iotask_active(&state->driver.event->iotask, current_cio_time)) {
+#if ERTS_CIO_DEFER_ACTIVE_EVENTS
+ ErtsPollEvents evs = state->events & ~state->driver.event->deferred_events;
+ if (evs) {
+ evoff |= evs;
+ state->driver.event->deferred_events |= evs;
+ }
+#endif
+ active = 1;
+ }
+ else if (state->type != ERTS_EV_TYPE_DRV_EV) {
+ free_event = state->driver.event;
+ state->driver.event = NULL;
+ }
+#if ERTS_CIO_DEFER_ACTIVE_EVENTS
+ else {
+ ErtsPollEvents evs = state->events & state->driver.event->deferred_events;
+ if (evs) {
+ evon |= evs;
+ state->driver.event->deferred_events = 0;
+ }
+ }
+#endif
+
+ }
+#endif
+
+#ifndef ERTS_SYS_CONTINOUS_FD_NUMBERS
+ if (((state->type != ERTS_EV_TYPE_NONE) | state->remove_cnt | active) == 0)
+ hash_erase_drv_ev_state(state);
+#endif
+
+ }
+
+ erts_smp_mtx_unlock(mtx);
+
+ if (free_select)
+ free_drv_select_data(free_select);
+#if ERTS_CIO_HAVE_DRV_EVENT
+ if (free_event)
+ free_drv_event_data(free_event);
+#endif
+
+#if ERTS_CIO_DEFER_ACTIVE_EVENTS
+ if (evoff) {
+ ErtsPollControlEntry *pcep = &pce[(*pce_ix)++];
+ pcep->fd = fd;
+ pcep->events = evoff;
+ pcep->on = 0;
+ }
+ if (evon) {
+ ErtsPollControlEntry *pcep = &pce[(*pce_ix)++];
+ pcep->fd = fd;
+ pcep->events = evon;
+ pcep->on = 1;
+ }
+#endif
+
+ return active;
+}
+
+static void
+check_cleanup_active_fds(erts_aint_t current_cio_time)
+{
+ int six = pollset.active_fd.six;
+ int eix = pollset.active_fd.eix;
+ erts_aint32_t no = erts_smp_atomic32_read_dirty(&pollset.active_fd.no);
+ int size = pollset.active_fd.size;
+ int ix = six;
+#if ERTS_CIO_DEFER_ACTIVE_EVENTS
+ /* every fd might add two entries */
+ Uint pce_sz = 2*sizeof(ErtsPollControlEntry)*no;
+ ErtsPollControlEntry *pctrl_entries = (pce_sz
+ ? erts_alloc(ERTS_ALC_T_TMP, pce_sz)
+ : NULL);
+ int pctrl_ix = 0;
+#endif
+
+ while (ix != eix) {
+ ErtsSysFdType fd = pollset.active_fd.array[ix];
+ int nix = ix + 1;
+ if (nix >= size)
+ nix = 0;
+ ASSERT(fd != ERTS_SYS_FD_INVALID);
+ if (!check_cleanup_active_fd(fd,
+#if ERTS_CIO_DEFER_ACTIVE_EVENTS
+ pctrl_entries,
+ &pctrl_ix,
+#endif
+ current_cio_time)) {
+ no--;
+ if (ix == six) {
+#ifdef DEBUG
+ pollset.active_fd.array[ix] = ERTS_SYS_FD_INVALID;
+#endif
+ six = nix;
+ }
+ else {
+ pollset.active_fd.array[ix] = pollset.active_fd.array[six];
+#ifdef DEBUG
+ pollset.active_fd.array[six] = ERTS_SYS_FD_INVALID;
+#endif
+ six++;
+ if (six >= size)
+ six = 0;
+ }
+ }
+ ix = nix;
+ }
+
+#if ERTS_CIO_DEFER_ACTIVE_EVENTS
+ ASSERT(pctrl_ix <= pce_sz/sizeof(ErtsPollControlEntry));
+ if (pctrl_ix)
+ ERTS_CIO_POLL_CTLV(pollset.ps, pctrl_entries, pctrl_ix);
+ if (pctrl_entries)
+ erts_free(ERTS_ALC_T_TMP, pctrl_entries);
+#endif
+
+ pollset.active_fd.six = six;
+ pollset.active_fd.eix = eix;
+ erts_smp_atomic32_set_relb(&pollset.active_fd.no, no);
+}
+
+static ERTS_INLINE void
+add_active_fd(ErtsSysFdType fd)
+{
+ int eix = pollset.active_fd.eix;
+ int size = pollset.active_fd.size;
+
+
+ pollset.active_fd.array[eix] = fd;
+
+ erts_smp_atomic32_set_relb(&pollset.active_fd.no,
+ (erts_smp_atomic32_read_dirty(&pollset.active_fd.no)
+ + 1));
+
+ eix++;
+ if (eix >= size)
+ eix = 0;
+ if (pollset.active_fd.six == eix) {
+ pollset.active_fd.six = 0;
+ eix = size;
+ size += ERTS_ACTIVE_FD_INC;
+ pollset.active_fd.array = erts_realloc(ERTS_ALC_T_ACTIVE_FD_ARR,
+ pollset.active_fd.array,
+ sizeof(ErtsSysFdType)*size);
+ pollset.active_fd.size = size;
+#ifdef DEBUG
+ {
+ int i;
+ for (i = eix + 1; i < size; i++)
+ pollset.active_fd.array[i] = ERTS_SYS_FD_INVALID;
+ }
+#endif
+
+ }
+
+ pollset.active_fd.eix = eix;
+}
int
ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix,
@@ -492,6 +849,10 @@ ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix,
ErtsDrvEventState *state;
int wake_poller;
int ret;
+#if ERTS_CIO_HAVE_DRV_EVENT
+ ErtsDrvEventDataState *free_event = NULL;
+#endif
+ ErtsDrvSelectDataState *free_select = NULL;
#ifdef USE_VM_PROBES
DTRACE_CHARBUF(name, 64);
#endif
@@ -593,9 +954,9 @@ ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix,
if (new_events & (ERTS_POLL_EV_ERR|ERTS_POLL_EV_NVAL)) {
if (state->type == ERTS_EV_TYPE_DRV_SEL && !state->events) {
state->type = ERTS_EV_TYPE_NONE;
- state->flags = 0;
- erts_free(ERTS_ALC_T_DRV_SEL_D_STATE, state->driver.select);
- state->driver.select = NULL;
+ state->flags &= ~ERTS_EV_FLAG_USED;
+ state->driver.select->inport = NIL;
+ state->driver.select->outport = NIL;
}
ret = -1;
goto done;
@@ -613,18 +974,10 @@ ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix,
state->events = new_events;
if (ctl_events) {
if (on) {
- if (state->type == ERTS_EV_TYPE_NONE) {
- ErtsDrvSelectDataState *dsdsp
- = erts_alloc(ERTS_ALC_T_DRV_SEL_D_STATE,
- sizeof(ErtsDrvSelectDataState));
- dsdsp->inport = NIL;
- dsdsp->outport = NIL;
- erts_port_task_handle_init(&dsdsp->intask);
- erts_port_task_handle_init(&dsdsp->outtask);
- ASSERT(state->driver.select == NULL);
- state->driver.select = dsdsp;
+ if (!state->driver.select)
+ state->driver.select = alloc_drv_select_data();
+ if (state->type == ERTS_EV_TYPE_NONE)
state->type = ERTS_EV_TYPE_DRV_SEL;
- }
ASSERT(state->type == ERTS_EV_TYPE_DRV_SEL);
if (ctl_events & ERTS_POLL_EV_IN)
state->driver.select->inport = id;
@@ -645,17 +998,12 @@ ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix,
state->driver.select->outport = NIL;
}
if (new_events == 0) {
- ASSERT(!erts_port_task_is_scheduled(&state->driver.select->intask));
- ASSERT(!erts_port_task_is_scheduled(&state->driver.select->outtask));
if (old_events != 0) {
remember_removed(state, &pollset);
}
if ((mode & ERL_DRV_USE) || !(state->flags & ERTS_EV_FLAG_USED)) {
state->type = ERTS_EV_TYPE_NONE;
- state->flags = 0;
- erts_free(ERTS_ALC_T_DRV_SEL_D_STATE,
- state->driver.select);
- state->driver.select = NULL;
+ state->flags &= ~ERTS_EV_FLAG_USED;
}
/*else keep it, as fd will probably be selected upon again */
}
@@ -686,13 +1034,15 @@ ERTS_CIO_EXPORT(driver_select)(ErlDrvPort ix,
ret = 0;
-done:;
-#ifndef ERTS_SYS_CONTINOUS_FD_NUMBERS
- if (state->type == ERTS_EV_TYPE_NONE && state->remove_cnt == 0) {
- hash_erase_drv_ev_state(state);
- }
+done:
+
+ check_fd_cleanup(state,
+#if ERTS_CIO_HAVE_DRV_EVENT
+ &free_event,
#endif
-done_unknown:
+ &free_select);
+
+done_unknown:
erts_smp_mtx_unlock(fd_mtx(fd));
if (stop_select_fn) {
int was_unmasked = erts_block_fpe();
@@ -700,6 +1050,12 @@ done_unknown:
(*stop_select_fn)(e, NULL);
erts_unblock_fpe(was_unmasked);
}
+ if (free_select)
+ free_drv_select_data(free_select);
+#if ERTS_CIO_HAVE_DRV_EVENT
+ if (free_event)
+ free_drv_event_data(free_event);
+#endif
return ret;
}
@@ -719,6 +1075,10 @@ ERTS_CIO_EXPORT(driver_event)(ErlDrvPort ix,
ErtsDrvEventState *state;
int do_wake = 0;
int ret;
+#if ERTS_CIO_HAVE_DRV_EVENT
+ ErtsDrvEventDataState *free_event;
+#endif
+ ErtsDrvSelectDataState *free_select;
Port *prt = erts_drvport2port(ix);
if (prt == ERTS_INVALID_ERL_DRV_PORT)
@@ -799,10 +1159,8 @@ ERTS_CIO_EXPORT(driver_event)(ErlDrvPort ix,
state->driver.event->removed_events |= remove_events;
}
else {
- state->driver.event
- = erts_alloc(ERTS_ALC_T_DRV_EV_D_STATE,
- sizeof(ErtsDrvEventDataState));
- erts_port_task_handle_init(&state->driver.event->task);
+ if (!state->driver.event)
+ state->driver.event = alloc_drv_event_data();
state->driver.event->port = id;
state->driver.event->removed_events = (ErtsPollEvents) 0;
state->type = ERTS_EV_TYPE_DRV_EV;
@@ -812,10 +1170,10 @@ ERTS_CIO_EXPORT(driver_event)(ErlDrvPort ix,
else {
if (state->type == ERTS_EV_TYPE_DRV_EV) {
abort_tasks(state, 0);
- erts_free(ERTS_ALC_T_DRV_EV_D_STATE,
- state->driver.event);
+ state->driver.event->port = NIL;
+ state->driver.event->data = NULL;
+ state->driver.event->removed_events = (ErtsPollEvents) 0;
}
- state->driver.select = NULL;
state->type = ERTS_EV_TYPE_NONE;
remember_removed(state, &pollset);
}
@@ -825,12 +1183,22 @@ ERTS_CIO_EXPORT(driver_event)(ErlDrvPort ix,
ret = 0;
done:
-#ifndef ERTS_SYS_CONTINOUS_FD_NUMBERS
- if (state->type == ERTS_EV_TYPE_NONE && state->remove_cnt == 0) {
- hash_erase_drv_ev_state(state);
- }
+
+ check_fd_cleanup(state,
+#if ERTS_CIO_HAVE_DRV_EVENT
+ &free_event,
#endif
+ &free_select);
+
erts_smp_mtx_unlock(fd_mtx(fd));
+
+ if (free_select)
+ free_drv_select_data(free_select);
+#if ERTS_CIO_HAVE_DRV_EVENT
+ if (free_event)
+ free_drv_event_data(free_event);
+#endif
+
return ret;
#endif
}
@@ -1027,7 +1395,7 @@ steal_pending_stop_select(erts_dsprintf_buf_t *dsbufp, ErlDrvPort ix,
* In either case stop_select should not be called.
*/
state->type = ERTS_EV_TYPE_NONE;
- state->flags = 0;
+ state->flags &= ~ERTS_EV_FLAG_USED;
if (state->driver.drv_ptr->handle) {
erts_ddll_dereference_driver(state->driver.drv_ptr->handle);
}
@@ -1099,38 +1467,103 @@ event_large_fd_error(ErlDrvPort ix, ErtsSysFdType fd, ErlDrvEventData event_data
#endif
#endif
+static ERTS_INLINE int
+io_task_schedule_allowed(ErtsDrvEventState *state,
+ ErtsPortTaskType type,
+ erts_aint_t current_cio_time)
+{
+ ErtsIoTask *io_task;
+
+ switch (type) {
+ case ERTS_PORT_TASK_INPUT:
+ if (!state->driver.select)
+ return 0;
+#if ERTS_CIO_HAVE_DRV_EVENT
+ if (state->driver.event)
+ return 0;
+#endif
+ io_task = &state->driver.select->iniotask;
+ break;
+ case ERTS_PORT_TASK_OUTPUT:
+ if (!state->driver.select)
+ return 0;
+#if ERTS_CIO_HAVE_DRV_EVENT
+ if (state->driver.event)
+ return 0;
+#endif
+ io_task = &state->driver.select->outiotask;
+ break;
+#if ERTS_CIO_HAVE_DRV_EVENT
+ case ERTS_PORT_TASK_EVENT:
+ if (!state->driver.event)
+ return 0;
+ if (state->driver.select)
+ return 0;
+ io_task = &state->driver.event->iotask;
+ break;
+#endif
+ default:
+ ERTS_INTERNAL_ERROR("Invalid I/O-task type");
+ return 0;
+ }
+
+ return !is_iotask_active(io_task, current_cio_time);
+}
+
static ERTS_INLINE void
-iready(Eterm id, ErtsDrvEventState *state)
+iready(Eterm id, ErtsDrvEventState *state, erts_aint_t current_cio_time)
{
- if (erts_port_task_schedule(id,
- &state->driver.select->intask,
- ERTS_PORT_TASK_INPUT,
- (ErlDrvEvent) state->fd) != 0) {
- stale_drv_select(id, state, ERL_DRV_READ);
+ if (io_task_schedule_allowed(state,
+ ERTS_PORT_TASK_INPUT,
+ current_cio_time)) {
+ ErtsIoTask *iotask = &state->driver.select->iniotask;
+ erts_smp_atomic_set_nob(&iotask->executed_time, current_cio_time);
+ if (erts_port_task_schedule(id,
+ &iotask->task,
+ ERTS_PORT_TASK_INPUT,
+ (ErlDrvEvent) state->fd) != 0) {
+ stale_drv_select(id, state, ERL_DRV_READ);
+ }
+ add_active_fd(state->fd);
}
}
static ERTS_INLINE void
-oready(Eterm id, ErtsDrvEventState *state)
+oready(Eterm id, ErtsDrvEventState *state, erts_aint_t current_cio_time)
{
- if (erts_port_task_schedule(id,
- &state->driver.select->outtask,
- ERTS_PORT_TASK_OUTPUT,
- (ErlDrvEvent) state->fd) != 0) {
- stale_drv_select(id, state, ERL_DRV_WRITE);
+ if (io_task_schedule_allowed(state,
+ ERTS_PORT_TASK_OUTPUT,
+ current_cio_time)) {
+ ErtsIoTask *iotask = &state->driver.select->outiotask;
+ erts_smp_atomic_set_nob(&iotask->executed_time, current_cio_time);
+ if (erts_port_task_schedule(id,
+ &iotask->task,
+ ERTS_PORT_TASK_OUTPUT,
+ (ErlDrvEvent) state->fd) != 0) {
+ stale_drv_select(id, state, ERL_DRV_WRITE);
+ }
+ add_active_fd(state->fd);
}
}
#if ERTS_CIO_HAVE_DRV_EVENT
static ERTS_INLINE void
-eready(Eterm id, ErtsDrvEventState *state, ErlDrvEventData event_data)
+eready(Eterm id, ErtsDrvEventState *state, ErlDrvEventData event_data,
+ erts_aint_t current_cio_time)
{
- if (erts_port_task_schedule(id,
- &state->driver.event->task,
- ERTS_PORT_TASK_EVENT,
- (ErlDrvEvent) state->fd,
- event_data) != 0) {
- stale_drv_select(id, state, 0);
+ if (io_task_schedule_allowed(state,
+ ERTS_PORT_TASK_EVENT,
+ current_cio_time)) {
+ ErtsIoTask *iotask = &state->driver.event->iotask;
+ erts_smp_atomic_set_nob(&iotask->executed_time, current_cio_time);
+ if (erts_port_task_schedule(id,
+ &iotask->task,
+ ERTS_PORT_TASK_EVENT,
+ (ErlDrvEvent) state->fd,
+ event_data) != 0) {
+ stale_drv_select(id, state, 0);
+ }
+ add_active_fd(state->fd);
}
}
#endif
@@ -1161,10 +1594,11 @@ ERTS_CIO_EXPORT(erts_check_io_interrupt_timed)(int set,
void
ERTS_CIO_EXPORT(erts_check_io)(int do_wait)
{
- ErtsPollResFd pollres[256];
+ ErtsPollResFd *pollres;
int pollres_len;
SysTimeval wait_time;
int poll_ret, i;
+ erts_aint_t current_cio_time;
restart:
@@ -1181,10 +1615,24 @@ ERTS_CIO_EXPORT(erts_check_io)(int do_wait)
wait_time.tv_usec = 0;
}
+ /*
+ * No need for an atomic inc op when incrementing
+ * erts_check_io_time, since only one thread can
+ * check io at a time.
+ */
+ current_cio_time = erts_smp_atomic_read_dirty(&erts_check_io_time);
+ current_cio_time++;
+ erts_smp_atomic_set_relb(&erts_check_io_time, current_cio_time);
+
+ check_cleanup_active_fds(current_cio_time);
+
#ifdef ERTS_ENABLE_LOCK_CHECK
erts_lc_check_exact(NULL, 0); /* No locks should be locked */
#endif
- pollres_len = sizeof(pollres)/sizeof(ErtsPollResFd);
+
+ pollres_len = erts_smp_atomic32_read_dirty(&pollset.active_fd.no) + ERTS_CHECK_IO_POLL_RES_LEN;
+
+ pollres = erts_alloc(ERTS_ALC_T_TMP, sizeof(ErtsPollResFd)*pollres_len);
erts_smp_atomic_set_nob(&pollset.in_poll_wait, 1);
@@ -1204,6 +1652,7 @@ ERTS_CIO_EXPORT(erts_check_io)(int do_wait)
if (poll_ret != 0) {
erts_smp_atomic_set_nob(&pollset.in_poll_wait, 0);
forget_removed(&pollset);
+ erts_free(ERTS_ALC_T_TMP, pollres);
if (poll_ret == EAGAIN) {
goto restart;
}
@@ -1263,15 +1712,15 @@ ERTS_CIO_EXPORT(erts_check_io)(int do_wait)
if ((revents & ERTS_POLL_EV_IN)
|| (!(revents & ERTS_POLL_EV_OUT)
&& state->events & ERTS_POLL_EV_IN)) {
- iready(state->driver.select->inport, state);
+ iready(state->driver.select->inport, state, current_cio_time);
}
else if (state->events & ERTS_POLL_EV_OUT) {
- oready(state->driver.select->outport, state);
+ oready(state->driver.select->outport, state, current_cio_time);
}
}
else if (revents & (ERTS_POLL_EV_IN|ERTS_POLL_EV_OUT)) {
if (revents & ERTS_POLL_EV_OUT) {
- oready(state->driver.select->outport, state);
+ oready(state->driver.select->outport, state, current_cio_time);
}
/* Someone might have deselected input since revents
was read (true also on the non-smp emulator since
@@ -1279,7 +1728,7 @@ ERTS_CIO_EXPORT(erts_check_io)(int do_wait)
revents... */
revents &= ~(~state->events & ERTS_POLL_EV_IN);
if (revents & ERTS_POLL_EV_IN) {
- iready(state->driver.select->inport, state);
+ iready(state->driver.select->inport, state, current_cio_time);
}
}
else if (revents & ERTS_POLL_EV_NVAL) {
@@ -1287,6 +1736,7 @@ ERTS_CIO_EXPORT(erts_check_io)(int do_wait)
state->driver.select->inport,
state->driver.select->outport,
state->events);
+ add_active_fd(state->fd);
}
break;
}
@@ -1304,8 +1754,7 @@ ERTS_CIO_EXPORT(erts_check_io)(int do_wait)
if (revents) {
event_data->events = state->events;
event_data->revents = revents;
-
- eready(state->driver.event->port, state, event_data);
+ eready(state->driver.event->port, state, event_data, current_cio_time);
}
break;
}
@@ -1323,6 +1772,7 @@ ERTS_CIO_EXPORT(erts_check_io)(int do_wait)
(int) state->type);
ASSERT(0);
deselect(state, 0);
+ add_active_fd(state->fd);
break;
}
}
@@ -1334,6 +1784,7 @@ ERTS_CIO_EXPORT(erts_check_io)(int do_wait)
}
erts_smp_atomic_set_nob(&pollset.in_poll_wait, 0);
+ erts_free(ERTS_ALC_T_TMP, pollres);
forget_removed(&pollset);
}
@@ -1469,10 +1920,27 @@ static void drv_ev_state_free(void *des)
void
ERTS_CIO_EXPORT(erts_init_check_io)(void)
{
+ erts_smp_atomic_init_nob(&erts_check_io_time, 0);
erts_smp_atomic_init_nob(&pollset.in_poll_wait, 0);
+
ERTS_CIO_POLL_INIT();
pollset.ps = ERTS_CIO_NEW_POLLSET();
+ pollset.active_fd.six = 0;
+ pollset.active_fd.eix = 0;
+ erts_smp_atomic32_init_nob(&pollset.active_fd.no, 0);
+ pollset.active_fd.size = ERTS_ACTIVE_FD_INC;
+ pollset.active_fd.array = erts_alloc(ERTS_ALC_T_ACTIVE_FD_ARR,
+ sizeof(ErtsSysFdType)*ERTS_ACTIVE_FD_INC);
+#ifdef DEBUG
+ {
+ int i;
+ for (i = 0; i < ERTS_ACTIVE_FD_INC; i++)
+ pollset.active_fd.array[i] = ERTS_SYS_FD_INVALID;
+ }
+#endif
+
+
#ifdef ERTS_SMP
init_removed_fd_alloc();
pollset.removed_list = NULL;
@@ -1548,12 +2016,27 @@ Eterm
ERTS_CIO_EXPORT(erts_check_io_info)(void *proc)
{
Process *p = (Process *) proc;
- Eterm tags[15], values[15], res;
+ Eterm tags[16], values[16], res;
Uint sz, *szp, *hp, **hpp, memory_size;
Sint i;
ErtsPollInfo pi;
-
- ERTS_CIO_POLL_INFO(pollset.ps, &pi);
+ erts_aint_t cio_time = erts_smp_atomic_read_acqb(&erts_check_io_time);
+ int active_fds = (int) erts_smp_atomic32_read_acqb(&pollset.active_fd.no);
+
+ while (1) {
+ erts_aint_t post_cio_time;
+ int post_active_fds;
+
+ ERTS_CIO_POLL_INFO(pollset.ps, &pi);
+
+ post_cio_time = erts_smp_atomic_read_mb(&erts_check_io_time);
+ post_active_fds = (int) erts_smp_atomic32_read_acqb(&pollset.active_fd.no);
+ if (cio_time == post_cio_time && active_fds == post_active_fds)
+ break;
+ cio_time = post_cio_time;
+ active_fds = post_active_fds;
+ }
+
memory_size = pi.memory_size;
#ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS
memory_size += sizeof(ErtsDrvEventState) * erts_smp_atomic_read_nob(&drv_ev_state_len);
@@ -1617,6 +2100,9 @@ ERTS_CIO_EXPORT(erts_check_io_info)(void *proc)
tags[i] = erts_bld_atom(hpp, szp, "max_fds");
values[i++] = erts_bld_uint(hpp, szp, (Uint) pi.max_fds);
+ tags[i] = erts_bld_atom(hpp, szp, "active_fds");
+ values[i++] = erts_bld_uint(hpp, szp, (Uint) active_fds);
+
#ifdef ERTS_POLL_COUNT_AVOIDED_WAKEUPS
tags[i] = erts_bld_atom(hpp, szp, "no_avoided_wakeups");
values[i++] = erts_bld_uint(hpp, szp, (Uint) pi.no_avoided_wakeups);
@@ -1671,6 +2157,8 @@ print_events(ErtsPollEvents ev)
typedef struct {
int used_fds;
int num_errors;
+ int no_driver_select_structs;
+ int no_driver_event_structs;
#ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS
int internal_fds;
ErtsPollEvents *epep;
@@ -1693,6 +2181,13 @@ static void doit_erts_check_io_debug(void *vstate, void *vcounters)
struct stat stat_buf;
#endif
+ if (state->driver.select)
+ counters->no_driver_select_structs++;
+#if ERTS_CIO_HAVE_DRV_EVENT
+ if (state->driver.event)
+ counters->no_driver_event_structs++;
+#endif
+
#ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS
if (state->events || ep_events) {
if (ep_events & ERTS_POLL_EV_NVAL) {
@@ -1831,6 +2326,7 @@ static void doit_erts_check_io_debug(void *vstate, void *vcounters)
}
}
}
+#if ERTS_CIO_HAVE_DRV_EVENT
else if (state->type == ERTS_EV_TYPE_DRV_EV) {
Eterm id;
erts_printf("driver_event ");
@@ -1866,6 +2362,7 @@ static void doit_erts_check_io_debug(void *vstate, void *vcounters)
erts_free_port_names(pnp);
}
}
+#endif
#ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS
else if (internal) {
erts_printf("internal ");
@@ -1905,7 +2402,7 @@ static void doit_erts_check_io_debug(void *vstate, void *vcounters)
}
int
-ERTS_CIO_EXPORT(erts_check_io_debug)(void)
+ERTS_CIO_EXPORT(erts_check_io_debug)(ErtsCheckIoDebugInfo *ciodip)
{
#ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS
int fd, len;
@@ -1915,6 +2412,10 @@ ERTS_CIO_EXPORT(erts_check_io_debug)(void)
ErtsDrvEventState null_des;
null_des.driver.select = NULL;
+#if ERTS_CIO_HAVE_DRV_EVENT
+ null_des.driver.event = NULL;
+#endif
+ null_des.driver.drv_ptr = NULL;
null_des.events = 0;
null_des.remove_cnt = 0;
null_des.type = ERTS_EV_TYPE_NONE;
@@ -1935,6 +2436,8 @@ ERTS_CIO_EXPORT(erts_check_io_debug)(void)
#endif
counters.used_fds = 0;
counters.num_errors = 0;
+ counters.no_driver_select_structs = 0;
+ counters.no_driver_event_structs = 0;
#ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS
len = erts_smp_atomic_read_nob(&drv_ev_state_len);
@@ -1951,8 +2454,16 @@ ERTS_CIO_EXPORT(erts_check_io_debug)(void)
erts_smp_thr_progress_unblock();
+ ciodip->no_used_fds = counters.used_fds;
+ ciodip->no_driver_select_structs = counters.no_driver_select_structs;
+ ciodip->no_driver_event_structs = counters.no_driver_event_structs;
+
erts_printf("\n");
erts_printf("used fds=%d\n", counters.used_fds);
+ erts_printf("Number of driver_select() structures=%d\n", counters.no_driver_select_structs);
+#if ERTS_CIO_HAVE_DRV_EVENT
+ erts_printf("Number of driver_event() structures=%d\n", counters.no_driver_event_structs);
+#endif
#ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS
erts_printf("internal fds=%d\n", counters.internal_fds);
#endif
@@ -1961,6 +2472,7 @@ ERTS_CIO_EXPORT(erts_check_io_debug)(void)
#ifdef ERTS_SYS_CONTINOUS_FD_NUMBERS
erts_free(ERTS_ALC_T_TMP, (void *) counters.epep);
#endif
+
return counters.num_errors;
}
diff --git a/erts/emulator/sys/common/erl_check_io.h b/erts/emulator/sys/common/erl_check_io.h
index edab7947ba..d01297d55c 100644
--- a/erts/emulator/sys/common/erl_check_io.h
+++ b/erts/emulator/sys/common/erl_check_io.h
@@ -26,6 +26,7 @@
#ifndef ERL_CHECK_IO_H__
#define ERL_CHECK_IO_H__
+#include "sys.h"
#include "erl_sys_driver.h"
#ifdef ERTS_ENABLE_KERNEL_POLL
@@ -52,8 +53,8 @@ void erts_check_io_kp(int);
void erts_check_io_nkp(int);
void erts_init_check_io_kp(void);
void erts_init_check_io_nkp(void);
-int erts_check_io_debug_kp(void);
-int erts_check_io_debug_nkp(void);
+int erts_check_io_debug_kp(ErtsCheckIoDebugInfo *);
+int erts_check_io_debug_nkp(ErtsCheckIoDebugInfo *);
#else /* !ERTS_ENABLE_KERNEL_POLL */
@@ -70,6 +71,27 @@ void erts_init_check_io(void);
#endif
+extern erts_smp_atomic_t erts_check_io_time;
+
+typedef struct {
+ ErtsPortTaskHandle task;
+ erts_smp_atomic_t executed_time;
+} ErtsIoTask;
+
+ERTS_GLB_INLINE void erts_io_notify_port_task_executed(ErtsPortTaskHandle *pthp);
+
+#if ERTS_GLB_INLINE_INCL_FUNC_DEF
+
+ERTS_GLB_INLINE void
+erts_io_notify_port_task_executed(ErtsPortTaskHandle *pthp)
+{
+ ErtsIoTask *itp = (ErtsIoTask *) (((char *) pthp) - offsetof(ErtsIoTask, task));
+ erts_aint_t ci_time = erts_smp_atomic_read_acqb(&erts_check_io_time);
+ erts_smp_atomic_set_relb(&itp->executed_time, ci_time);
+}
+
+#endif
+
#endif /* ERL_CHECK_IO_H__ */
#if !defined(ERL_CHECK_IO_C__) && !defined(ERTS_ALLOC_C__)
@@ -81,6 +103,16 @@ void erts_init_check_io(void);
#include "erl_poll.h"
#include "erl_port_task.h"
+#ifdef __WIN32__
+/*
+ * Current erts_poll implementation for Windows cannot handle
+ * active events in the set of events polled.
+ */
+# define ERTS_CIO_DEFER_ACTIVE_EVENTS 1
+#else
+# define ERTS_CIO_DEFER_ACTIVE_EVENTS 0
+#endif
+
/*
* ErtsDrvEventDataState is used by driver_event() which is almost never
* used. We allocate ErtsDrvEventDataState separate since we dont wan't
@@ -91,13 +123,16 @@ typedef struct {
Eterm port;
ErlDrvEventData data;
ErtsPollEvents removed_events;
- ErtsPortTaskHandle task;
+#if ERTS_CIO_DEFER_ACTIVE_EVENTS
+ ErtsPollEvents deferred_events;
+#endif
+ ErtsIoTask iotask;
} ErtsDrvEventDataState;
typedef struct {
Eterm inport;
Eterm outport;
- ErtsPortTaskHandle intask;
- ErtsPortTaskHandle outtask;
+ ErtsIoTask iniotask;
+ ErtsIoTask outiotask;
} ErtsDrvSelectDataState;
#endif /* #ifndef ERL_CHECK_IO_INTERNAL__ */
diff --git a/erts/emulator/sys/common/erl_sys_common_misc.c b/erts/emulator/sys/common/erl_sys_common_misc.c
index e3ba741058..e63f0bda54 100644
--- a/erts/emulator/sys/common/erl_sys_common_misc.c
+++ b/erts/emulator/sys/common/erl_sys_common_misc.c
@@ -44,6 +44,14 @@
#endif
#endif
+/*
+ * erts_check_io_time is used by the erl_check_io implementation. The
+ * global erts_check_io_time variable is declared here since there
+ * (often) exist two versions of erl_check_io (kernel-poll and
+ * non-kernel-poll), and we dont want two versions of this variable.
+ */
+erts_smp_atomic_t erts_check_io_time;
+
/* Written once and only once */
static int filename_encoding = ERL_FILENAME_UNKNOWN;
diff --git a/erts/emulator/sys/unix/erl_unix_sys.h b/erts/emulator/sys/unix/erl_unix_sys.h
index 176fc049a7..b2fe34c1ac 100644
--- a/erts/emulator/sys/unix/erl_unix_sys.h
+++ b/erts/emulator/sys/unix/erl_unix_sys.h
@@ -135,9 +135,6 @@
/* File descriptors are numbers anc consecutively allocated on Unix */
#define ERTS_SYS_CONTINOUS_FD_NUMBERS
-#define HAVE_ERTS_CHECK_IO_DEBUG
-int erts_check_io_debug(void);
-
#ifndef ERTS_SMP
# undef ERTS_POLL_NEED_ASYNC_INTERRUPT_SUPPORT
# define ERTS_POLL_NEED_ASYNC_INTERRUPT_SUPPORT
diff --git a/erts/emulator/sys/unix/sys.c b/erts/emulator/sys/unix/sys.c
index c3d7440409..0d677d5f34 100644
--- a/erts/emulator/sys/unix/sys.c
+++ b/erts/emulator/sys/unix/sys.c
@@ -284,7 +284,7 @@ struct {
void (*check_io)(int);
Uint (*size)(void);
Eterm (*info)(void *);
- int (*check_io_debug)(void);
+ int (*check_io_debug)(ErtsCheckIoDebugInfo *);
} io_func = {0};
@@ -306,9 +306,9 @@ Eterm erts_check_io_info(void *p)
}
int
-erts_check_io_debug(void)
+erts_check_io_debug(ErtsCheckIoDebugInfo *ip)
{
- return (*io_func.check_io_debug)();
+ return (*io_func.check_io_debug)(ip);
}
diff --git a/erts/emulator/sys/win32/erl_poll.c b/erts/emulator/sys/win32/erl_poll.c
index 7a1d129cd5..972170d465 100644
--- a/erts/emulator/sys/win32/erl_poll.c
+++ b/erts/emulator/sys/win32/erl_poll.c
@@ -1085,7 +1085,7 @@ void erts_poll_controlv(ErtsPollSet ps,
pcev[i].events,
pcev[i].on);
}
- ERTS_POLLSET_LOCK(ps);
+ ERTS_POLLSET_UNLOCK(ps);
HARDTRACEF(("Out erts_poll_controlv"));
}
diff --git a/erts/emulator/sys/win32/erl_win_sys.h b/erts/emulator/sys/win32/erl_win_sys.h
index a78dbf64af..838f0c61eb 100644
--- a/erts/emulator/sys/win32/erl_win_sys.h
+++ b/erts/emulator/sys/win32/erl_win_sys.h
@@ -113,12 +113,10 @@
/*
* Our own type of "FD's"
*/
+#define ERTS_SYS_FD_INVALID INVALID_HANDLE_VALUE
#define ERTS_SYS_FD_TYPE HANDLE
#define NO_FSTAT_ON_SYS_FD_TYPE 1 /* They are events, not files */
-#define HAVE_ERTS_CHECK_IO_DEBUG
-int erts_check_io_debug(void);
-
/*
* For erl_time_sup
*/
diff --git a/erts/emulator/test/a_SUITE.erl b/erts/emulator/test/a_SUITE.erl
index 195c9c0a5f..17579be416 100644
--- a/erts/emulator/test/a_SUITE.erl
+++ b/erts/emulator/test/a_SUITE.erl
@@ -97,23 +97,13 @@ display_check_io(ChkIo) ->
catch erlang:display('--- CHECK IO INFO ---'),
catch erlang:display(ChkIo),
catch erts_debug:set_internal_state(available_internal_state, true),
- NoOfErrorFds = (catch erts_debug:get_internal_state(check_io_debug)),
+ NoOfErrorFds = (catch element(1, erts_debug:get_internal_state(check_io_debug))),
catch erlang:display({'NoOfErrorFds', NoOfErrorFds}),
catch erts_debug:set_internal_state(available_internal_state, false),
catch erlang:display('--- CHECK IO INFO ---'),
ok.
get_check_io_info() ->
- ChkIo = erlang:system_info(check_io),
- case lists:keysearch(pending_updates, 1, ChkIo) of
- {value, {pending_updates, 0}} ->
- display_check_io(ChkIo),
- ChkIo;
- false ->
- ChkIo;
- _ ->
- receive after 10 -> ok end,
- get_check_io_info()
- end.
+ z_SUITE:get_check_io_info().
diff --git a/erts/emulator/test/driver_SUITE.erl b/erts/emulator/test/driver_SUITE.erl
index 336b6188f6..8d2c620be0 100644
--- a/erts/emulator/test/driver_SUITE.erl
+++ b/erts/emulator/test/driver_SUITE.erl
@@ -31,8 +31,9 @@
end_per_suite/1, init_per_group/2,end_per_group/2,
init_per_testcase/2,
end_per_testcase/2,
+
+ a_test/1,
outputv_echo/1,
-
timer_measure/1,
timer_cancel/1,
timer_change/1,
@@ -79,7 +80,8 @@
thr_free_drv/1,
async_blast/1,
thr_msg_blast/1,
- consume_timeslice/1]).
+ consume_timeslice/1,
+ z_test/1]).
-export([bin_prefix/2]).
@@ -122,19 +124,19 @@ init_per_testcase(Case, Config) when is_atom(Case), is_list(Config) ->
_ -> erts_debug:set_internal_state(available_internal_state, true)
end,
erlang:display({init_per_testcase, Case}),
- ?line 0 = erts_debug:get_internal_state(check_io_debug),
+ ?line 0 = element(1, erts_debug:get_internal_state(check_io_debug)),
[{watchdog, Dog},{testcase, Case}|Config].
end_per_testcase(Case, Config) ->
Dog = ?config(watchdog, Config),
erlang:display({end_per_testcase, Case}),
- ?line 0 = erts_debug:get_internal_state(check_io_debug),
+ ?line 0 = element(1, erts_debug:get_internal_state(check_io_debug)),
?t:timetrap_cancel(Dog).
suite() -> [{ct_hooks,[ts_install_cth]}].
-all() ->
- [outputv_errors, outputv_echo, queue_echo, {group, timer},
+all() -> %% Keep a_test first and z_test last...
+ [a_test, outputv_errors, outputv_echo, queue_echo, {group, timer},
driver_unloaded, io_ready_exit, use_fallback_pollset,
bad_fd_in_pollset, driver_event, fd_change,
steal_control, otp_6602, driver_system_info_base_ver,
@@ -151,7 +153,8 @@ all() ->
thr_free_drv,
async_blast,
thr_msg_blast,
- consume_timeslice].
+ consume_timeslice,
+ z_test].
groups() ->
[{timer, [],
@@ -917,8 +920,7 @@ steal_control_test(Hndl = {erts_poll_info, Before}) ->
end.
chkio_test_init(Config) when is_list(Config) ->
- ?line wait_until_no_pending_updates(),
- ?line ChkIo = erlang:system_info(check_io),
+ ?line ChkIo = get_stable_check_io_info(),
?line case catch lists:keysearch(name, 1, ChkIo) of
{value, {name, erts_poll}} ->
?line ?t:format("Before test: ~p~n", [ChkIo]),
@@ -937,8 +939,7 @@ chkio_test_fini({skipped, _} = Res) ->
chkio_test_fini({chkio_test_result, Res, Before}) ->
?line ok = erl_ddll:unload_driver('chkio_drv'),
?line ok = erl_ddll:stop(),
- ?line wait_until_no_pending_updates(),
- ?line After = erlang:system_info(check_io),
+ ?line After = get_stable_check_io_info(),
?line ?t:format("After test: ~p~n", [After]),
?line verify_chkio_state(Before, After),
?line Res.
@@ -985,7 +986,7 @@ chkio_test({erts_poll_info, Before},
?line Fun(),
?line During = erlang:system_info(check_io),
?line erlang:display(During),
- ?line 0 = erts_debug:get_internal_state(check_io_debug),
+ ?line 0 = element(1, erts_debug:get_internal_state(check_io_debug)),
?line ?t:format("During test: ~p~n", [During]),
?line chk_chkio_port(Port),
?line case erlang:port_control(Port, ?CHKIO_STOP, "") of
@@ -1034,18 +1035,22 @@ verify_chkio_state(Before, After) ->
After)
end,
?line ok.
-
-
-wait_until_no_pending_updates() ->
- case lists:keysearch(pending_updates, 1, erlang:system_info(check_io)) of
- {value, {pending_updates, 0}} ->
- ok;
- false ->
- ok;
+get_stable_check_io_info() ->
+ ChkIo = erlang:system_info(check_io),
+ PendUpdNo = case lists:keysearch(pending_updates, 1, ChkIo) of
+ {value, {pending_updates, PendNo}} ->
+ PendNo;
+ false ->
+ 0
+ end,
+ {value, {active_fds, ActFds}} = lists:keysearch(active_fds, 1, ChkIo),
+ case {PendUpdNo, ActFds} of
+ {0, 0} ->
+ ChkIo;
_ ->
receive after 10 -> ok end,
- wait_until_no_pending_updates()
+ get_stable_check_io_info()
end.
otp_6602(doc) -> ["Missed port lock when stealing control of fd from a "
@@ -2387,10 +2392,25 @@ count_proc_sched(Ps, PNs) ->
PNs
end.
+a_test(Config) when is_list(Config) ->
+ check_io_debug().
+
+z_test(Config) when is_list(Config) ->
+ check_io_debug().
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Utilities
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+check_io_debug() ->
+ get_stable_check_io_info(),
+ {NoErrorFds, NoUsedFds, NoDrvSelStructs, NoDrvEvStructs}
+ = erts_debug:get_internal_state(check_io_debug),
+ 0 = NoErrorFds,
+ NoUsedFds = NoDrvSelStructs,
+ 0 = NoDrvEvStructs,
+ ok.
+
%flush_msgs() ->
% receive
% M ->
diff --git a/erts/emulator/test/port_SUITE.erl b/erts/emulator/test/port_SUITE.erl
index e01b2f253b..8792f25d76 100644
--- a/erts/emulator/test/port_SUITE.erl
+++ b/erts/emulator/test/port_SUITE.erl
@@ -90,6 +90,7 @@
mix_up_ports/1, otp_5112/1, otp_5119/1, otp_6224/1,
exit_status_multi_scheduling_block/1, ports/1,
spawn_driver/1, spawn_executable/1, close_deaf_port/1,
+ port_setget_data/1,
unregister_name/1, parallelism_option/1]).
-export([do_iter_max_ports/2]).
@@ -115,6 +116,7 @@ all() ->
mix_up_ports, otp_5112, otp_5119,
exit_status_multi_scheduling_block, ports, spawn_driver,
spawn_executable, close_deaf_port, unregister_name,
+ port_setget_data,
parallelism_option].
groups() ->
@@ -2333,6 +2335,55 @@ close_deaf_port_1(N, Cmd) ->
{comment, "Could not spawn more than " ++ integer_to_list(N) ++ " OS processes."}
end.
+%% Test undocumented port_set_data/2 and port_get_data/1
+%% Hammer from multiple processes a while
+%% and then abrubtly close the port (OTP-12208).
+port_setget_data(Config) when is_list(Config) ->
+ ok = load_driver(?config(data_dir, Config), "echo_drv"),
+ Port = erlang:open_port({spawn_driver, "echo_drv"}, []),
+
+ NSched = erlang:system_info(schedulers_online),
+ PRs = lists:map(fun(I) ->
+ spawn_opt(fun() -> port_setget_data_hammer(Port,1) end,
+ [monitor, {scheduler, I rem NSched}])
+ end,
+ lists:seq(1,10)),
+ receive after 100 -> ok end,
+ Papa = self(),
+ lists:foreach(fun({Pid,_}) -> Pid ! {Papa,prepare_for_close} end, PRs),
+ lists:foreach(fun({Pid,_}) ->
+ receive {Pid,prepare_for_close} -> ok end
+ end,
+ PRs),
+ port_close(Port),
+ lists:foreach(fun({Pid,Ref}) ->
+ receive {'DOWN', Ref, process, Pid, normal} -> ok end
+ end,
+ PRs),
+ ok.
+
+port_setget_data_hammer(Port, N) ->
+ Rand = random:uniform(3),
+ try case Rand of
+ 1 -> true = erlang:port_set_data(Port, atom);
+ 2 -> true = erlang:port_set_data(Port, {1,2,3});
+ 3 -> erlang:port_get_data(Port)
+ end
+ catch
+ error:badarg ->
+ true = get(prepare_for_close),
+ io:format("~p did ~p rounds before port closed\n", [self(), N]),
+ exit(normal)
+ end,
+ receive {Papa, prepare_for_close} ->
+ put(prepare_for_close, true),
+ Papa ! {self(),prepare_for_close}
+ after 0 ->
+ ok
+ end,
+ port_setget_data_hammer(Port, N+1).
+
+
wait_until(Fun) ->
case catch Fun() of
true ->
diff --git a/erts/emulator/test/z_SUITE.erl b/erts/emulator/test/z_SUITE.erl
index 4b3075a164..b0c6224dfe 100644
--- a/erts/emulator/test/z_SUITE.erl
+++ b/erts/emulator/test/z_SUITE.erl
@@ -38,7 +38,7 @@
-export([schedulers_alive/1, node_container_refc_check/1,
long_timers/1, pollset_size/1,
- check_io_debug/1]).
+ check_io_debug/1, get_check_io_info/0]).
-define(DEFAULT_TIMEOUT, ?t:minutes(5)).
@@ -288,11 +288,14 @@ check_io_debug(Config) when is_list(Config) ->
end.
check_io_debug_test() ->
+ ?line erlang:display(get_check_io_info()),
?line erts_debug:set_internal_state(available_internal_state, true),
- ?line erlang:display(erlang:system_info(check_io)),
- ?line NoOfErrorFds = erts_debug:get_internal_state(check_io_debug),
+ ?line {NoErrorFds, NoUsedFds, NoDrvSelStructs, NoDrvEvStructs}
+ = erts_debug:get_internal_state(check_io_debug),
?line erts_debug:set_internal_state(available_internal_state, false),
- ?line 0 = NoOfErrorFds,
+ ?line 0 = NoErrorFds,
+ ?line NoUsedFds = NoDrvSelStructs,
+ ?line 0 = NoDrvEvStructs,
?line ok.
@@ -305,7 +308,7 @@ display_check_io(ChkIo) ->
catch erlang:display('--- CHECK IO INFO ---'),
catch erlang:display(ChkIo),
catch erts_debug:set_internal_state(available_internal_state, true),
- NoOfErrorFds = (catch erts_debug:get_internal_state(check_io_debug)),
+ NoOfErrorFds = (catch element(1, erts_debug:get_internal_state(check_io_debug))),
catch erlang:display({'NoOfErrorFds', NoOfErrorFds}),
catch erts_debug:set_internal_state(available_internal_state, false),
catch erlang:display('--- CHECK IO INFO ---'),
@@ -313,14 +316,19 @@ display_check_io(ChkIo) ->
get_check_io_info() ->
ChkIo = erlang:system_info(check_io),
- case lists:keysearch(pending_updates, 1, ChkIo) of
- {value, {pending_updates, 0}} ->
+ PendUpdNo = case lists:keysearch(pending_updates, 1, ChkIo) of
+ {value, {pending_updates, PendNo}} ->
+ PendNo;
+ false ->
+ 0
+ end,
+ {value, {active_fds, ActFds}} = lists:keysearch(active_fds, 1, ChkIo),
+ case {PendUpdNo, ActFds} of
+ {0, 0} ->
display_check_io(ChkIo),
ChkIo;
- false ->
- ChkIo;
_ ->
- receive after 10 -> ok end,
+ receive after 100 -> ok end,
get_check_io_info()
end.
diff --git a/erts/etc/common/erlexec.c b/erts/etc/common/erlexec.c
index 709c6f02d1..5ebde8ca3c 100644
--- a/erts/etc/common/erlexec.c
+++ b/erts/etc/common/erlexec.c
@@ -128,6 +128,7 @@ static char *pluss_val_switches[] = {
"bwt",
"cl",
"ct",
+ "ecio",
"fwi",
"tbt",
"wct",
diff --git a/erts/preloaded/ebin/erlang.beam b/erts/preloaded/ebin/erlang.beam
index c5cf4b459f..1893dd84ac 100644
--- a/erts/preloaded/ebin/erlang.beam
+++ b/erts/preloaded/ebin/erlang.beam
Binary files differ
diff --git a/erts/preloaded/src/erlang.erl b/erts/preloaded/src/erlang.erl
index 98d7a942a6..4ba623d921 100644
--- a/erts/preloaded/src/erlang.erl
+++ b/erts/preloaded/src/erlang.erl
@@ -2233,6 +2233,7 @@ tuple_to_list(_Tuple) ->
(dynamic_trace) -> none | dtrace | systemtap;
(dynamic_trace_probes) -> boolean();
(elib_malloc) -> false;
+ (eager_check_io) -> boolean();
(ets_limit) -> pos_integer();
(fullsweep_after) -> {fullsweep_after, non_neg_integer()};
(garbage_collection) -> [{atom(), integer()}];
diff --git a/lib/asn1/c_src/asn1_erl_nif.c b/lib/asn1/c_src/asn1_erl_nif.c
index 53e3aa1678..317a464060 100644
--- a/lib/asn1/c_src/asn1_erl_nif.c
+++ b/lib/asn1/c_src/asn1_erl_nif.c
@@ -949,7 +949,7 @@ static int ber_decode_value(ErlNifEnv* env, ERL_NIF_TERM *value, unsigned char *
} else if (in_buf[*ib_index] == ASN1_INDEFINITE_LENGTH) {
(*ib_index)++;
curr_head = enif_make_list(env, 0);
- if (*ib_index+1 >= in_buf_len) {
+ if (*ib_index+1 >= in_buf_len || form == ASN1_PRIMITIVE) {
return ASN1_INDEF_LEN_ERROR;
}
while (!(in_buf[*ib_index] == 0 && in_buf[*ib_index + 1] == 0)) {
diff --git a/lib/asn1/test/asn1_SUITE_data/Constructed.asn b/lib/asn1/test/asn1_SUITE_data/Constructed.asn
index 09a66d0c0d..bd49741726 100644
--- a/lib/asn1/test/asn1_SUITE_data/Constructed.asn
+++ b/lib/asn1/test/asn1_SUITE_data/Constructed.asn
@@ -1,6 +1,3 @@
-
-
-
Constructed DEFINITIONS ::=
BEGIN
@@ -20,4 +17,7 @@ C ::= CHOICE {
S3 ::= SEQUENCE {i INTEGER}
S3ext ::= SEQUENCE {i INTEGER, ...}
+
+OS ::= OCTET STRING
+
END
diff --git a/lib/asn1/test/ber_decode_error.erl b/lib/asn1/test/ber_decode_error.erl
index 6fd2450c62..ef11717c45 100644
--- a/lib/asn1/test/ber_decode_error.erl
+++ b/lib/asn1/test/ber_decode_error.erl
@@ -61,6 +61,10 @@ run([]) ->
(catch 'Constructed':decode('S', sub(<<40,16#80,1,1,255,0,0>>, 6))),
{error,{asn1,{invalid_length,_}}} =
(catch 'Constructed':decode('S', sub(<<40,16#80,1,1,255,0,0>>, 5))),
+
+ %% A primitive must not be encoded with an indefinite length.
+ {error,{asn1,{invalid_length,_}}} =
+ (catch 'Constructed':decode('OS', <<4,128,4,3,97,98,99,0,0>>)),
ok.
sub(Bin, Bytes) ->
diff --git a/lib/common_test/src/ct_logs.erl b/lib/common_test/src/ct_logs.erl
index 43eabb18d5..7037cdca73 100644
--- a/lib/common_test/src/ct_logs.erl
+++ b/lib/common_test/src/ct_logs.erl
@@ -129,7 +129,13 @@ datestr_from_dirname([]) ->
close(Info, StartDir) ->
%% close executes on the ct_util process, not on the logger process
%% so we need to use a local copy of the log cache data
- LogCacheBin = make_last_run_index(),
+ LogCacheBin =
+ case make_last_run_index() of
+ {error,_} -> % log server not responding
+ undefined;
+ LCB ->
+ LCB
+ end,
put(ct_log_cache,LogCacheBin),
Cache2File = fun() ->
case get(ct_log_cache) of
@@ -710,6 +716,7 @@ logger_loop(State) ->
end
end,
if Importance >= (100-VLvl) ->
+ CtLogFd = State#logger_state.ct_log_fd,
case get_groupleader(Pid, GL, State) of
{tc_log,TCGL,TCGLs} ->
case erlang:is_process_alive(TCGL) of
@@ -723,14 +730,15 @@ logger_loop(State) ->
%% Group leader is dead, so write to the
%% CtLog or unexpected_io log instead
unexpected_io(Pid,Category,Importance,
- List,State),
+ List,CtLogFd),
+
logger_loop(State)
end;
{ct_log,_Fd,TCGLs} ->
%% If category is ct_internal then write
%% to ct_log, else write to unexpected_io
%% log
- unexpected_io(Pid,Category,Importance,List,State),
+ unexpected_io(Pid,Category,Importance,List,CtLogFd),
logger_loop(State#logger_state{
tc_groupleaders = TCGLs})
end;
@@ -803,16 +811,15 @@ logger_loop(State) ->
ok
end.
-create_io_fun(FromPid, State) ->
+create_io_fun(FromPid, CtLogFd) ->
%% we have to build one io-list of all strings
%% before printing, or other io printouts (made in
%% parallel) may get printed between this header
%% and footer
- Fd = State#logger_state.ct_log_fd,
fun({Str,Args}, IoList) ->
case catch io_lib:format(Str,Args) of
{'EXIT',_Reason} ->
- io:format(Fd, "Logging fails! Str: ~p, Args: ~p~n",
+ io:format(CtLogFd, "Logging fails! Str: ~p, Args: ~p~n",
[Str,Args]),
%% stop the testcase, we need to see the fault
exit(FromPid, {log_printout_error,Str,Args}),
@@ -827,28 +834,53 @@ create_io_fun(FromPid, State) ->
print_to_log(sync, FromPid, Category, TCGL, List, State) ->
%% in some situations (exceptions), the printout is made from the
%% test server IO process and there's no valid group leader to send to
+ CtLogFd = State#logger_state.ct_log_fd,
if FromPid /= TCGL ->
- IoFun = create_io_fun(FromPid, State),
+ IoFun = create_io_fun(FromPid, CtLogFd),
io:format(TCGL,"~ts", [lists:foldl(IoFun, [], List)]);
true ->
- unexpected_io(FromPid,Category,?MAX_IMPORTANCE,List,State)
+ unexpected_io(FromPid,Category,?MAX_IMPORTANCE,List,CtLogFd)
end,
State;
print_to_log(async, FromPid, Category, TCGL, List, State) ->
%% in some situations (exceptions), the printout is made from the
%% test server IO process and there's no valid group leader to send to
+ CtLogFd = State#logger_state.ct_log_fd,
Printer =
if FromPid /= TCGL ->
- IoFun = create_io_fun(FromPid, State),
+ IoFun = create_io_fun(FromPid, CtLogFd),
fun() ->
test_server:permit_io(TCGL, self()),
- io:format(TCGL, "~ts", [lists:foldl(IoFun, [], List)])
+
+ %% Since asynchronous io gets can get buffered if
+ %% the file system is slow, there is also a risk that
+ %% the group leader has terminated before we get to
+ %% the io:format(GL, ...) call. We check this and
+ %% print "expired" messages to the unexpected io
+ %% log instead (best we can do).
+
+ case erlang:is_process_alive(TCGL) of
+ true ->
+ try io:format(TCGL, "~ts",
+ [lists:foldl(IoFun,[],List)]) of
+ _ -> ok
+ catch
+ _:terminated ->
+ unexpected_io(FromPid, Category,
+ ?MAX_IMPORTANCE,
+ List, CtLogFd)
+ end;
+ false ->
+ unexpected_io(FromPid, Category,
+ ?MAX_IMPORTANCE,
+ List, CtLogFd)
+ end
end;
true ->
fun() ->
- unexpected_io(FromPid,Category,?MAX_IMPORTANCE,
- List,State)
+ unexpected_io(FromPid, Category, ?MAX_IMPORTANCE,
+ List, CtLogFd)
end
end,
case State#logger_state.async_print_jobs of
@@ -3149,12 +3181,11 @@ html_encoding(latin1) ->
html_encoding(utf8) ->
"utf-8".
-unexpected_io(Pid,ct_internal,_Importance,List,State) ->
- IoFun = create_io_fun(Pid,State),
- io:format(State#logger_state.ct_log_fd, "~ts",
- [lists:foldl(IoFun, [], List)]);
-unexpected_io(Pid,_Category,_Importance,List,State) ->
- IoFun = create_io_fun(Pid,State),
+unexpected_io(Pid,ct_internal,_Importance,List,CtLogFd) ->
+ IoFun = create_io_fun(Pid,CtLogFd),
+ io:format(CtLogFd, "~ts", [lists:foldl(IoFun, [], List)]);
+unexpected_io(Pid,_Category,_Importance,List,CtLogFd) ->
+ IoFun = create_io_fun(Pid,CtLogFd),
Data = io_lib:format("~ts", [lists:foldl(IoFun, [], List)]),
test_server_io:print_unexpected(Data),
ok.
diff --git a/lib/debugger/src/dbg_icmd.erl b/lib/debugger/src/dbg_icmd.erl
index b1bf4ebecc..ce12c1beb3 100644
--- a/lib/debugger/src/dbg_icmd.erl
+++ b/lib/debugger/src/dbg_icmd.erl
@@ -1,7 +1,7 @@
%%
%% %CopyrightBegin%
%%
-%% Copyright Ericsson AB 1998-2013. All Rights Reserved.
+%% Copyright Ericsson AB 1998-2014. All Rights Reserved.
%%
%% The contents of this file are subject to the Erlang Public License,
%% Version 1.1, (the "License"); you may not use this file except in
@@ -49,10 +49,6 @@
%% specifies if the process should break.
%%--------------------------------------------------------------------
-%% Common Test adaptation
-cmd({call_remote,0,ct_line,line,_As}, Bs, _Ieval) ->
- Bs;
-
cmd(Expr, Bs, Ieval) ->
cmd(Expr, Bs, get(next_break), Ieval).
diff --git a/lib/debugger/src/dbg_ieval.erl b/lib/debugger/src/dbg_ieval.erl
index 77297de0f3..96f9f91808 100644
--- a/lib/debugger/src/dbg_ieval.erl
+++ b/lib/debugger/src/dbg_ieval.erl
@@ -1,7 +1,7 @@
%%
%% %CopyrightBegin%
%%
-%% Copyright Ericsson AB 1998-2013. All Rights Reserved.
+%% Copyright Ericsson AB 1998-2014. All Rights Reserved.
%%
%% The contents of this file are subject to the Erlang Public License,
%% Version 1.1, (the "License"); you may not use this file except in
@@ -457,11 +457,6 @@ do_eval_function(Mod, Fun, As0, Bs0, _, Ieval0) when is_function(Fun);
exception(error, Reason, Bs0, Ieval0)
end;
-%% Common Test adaptation
-do_eval_function(ct_line, line, As, Bs, extern, #ieval{level=Le}=Ieval) ->
- debugged_cmd({apply,ct_line,line,As}, Bs, Ieval#ieval{level=Le+1}),
- {value, ignore, Bs};
-
do_eval_function(Mod, Name, As0, Bs0, Called, Ieval0) ->
#ieval{level=Le,line=Li,top=Top} = Ieval0,
trace(call, {Called, {Le,Li,Mod,Name,As0}}),
@@ -896,11 +891,6 @@ expr({make_ext_fun,Line,MFA0}, Bs0, Ieval0) ->
exception(error, badarg, Bs, Ieval, true)
end;
-%% Common test adaptation
-expr({call_remote,0,ct_line,line,As0,Lc}, Bs0, Ieval0) ->
- {As,_Bs} = eval_list(As0, Bs0, Ieval0),
- eval_function(ct_line, line, As, Bs0, extern, Ieval0, Lc);
-
%% Local function call
expr({local_call,Line,F,As0,Lc}, Bs0, #ieval{module=M} = Ieval0) ->
Ieval = Ieval0#ieval{line=Line},
diff --git a/lib/debugger/src/int.erl b/lib/debugger/src/int.erl
index 2755db64b8..908390ce50 100644
--- a/lib/debugger/src/int.erl
+++ b/lib/debugger/src/int.erl
@@ -1,7 +1,7 @@
%%
%% %CopyrightBegin%
%%
-%% Copyright Ericsson AB 1998-2013. All Rights Reserved.
+%% Copyright Ericsson AB 1998-2014. All Rights Reserved.
%%
%% The contents of this file are subject to the Erlang Public License,
%% Version 1.1, (the "License"); you may not use this file except in
@@ -265,9 +265,6 @@ first_lines(Clauses) ->
first_line({clause,_L,_Vars,_,Exprs}) ->
first_line(Exprs);
-%% Common Test adaptation
-first_line([{call_remote,0,ct_line,line,_As}|Exprs]) ->
- first_line(Exprs);
first_line([Expr|_Exprs]) -> % Expr = {Op, Line, ..varying no of args..}
element(2, Expr).
diff --git a/lib/eldap/test/eldap_connections_SUITE.erl b/lib/eldap/test/eldap_connections_SUITE.erl
index 4c8aa9c2cf..c5460fef09 100644
--- a/lib/eldap/test/eldap_connections_SUITE.erl
+++ b/lib/eldap/test/eldap_connections_SUITE.erl
@@ -27,27 +27,59 @@
all() ->
[
- tcp_connection,
- tcp_inet6_connection,
- tcp_connection_option,
- tcp_inet6_connection_option
+ {group, v4},
+ {group, v6}
+ ].
+
+
+init_per_group(v4, Config) ->
+ [{listen_opts, []},
+ {listen_host, "localhost"},
+ {connect_opts, []}
+ | Config];
+init_per_group(v6, Config) ->
+ {ok, Hostname} = inet:gethostname(),
+ case lists:member(list_to_atom(Hostname), ct:get_config(ipv6_hosts,[])) of
+ true ->
+ [{listen_opts, [inet6]},
+ {listen_host, "::"},
+ {connect_opts, [{tcpopts,[inet6]}]}
+ | Config];
+ false ->
+ {skip, io_lib:format("~p is not an ipv6_host",[Hostname])}
+ end.
+
+
+end_per_group(_GroupName, Config) ->
+ Config.
+
+
+groups() ->
+ [{v4, [], [tcp_connection, tcp_connection_option]},
+ {v6, [], [tcp_connection, tcp_connection_option]}
].
init_per_suite(Config) -> Config.
+
end_per_suite(_Config) -> ok.
init_per_testcase(_TestCase, Config) ->
- {ok,Sl} = gen_tcp:listen(0,[]),
- {ok,Sl6} = gen_tcp:listen(0,[inet6]),
- [{listen_socket,Sl}, {listen_socket6,Sl6} | Config].
+ case gen_tcp:listen(0, proplists:get_value(listen_opts,Config)) of
+ {ok,LSock} ->
+ {ok,{_,Port}} = inet:sockname(LSock),
+ [{listen_socket,LSock},
+ {listen_port,Port}
+ | Config];
+ Other ->
+ {fail, Other}
+ end.
+
end_per_testcase(_TestCase, Config) ->
- catch gen_tcp:close( proplists:get_value(listen_socket, Config) ),
- catch gen_tcp:close( proplists:get_value(listen_socket6, Config) ),
- ok.
+ catch gen_tcp:close( proplists:get_value(listen_socket, Config) ).
%%%================================================================
%%%
@@ -55,35 +87,26 @@ end_per_testcase(_TestCase, Config) ->
%%%
%%%----------------------------------------------------------------
tcp_connection(Config) ->
- do_tcp_connection(Config, listen_socket, "localhost", []).
-
-tcp_inet6_connection(Config) ->
- do_tcp_connection(Config, listen_socket6, "::", [{tcpopts,[inet6]}]).
-
-
-do_tcp_connection(Config, SockKey, Host, Opts) ->
- Sl = proplists:get_value(SockKey, Config),
- {ok,{_,Port}} = inet:sockname(Sl),
+ Host = proplists:get_value(listen_host, Config),
+ Port = proplists:get_value(listen_port, Config),
+ Opts = proplists:get_value(connect_opts, Config),
case eldap:open([Host], [{port,Port}|Opts]) of
{ok,_H} ->
+ Sl = proplists:get_value(listen_socket, Config),
case gen_tcp:accept(Sl,1000) of
{ok,_S} -> ok;
{error,timeout} -> ct:fail("server side accept timeout",[])
end;
Other -> ct:fail("eldap:open failed: ~p",[Other])
end.
-
-%%%----------------------------------------------------------------
-tcp_connection_option(Config) ->
- do_tcp_connection_option(Config, listen_socket, "localhost", []).
-tcp_inet6_connection_option(Config) ->
- do_tcp_connection_option(Config, listen_socket6, "::", [{tcpopts,[inet6]}]).
-
-do_tcp_connection_option(Config, SockKey, Host, Opts) ->
- Sl = proplists:get_value(SockKey, Config),
- {ok,{_,Port}} = inet:sockname(Sl),
+%%%----------------------------------------------------------------
+tcp_connection_option(Config) ->
+ Host = proplists:get_value(listen_host, Config),
+ Port = proplists:get_value(listen_port, Config),
+ Opts = proplists:get_value(connect_opts, Config),
+ Sl = proplists:get_value(listen_socket, Config),
%% Make an option value to test. The option must be implemented on all
%% platforms that we test on. Must check what the default value is
@@ -95,7 +118,7 @@ do_tcp_connection_option(Config, SockKey, Host, Opts) ->
end,
case catch eldap:open([Host],
- [{port,Port},{tcpopts,[{linger,TestLinger}]}|Opts]) of
+ [{port,Port},{tcpopts,[{linger,TestLinger}]}|Opts]) of
{ok,H} ->
case gen_tcp:accept(Sl,1000) of
{ok,_} ->
@@ -122,5 +145,3 @@ do_tcp_connection_option(Config, SockKey, Host, Opts) ->
Other ->
ct:fail("eldap:open failed: ~p",[Other])
end.
-
-%%%----------------------------------------------------------------
diff --git a/lib/inets/src/ftp/ftp.erl b/lib/inets/src/ftp/ftp.erl
index 5674599ac5..8e51b1be5a 100644
--- a/lib/inets/src/ftp/ftp.erl
+++ b/lib/inets/src/ftp/ftp.erl
@@ -60,6 +60,7 @@
-define(DATA_ACCEPT_TIMEOUT, infinity).
-define(DEFAULT_MODE, passive).
-define(PROGRESS_DEFAULT, ignore).
+-define(FTP_EXT_DEFAULT, false).
%% Internal Constants
-define(FTP_PORT, 21).
@@ -94,7 +95,8 @@
ipfamily, % inet | inet6 | inet6fb4
progress = ignore, % ignore | pid()
dtimeout = ?DATA_ACCEPT_TIMEOUT, % non_neg_integer() | infinity
- tls_upgrading_data_connection = false
+ tls_upgrading_data_connection = false,
+ ftp_extension = ?FTP_EXT_DEFAULT
}).
@@ -969,6 +971,8 @@ start_options(Options) ->
%% timeout
%% dtimeout
%% progress
+%% ftp_extension
+
open_options(Options) ->
?fcrt("open_options", [{options, Options}]),
ValidateMode =
@@ -1013,6 +1017,11 @@ open_options(Options) ->
(_) ->
false
end,
+ ValidateFtpExtension =
+ fun(true) -> true;
+ (false) -> true;
+ (_) -> false
+ end,
ValidOptions =
[{mode, ValidateMode, false, ?DEFAULT_MODE},
{host, ValidateHost, true, ehost},
@@ -1020,7 +1029,8 @@ open_options(Options) ->
{ipfamily, ValidateIpFamily, false, inet},
{timeout, ValidateTimeout, false, ?CONNECTION_TIMEOUT},
{dtimeout, ValidateDTimeout, false, ?DATA_ACCEPT_TIMEOUT},
- {progress, ValidateProgress, false, ?PROGRESS_DEFAULT}],
+ {progress, ValidateProgress, false, ?PROGRESS_DEFAULT},
+ {ftp_extension, ValidateFtpExtension, false, ?FTP_EXT_DEFAULT}],
validate_options(Options, ValidOptions, []).
tls_options(Options) ->
@@ -1174,12 +1184,14 @@ handle_call({_, {open, ip_comm, Opts}}, From, State) ->
DTimeout = key_search(dtimeout, Opts, ?DATA_ACCEPT_TIMEOUT),
Progress = key_search(progress, Opts, ignore),
IpFamily = key_search(ipfamily, Opts, inet),
+ FtpExt = key_search(ftp_extension, Opts, ?FTP_EXT_DEFAULT),
State2 = State#state{client = From,
mode = Mode,
progress = progress(Progress),
ipfamily = IpFamily,
- dtimeout = DTimeout},
+ dtimeout = DTimeout,
+ ftp_extension = FtpExt},
?fcrd("handle_call(open) -> setup ctrl connection with",
[{host, Host}, {port, Port}, {timeout, Timeout}]),
@@ -1202,11 +1214,13 @@ handle_call({_, {open, ip_comm, Host, Opts}}, From, State) ->
Timeout = key_search(timeout, Opts, ?CONNECTION_TIMEOUT),
DTimeout = key_search(dtimeout, Opts, ?DATA_ACCEPT_TIMEOUT),
Progress = key_search(progress, Opts, ignore),
+ FtpExt = key_search(ftp_extension, Opts, ?FTP_EXT_DEFAULT),
State2 = State#state{client = From,
mode = Mode,
progress = progress(Progress),
- dtimeout = DTimeout},
+ dtimeout = DTimeout,
+ ftp_extension = FtpExt},
case setup_ctrl_connection(Host, Port, Timeout, State2) of
{ok, State3, WaitTimeout} ->
@@ -1785,7 +1799,8 @@ handle_ctrl_result({pos_compl, Lines},
ipfamily = inet,
client = From,
caller = {setup_data_connection, Caller},
- timeout = Timeout} = State) ->
+ timeout = Timeout,
+ ftp_extension = false} = State) ->
{_, [?LEFT_PAREN | Rest]} =
lists:splitwith(fun(?LEFT_PAREN) -> false; (_) -> true end, Lines),
@@ -1806,6 +1821,28 @@ handle_ctrl_result({pos_compl, Lines},
{noreply,State#state{client = undefined, caller = undefined}}
end;
+handle_ctrl_result({pos_compl, Lines},
+ #state{mode = passive,
+ ipfamily = inet,
+ client = From,
+ caller = {setup_data_connection, Caller},
+ csock = CSock,
+ timeout = Timeout,
+ ftp_extension = true} = State) ->
+
+ [_, PortStr | _] = lists:reverse(string:tokens(Lines, "|")),
+ {ok, {IP, _}} = peername(CSock),
+
+ ?DBG('<--data tcp connect to ~p:~p, Caller=~p~n',[IP,PortStr,Caller]),
+ case connect(IP, list_to_integer(PortStr), Timeout, State) of
+ {ok, _, Socket} ->
+ handle_caller(State#state{caller = Caller, dsock = {tcp, Socket}});
+ {error, _Reason} = Error ->
+ gen_server:reply(From, Error),
+ {noreply, State#state{client = undefined, caller = undefined}}
+ end;
+
+
%% FTP server does not support passive mode: try to fallback on active mode
handle_ctrl_result(_,
#state{mode = passive,
@@ -2157,7 +2194,8 @@ setup_ctrl_connection(Host, Port, Timeout, State) ->
setup_data_connection(#state{mode = active,
caller = Caller,
- csock = CSock} = State) ->
+ csock = CSock,
+ ftp_extension = FtpExt} = State) ->
case (catch sockname(CSock)) of
{ok, {{_, _, _, _, _, _, _, _} = IP, _}} ->
{ok, LSock} =
@@ -2174,11 +2212,18 @@ setup_data_connection(#state{mode = active,
{ok, LSock} = gen_tcp:listen(0, [{ip, IP}, {active, false},
binary, {packet, 0}]),
{ok, Port} = inet:port(LSock),
- {IP1, IP2, IP3, IP4} = IP,
- {Port1, Port2} = {Port div 256, Port rem 256},
- send_ctrl_message(State,
- mk_cmd("PORT ~w,~w,~w,~w,~w,~w",
- [IP1, IP2, IP3, IP4, Port1, Port2])),
+ case FtpExt of
+ false ->
+ {IP1, IP2, IP3, IP4} = IP,
+ {Port1, Port2} = {Port div 256, Port rem 256},
+ send_ctrl_message(State,
+ mk_cmd("PORT ~w,~w,~w,~w,~w,~w",
+ [IP1, IP2, IP3, IP4, Port1, Port2]));
+ true ->
+ IpAddress = inet_parse:ntoa(IP),
+ Cmd = mk_cmd("EPRT |1|~s|~p|", [IpAddress, Port]),
+ send_ctrl_message(State, Cmd)
+ end,
activate_ctrl_connection(State),
{noreply, State#state{caller = {setup_data_connection,
{LSock, Caller}}}}
@@ -2191,9 +2236,17 @@ setup_data_connection(#state{mode = passive, ipfamily = inet6,
{noreply, State#state{caller = {setup_data_connection, Caller}}};
setup_data_connection(#state{mode = passive, ipfamily = inet,
- caller = Caller} = State) ->
+ caller = Caller,
+ ftp_extension = false} = State) ->
send_ctrl_message(State, mk_cmd("PASV", [])),
activate_ctrl_connection(State),
+ {noreply, State#state{caller = {setup_data_connection, Caller}}};
+
+setup_data_connection(#state{mode = passive, ipfamily = inet,
+ caller = Caller,
+ ftp_extension = true} = State) ->
+ send_ctrl_message(State, mk_cmd("EPSV", [])),
+ activate_ctrl_connection(State),
{noreply, State#state{caller = {setup_data_connection, Caller}}}.
connect(Host, Port, Timeout, #state{ipfamily = inet = IpFam}) ->
diff --git a/lib/inets/src/http_client/httpc_handler.erl b/lib/inets/src/http_client/httpc_handler.erl
index d152d9f0be..0bbd40d656 100644
--- a/lib/inets/src/http_client/httpc_handler.erl
+++ b/lib/inets/src/http_client/httpc_handler.erl
@@ -350,7 +350,7 @@ handle_call(#request{address = Addr} = Request, _,
{reply, ok, State0#state{keep_alive = NewKeepAlive,
session = NewSession}};
undefined ->
- %% Note: tcp-message reciving has already been
+ %% Note: tcp-message receiving has already been
%% activated by handle_pipeline/2.
?hcrd("no current request", []),
cancel_timer(Timers#timers.queue_timer,
@@ -632,7 +632,7 @@ handle_info({timeout, RequestId},
handle_info(timeout_queue, State = #state{request = undefined}) ->
{stop, normal, State};
-%% Timing was such as the pipeline_timout was not canceled!
+%% Timing was such as the queue_timeout was not canceled!
handle_info(timeout_queue, #state{timers = Timers} = State) ->
{noreply, State#state{timers =
Timers#timers{queue_timer = undefined}}};
diff --git a/lib/kernel/src/application_master.erl b/lib/kernel/src/application_master.erl
index bc15b5a7de..7cdbe31ab2 100644
--- a/lib/kernel/src/application_master.erl
+++ b/lib/kernel/src/application_master.erl
@@ -103,9 +103,9 @@ call(AppMaster, Req) ->
%%% The reason for not using the logical structrure is that
%%% the application start function is synchronous, and
%%% that the AM is GL. This means that if AM executed the start
-%%% function, and this function uses spawn_request/1
-%%% or io, deadlock would occur. Therefore, this function is
-%%% executed by the process X. Also, AM needs three loops;
+%%% function, and this function uses io, deadlock would occur.
+%%% Therefore, this function is executed by the process X.
+%%% Also, AM needs three loops;
%%% init_loop (waiting for the start function to return)
%%% main_loop
%%% terminate_loop (waiting for the process to die)
diff --git a/lib/observer/doc/src/observer_ug.xml b/lib/observer/doc/src/observer_ug.xml
index 3aeaf1997a..62f99c5210 100644
--- a/lib/observer/doc/src/observer_ug.xml
+++ b/lib/observer/doc/src/observer_ug.xml
@@ -4,7 +4,7 @@
<chapter>
<header>
<copyright>
- <year>2011</year><year>2013</year>
+ <year>2011</year><year>2014</year>
<holder>Ericsson AB. All Rights Reserved.</holder>
</copyright>
<legalnotice>
@@ -54,9 +54,6 @@
impact only the active viewer is updated and the other
views will be updated when activated.
</p>
- <note>
- <p>Only R15B nodes can be observed.</p>
- </note>
<p> In general the mouse buttons behaves as expected, use left click
to select objects, right click to pop up a menu with most used
diff --git a/lib/parsetools/include/leexinc.hrl b/lib/parsetools/include/leexinc.hrl
index dbbb688d2d..938aef58f9 100644
--- a/lib/parsetools/include/leexinc.hrl
+++ b/lib/parsetools/include/leexinc.hrl
@@ -36,8 +36,8 @@ string(Ics0, L0, Tcs, Ts) ->
string_cont(Ics1, L1, yyaction(A, Alen, Tcs, L0), Ts);
{reject,_Alen,Tlen,_Ics1,L1,_S1} -> % After a non-accepting state
{error,{L0,?MODULE,{illegal,yypre(Tcs, Tlen+1)}},L1};
- {A,Alen,_Tlen,_Ics1,L1,_S1} ->
- string_cont(yysuf(Tcs, Alen), L1, yyaction(A, Alen, Tcs, L0), Ts)
+ {A,Alen,_Tlen,_Ics1,_L1,_S1} ->
+ string_cont(yysuf(Tcs, Alen), L0, yyaction(A, Alen, Tcs, L0), Ts)
end.
%% string_cont(RestChars, Line, Token, Tokens)
@@ -105,8 +105,8 @@ token(S0, Ics0, L0, Tcs, Tlen0, Tline, A0, Alen0) ->
{reject,_Alen1,Tlen1,Ics1,L1,_S1} -> % No token match
Error = {Tline,?MODULE,{illegal,yypre(Tcs, Tlen1+1)}},
{done,{error,Error,L1},Ics1};
- {A1,Alen1,_Tlen1,_Ics1,L1,_S1} -> % Use last accept match
- token_cont(yysuf(Tcs, Alen1), L1, yyaction(A1, Alen1, Tcs, Tline))
+ {A1,Alen1,_Tlen1,_Ics1,_L1,_S1} -> % Use last accept match
+ token_cont(yysuf(Tcs, Alen1), L0, yyaction(A1, Alen1, Tcs, Tline))
end.
%% token_cont(RestChars, Line, Token)
@@ -177,9 +177,9 @@ tokens(S0, Ics0, L0, Tcs, Tlen0, Tline, Ts, A0, Alen0) ->
%% Skip rest of tokens.
Error = {L1,?MODULE,{illegal,yypre(Tcs, Tlen1+1)}},
skip_tokens(yysuf(Tcs, Tlen1+1), L1, Error);
- {A1,Alen1,_Tlen1,_Ics1,L1,_S1} ->
+ {A1,Alen1,_Tlen1,_Ics1,_L1,_S1} ->
Token = yyaction(A1, Alen1, Tcs, Tline),
- tokens_cont(yysuf(Tcs, Alen1), L1, Token, Ts)
+ tokens_cont(yysuf(Tcs, Alen1), L0, Token, Ts)
end.
%% tokens_cont(RestChars, Line, Token, Tokens)
diff --git a/lib/parsetools/test/leex_SUITE.erl b/lib/parsetools/test/leex_SUITE.erl
index eb15bebf63..6d2afe061e 100644
--- a/lib/parsetools/test/leex_SUITE.erl
+++ b/lib/parsetools/test/leex_SUITE.erl
@@ -43,8 +43,8 @@
file/1, compile/1, syntax/1,
pt/1, man/1, ex/1, ex2/1, not_yet/1,
-
- otp_10302/1, otp_11286/1, unicode/1]).
+ line_wrap/1,
+ otp_10302/1, otp_11286/1, unicode/1]).
% Default timetrap timeout (set in init_per_testcase).
-define(default_timeout, ?t:minutes(1)).
@@ -61,12 +61,13 @@ end_per_testcase(_Case, Config) ->
suite() -> [{ct_hooks,[ts_install_cth]}].
all() ->
- [{group, checks}, {group, examples}].
+ [{group, checks}, {group, examples}, {group, bugs}].
groups() ->
[{checks, [], [file, compile, syntax]},
{examples, [], [pt, man, ex, ex2, not_yet, unicode]},
- {tickets, [], [otp_10302, otp_11286]}].
+ {tickets, [], [otp_10302, otp_11286]},
+ {bugs, [], [line_wrap]}].
init_per_suite(Config) ->
Config.
@@ -871,6 +872,48 @@ scan_token_1({more, Cont}, [C | Cs], Fun, Loc, Rs) ->
%% End of ex2
+line_wrap(doc) -> "Much more examples.";
+line_wrap(suite) -> [];
+line_wrap(Config) when is_list(Config) ->
+ Xrl =
+ <<"
+Definitions.
+Rules.
+[a]+[\\n]*= : {token, {first, TokenLine}}.
+[a]+ : {token, {second, TokenLine}}.
+[\\s\\r\\n\\t]+ : skip_token.
+Erlang code.
+ ">>,
+ Dir = ?privdir,
+ XrlFile = filename:join(Dir, "test_line_wrap.xrl"),
+ ?line ok = file:write_file(XrlFile, Xrl),
+ ErlFile = filename:join(Dir, "test_line_wrap.erl"),
+ {ok, _} = leex:file(XrlFile, []),
+ {ok, _} = compile:file(ErlFile, [{outdir,Dir}]),
+ code:purge(test_line_wrap),
+ AbsFile = filename:rootname(ErlFile, ".erl"),
+ code:load_abs(AbsFile, test_line_wrap),
+ fun() ->
+ S = "aaa\naaa",
+ {ok,[{second,1},{second,2}],2} = test_line_wrap:string(S)
+ end(),
+ fun() ->
+ S = "aaa\naaa",
+ {ok,[{second,3},{second,4}],4} = test_line_wrap:string(S, 3)
+ end(),
+ fun() ->
+ {done,{ok,{second,1},1},"\na"} = test_line_wrap:token([], "a\na"),
+ {more,Cont1} = test_line_wrap:token([], "\na"),
+ {done,{ok,{second,2},2},eof} = test_line_wrap:token(Cont1, eof)
+ end(),
+ fun() ->
+ {more,Cont1} = test_line_wrap:tokens([], "a\na"),
+ {done,{ok,[{second,1},{second,2}],2},eof} = test_line_wrap:tokens(Cont1, eof)
+ end(),
+ ok.
+
+%% End of line_wrap
+
not_yet(doc) ->
"Not yet implemented.";
not_yet(suite) -> [];
diff --git a/lib/ssh/doc/src/notes.xml b/lib/ssh/doc/src/notes.xml
index 467e2ab27e..f3db05192e 100644
--- a/lib/ssh/doc/src/notes.xml
+++ b/lib/ssh/doc/src/notes.xml
@@ -29,6 +29,53 @@
<file>notes.xml</file>
</header>
+<section><title>Ssh 3.0.8</title>
+
+ <section><title>Fixed Bugs and Malfunctions</title>
+ <list>
+ <item>
+ <p>
+ Fixes of login blocking after port scanning.</p>
+ <p>
+ Own Id: OTP-12247 Aux Id: seq12726 </p>
+ </item>
+ </list>
+ </section>
+
+</section>
+
+<section><title>Ssh 3.0.7</title>
+
+ <section><title>Fixed Bugs and Malfunctions</title>
+ <list>
+ <item>
+ <p>
+ Add option sftp_vsn to SFTP</p>
+ <p>
+ Own Id: OTP-12227</p>
+ </item>
+ </list>
+ </section>
+
+
+ <section><title>Improvements and New Features</title>
+ <list>
+ <item>
+ <p>
+ Fix option user_interaction to work as expected. When
+ password authentication is implemented with ssh
+ keyboard-interactive method and the password is already
+ supplied, so that we do not need to query user, then
+ connections should succeed even though user_interaction
+ option is set to false.</p>
+ <p>
+ Own Id: OTP-11329 Aux Id: seq12420, seq12335 </p>
+ </item>
+ </list>
+ </section>
+
+</section>
+
<section><title>Ssh 3.0.6</title>
<section><title>Fixed Bugs and Malfunctions</title>
diff --git a/lib/ssh/doc/src/ssh_sftp.xml b/lib/ssh/doc/src/ssh_sftp.xml
index e55d092fe2..f1091e9eca 100644
--- a/lib/ssh/doc/src/ssh_sftp.xml
+++ b/lib/ssh/doc/src/ssh_sftp.xml
@@ -4,7 +4,7 @@
<erlref>
<header>
<copyright>
- <year>2005</year><year>2013</year>
+ <year>2005</year><year>2014</year>
<holder>Ericsson AB. All Rights Reserved.</holder>
</copyright>
<legalnotice>
@@ -81,6 +81,17 @@
<p>The timeout is passed to the ssh_channel start function,
and defaults to infinity.</p>
</item>
+ <tag>
+ <p><c><![CDATA[{sftp_vsn, integer()}]]></c></p>
+ </tag>
+ <item>
+ <p>
+ Desired SFTP protocol version.
+ The actual version will be the minimum of
+ the desired version and the maximum supported
+ versions by the SFTP server.
+ </p>
+ </item>
</taglist>
<p>All other options are directly passed to
<seealso marker="ssh">ssh:connect/3</seealso> or ignored if a
diff --git a/lib/ssh/src/Makefile b/lib/ssh/src/Makefile
index 53c755d3cb..90d71107ad 100644
--- a/lib/ssh/src/Makefile
+++ b/lib/ssh/src/Makefile
@@ -65,6 +65,7 @@ MODULES= \
ssh_cli \
ssh_file \
ssh_io \
+ ssh_info \
ssh_math \
ssh_message \
ssh_no_io \
diff --git a/lib/ssh/src/ssh.app.src b/lib/ssh/src/ssh.app.src
index e0a51b3574..4ad55b34ca 100644
--- a/lib/ssh/src/ssh.app.src
+++ b/lib/ssh/src/ssh.app.src
@@ -23,6 +23,7 @@
sshd_sup,
ssh_file,
ssh_io,
+ ssh_info,
ssh_math,
ssh_no_io,
ssh_server_key_api,
diff --git a/lib/ssh/src/ssh.appup.src b/lib/ssh/src/ssh.appup.src
index 1917c95f5a..600c01454c 100644
--- a/lib/ssh/src/ssh.appup.src
+++ b/lib/ssh/src/ssh.appup.src
@@ -19,9 +19,49 @@
{"%VSN%",
[
+ {"3.0.7", [{load_module, ssh_auth, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_acceptor, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_channel, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_connection, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_connection_handler, soft_purge, soft_purge, []},
+ {load_module, ssh_info, soft_purge, soft_purge, []},
+ {load_module, ssh_message, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_io, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_sftp, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_xfer, soft_purge, soft_purge, [ssh_connection_handler]}]},
+ {"3.0.6", [{load_module, ssh_auth, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_acceptor, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_channel, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_connection, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_connection_handler, soft_purge, soft_purge, []},
+ {load_module, ssh_info, soft_purge, soft_purge, []},
+ {load_module, ssh_message, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_io, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_sftp, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_xfer, soft_purge, soft_purge, [ssh_connection_handler]}]},
{<<".*">>, [{restart_application, ssh}]}
],
[
+ {"3.0.7", [{load_module, ssh_auth, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_acceptor, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_channel, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_connection, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_connection_handler, soft_purge, soft_purge, []},
+ {load_module, ssh_info, soft_purge, soft_purge, []},
+ {load_module, ssh_message, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_io, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_sftp, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_xfer, soft_purge, soft_purge, [ssh_connection_handler]}]},
+ {"3.0.6", [{load_module, ssh_auth, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_acceptor, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_channel, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_connection, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_connection_handler, soft_purge, soft_purge, []},
+ {load_module, ssh_info, soft_purge, soft_purge, []},
+ {load_module, ssh_message, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_io, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_sftp, soft_purge, soft_purge, [ssh_connection_handler]},
+ {load_module, ssh_xfer, soft_purge, soft_purge, [ssh_connection_handler]}]},
{<<".*">>, [{restart_application, ssh}]}
]
}.
diff --git a/lib/ssh/src/ssh_acceptor.erl b/lib/ssh/src/ssh_acceptor.erl
index 7302196674..6c443eeb9c 100644
--- a/lib/ssh/src/ssh_acceptor.erl
+++ b/lib/ssh/src/ssh_acceptor.erl
@@ -22,7 +22,8 @@
-module(ssh_acceptor).
%% Internal application API
--export([start_link/5]).
+-export([start_link/5,
+ number_of_connections/1]).
%% spawn export
-export([acceptor_init/6, acceptor_loop/6]).
@@ -140,5 +141,6 @@ handle_error(Reason) ->
number_of_connections(SystemSup) ->
length([X ||
{R,X,supervisor,[ssh_subsystem_sup]} <- supervisor:which_children(SystemSup),
+ is_pid(X),
is_reference(R)
]).
diff --git a/lib/ssh/src/ssh_auth.erl b/lib/ssh/src/ssh_auth.erl
index 45fd907383..45c4d52d7e 100644
--- a/lib/ssh/src/ssh_auth.erl
+++ b/lib/ssh/src/ssh_auth.erl
@@ -1,7 +1,7 @@
%%
%% %CopyrightBegin%
%%
-%% Copyright Ericsson AB 2008-2013. All Rights Reserved.
+%% Copyright Ericsson AB 2008-2014. All Rights Reserved.
%%
%% The contents of this file are subject to the Erlang Public License,
%% Version 1.1, (the "License"); you may not use this file except in
@@ -119,8 +119,7 @@ init_userauth_request_msg(#ssh{opts = Opts} = Ssh) ->
false ->
FirstAlg = proplists:get_value(public_key_alg, Opts, ?PREFERRED_PK_ALG),
SecondAlg = other_alg(FirstAlg),
- AllowUserInt = proplists:get_value(user_interaction, Opts, true),
- Prefs = method_preference(FirstAlg, SecondAlg, AllowUserInt),
+ Prefs = method_preference(FirstAlg, SecondAlg),
ssh_transport:ssh_packet(Msg, Ssh#ssh{user = User,
userauth_preference = Prefs,
userauth_methods = none,
@@ -130,15 +129,13 @@ init_userauth_request_msg(#ssh{opts = Opts} = Ssh) ->
case length(Algs) =:= 2 of
true ->
SecondAlg = other_alg(FirstAlg),
- AllowUserInt = proplists:get_value(user_interaction, Opts, true),
- Prefs = method_preference(FirstAlg, SecondAlg, AllowUserInt),
+ Prefs = method_preference(FirstAlg, SecondAlg),
ssh_transport:ssh_packet(Msg, Ssh#ssh{user = User,
userauth_preference = Prefs,
userauth_methods = none,
service = "ssh-connection"});
_ ->
- AllowUserInt = proplists:get_value(user_interaction, Opts, true),
- Prefs = method_preference(FirstAlg, AllowUserInt),
+ Prefs = method_preference(FirstAlg),
ssh_transport:ssh_packet(Msg, Ssh#ssh{user = User,
userauth_preference = Prefs,
userauth_methods = none,
@@ -187,9 +184,8 @@ handle_userauth_request(#ssh_msg_service_request{name =
handle_userauth_request(#ssh_msg_userauth_request{user = User,
service = "ssh-connection",
method = "password",
- data = Data}, _,
+ data = <<?FALSE, ?UINT32(Sz), BinPwd:Sz/binary>>}, _,
#ssh{opts = Opts} = Ssh) ->
- <<_:8, ?UINT32(Sz), BinPwd:Sz/binary>> = Data,
Password = unicode:characters_to_list(BinPwd),
case check_password(User, Password, Opts) of
true ->
@@ -204,6 +200,27 @@ handle_userauth_request(#ssh_msg_userauth_request{user = User,
handle_userauth_request(#ssh_msg_userauth_request{user = User,
service = "ssh-connection",
+ method = "password",
+ data = <<?TRUE,
+ _/binary
+ %% ?UINT32(Sz1), OldBinPwd:Sz1/binary,
+ %% ?UINT32(Sz2), NewBinPwd:Sz2/binary
+ >>
+ }, _,
+ Ssh) ->
+ %% Password change without us having sent SSH_MSG_USERAUTH_PASSWD_CHANGEREQ (because we never do)
+ %% RFC 4252 says:
+ %% SSH_MSG_USERAUTH_FAILURE without partial success - The password
+ %% has not been changed. Either password changing was not supported,
+ %% or the old password was bad.
+
+ {not_authorized, {User, {error,"Password change not supported"}},
+ ssh_transport:ssh_packet(#ssh_msg_userauth_failure{
+ authentications = "",
+ partial_success = false}, Ssh)};
+
+handle_userauth_request(#ssh_msg_userauth_request{user = User,
+ service = "ssh-connection",
method = "none"}, _,
#ssh{userauth_supported_methods = Methods} = Ssh) ->
{not_authorized, {User, undefined},
@@ -256,15 +273,12 @@ handle_userauth_info_request(
data = Data}, IoCb,
#ssh{opts = Opts} = Ssh) ->
PromptInfos = decode_keyboard_interactive_prompts(NumPrompts,Data),
- Resps = keyboard_interact_get_responses(IoCb, Opts,
+ Responses = keyboard_interact_get_responses(IoCb, Opts,
Name, Instr, PromptInfos),
- RespBin = list_to_binary(
- lists:map(fun(S) -> <<?STRING(list_to_binary(S))>> end,
- Resps)),
{ok,
ssh_transport:ssh_packet(
#ssh_msg_userauth_info_response{num_responses = NumPrompts,
- data = RespBin}, Ssh)}.
+ data = Responses}, Ssh)}.
handle_userauth_info_response(#ssh_msg_userauth_info_response{},
_Auth) ->
@@ -276,25 +290,16 @@ handle_userauth_info_response(#ssh_msg_userauth_info_response{},
%%--------------------------------------------------------------------
%%% Internal functions
%%--------------------------------------------------------------------
-method_preference(Alg1, Alg2, true) ->
+method_preference(Alg1, Alg2) ->
[{"publickey", ?MODULE, publickey_msg, [Alg1]},
{"publickey", ?MODULE, publickey_msg,[Alg2]},
{"password", ?MODULE, password_msg, []},
{"keyboard-interactive", ?MODULE, keyboard_interactive_msg, []}
- ];
-method_preference(Alg1, Alg2, false) ->
- [{"publickey", ?MODULE, publickey_msg, [Alg1]},
- {"publickey", ?MODULE, publickey_msg,[Alg2]},
- {"password", ?MODULE, password_msg, []}
].
-method_preference(Alg1, true) ->
+method_preference(Alg1) ->
[{"publickey", ?MODULE, publickey_msg, [Alg1]},
{"password", ?MODULE, password_msg, []},
{"keyboard-interactive", ?MODULE, keyboard_interactive_msg, []}
- ];
-method_preference(Alg1, false) ->
- [{"publickey", ?MODULE, publickey_msg, [Alg1]},
- {"password", ?MODULE, password_msg, []}
].
user_name(Opts) ->
@@ -362,35 +367,29 @@ build_sig_data(SessionId, User, Service, KeyBlob, Alg) ->
algorithm_string('ssh-rsa') ->
"ssh-rsa";
algorithm_string('ssh-dss') ->
- "ssh-dss".
+ "ssh-dss".
decode_keyboard_interactive_prompts(_NumPrompts, Data) ->
ssh_message:decode_keyboard_interactive_prompts(Data, []).
keyboard_interact_get_responses(IoCb, Opts, Name, Instr, PromptInfos) ->
NumPrompts = length(PromptInfos),
- case proplists:get_value(keyboard_interact_fun, Opts) of
- undefined when NumPrompts == 1 ->
- %% Special case/fallback for just one prompt
- %% (assumed to be the password prompt)
- case proplists:get_value(password, Opts) of
- undefined -> keyboard_interact(IoCb, Name, Instr, PromptInfos, Opts);
- PW -> [PW]
- end;
- undefined ->
- keyboard_interact(IoCb, Name, Instr, PromptInfos, Opts);
- KbdInteractFun ->
- Prompts = lists:map(fun({Prompt, _Echo}) -> Prompt end,
- PromptInfos),
- case KbdInteractFun(Name, Instr, Prompts) of
- Rs when length(Rs) == NumPrompts ->
- Rs;
- Rs ->
- erlang:error({mismatching_number_of_responses,
- {got,Rs},
- {expected,NumPrompts}})
- end
- end.
+ keyboard_interact_get_responses(proplists:get_value(user_interaction, Opts, true),
+ proplists:get_value(keyboard_interact_fun, Opts),
+ proplists:get_value(password, Opts, undefined), IoCb, Name,
+ Instr, PromptInfos, Opts, NumPrompts).
+
+keyboard_interact_get_responses(_, undefined, Password, _, _, _, _, _,
+ 1) when Password =/= undefined ->
+ [Password]; %% Password auth implemented with keyboard-interaction and passwd is known
+keyboard_interact_get_responses(_, _, _, _, _, _, _, _, 0) ->
+ [""];
+keyboard_interact_get_responses(false, undefined, undefined, _, _, _, [Prompt|_], Opts, _) ->
+ ssh_no_io:read_line(Prompt, Opts); %% Throws error as keyboard interaction is not allowed
+keyboard_interact_get_responses(true, undefined, _,IoCb, Name, Instr, PromptInfos, Opts, _) ->
+ keyboard_interact(IoCb, Name, Instr, PromptInfos, Opts);
+keyboard_interact_get_responses(true, Fun, _, Name, Instr, PromptInfos, _, _, NumPrompts) ->
+ keyboard_interact_fun(Fun, Name, Instr, PromptInfos, NumPrompts).
keyboard_interact(IoCb, Name, Instr, Prompts, Opts) ->
if Name /= "" -> IoCb:format("~s", [Name]);
@@ -404,6 +403,21 @@ keyboard_interact(IoCb, Name, Instr, Prompts, Opts) ->
end,
Prompts).
+keyboard_interact_fun(KbdInteractFun, Name, Instr, PromptInfos, NumPrompts) ->
+ Prompts = lists:map(fun({Prompt, _Echo}) -> Prompt end,
+ PromptInfos),
+ case KbdInteractFun(Name, Instr, Prompts) of
+ Rs when length(Rs) == NumPrompts ->
+ Rs;
+ Rs ->
+ throw({mismatching_number_of_responses,
+ {got,Rs},
+ {expected, NumPrompts},
+ #ssh_msg_disconnect{code = ?SSH_DISCONNECT_SERVICE_NOT_AVAILABLE,
+ description = "User interaction failed",
+ language = "en"}})
+ end.
+
other_alg('ssh-rsa') ->
'ssh-dss';
other_alg('ssh-dss') ->
diff --git a/lib/ssh/src/ssh_channel.erl b/lib/ssh/src/ssh_channel.erl
index 508ae637cf..5c24f362b1 100644
--- a/lib/ssh/src/ssh_channel.erl
+++ b/lib/ssh/src/ssh_channel.erl
@@ -67,7 +67,8 @@
%% Internal application API
-export([cache_create/0, cache_lookup/2, cache_update/2,
cache_delete/1, cache_delete/2, cache_foldl/3,
- cache_find/2]).
+ cache_find/2,
+ get_print_info/1]).
-record(state, {
cm,
@@ -190,6 +191,14 @@ init([Options]) ->
%% {stop, Reason, State}
%% Description: Handling call messages
%%--------------------------------------------------------------------
+handle_call(get_print_info, _From, State) ->
+ Reply =
+ {{State#state.cm,
+ State#state.channel_id},
+ io_lib:format('CB=~p',[State#state.channel_cb])
+ },
+ {reply, Reply, State};
+
handle_call(Request, From, #state{channel_cb = Module,
channel_state = ChannelState} = State) ->
try Module:handle_call(Request, From, ChannelState) of
@@ -333,6 +342,9 @@ cache_find(ChannelPid, Cache) ->
Channel
end.
+get_print_info(Pid) ->
+ call(Pid, get_print_info, 1000).
+
%%--------------------------------------------------------------------
%%% Internal functions
%%--------------------------------------------------------------------
diff --git a/lib/ssh/src/ssh_connection.erl b/lib/ssh/src/ssh_connection.erl
index 33849f4527..f3ff9ae67a 100644
--- a/lib/ssh/src/ssh_connection.erl
+++ b/lib/ssh/src/ssh_connection.erl
@@ -1,7 +1,7 @@
%%
%% %CopyrightBegin%
%%
-%% Copyright Ericsson AB 2008-2013. All Rights Reserved.
+%% Copyright Ericsson AB 2008-2014. All Rights Reserved.
%%
%% The contents of this file are subject to the Erlang Public License,
%% Version 1.1, (the "License"); you may not use this file except in
@@ -107,9 +107,15 @@ shell(ConnectionHandler, ChannelId) ->
%% Description: Executes a predefined subsystem.
%%--------------------------------------------------------------------
subsystem(ConnectionHandler, ChannelId, SubSystem, TimeOut) ->
- ssh_connection_handler:request(ConnectionHandler, self(),
- ChannelId, "subsystem",
- true, [?string(SubSystem)], TimeOut).
+ case ssh_connection_handler:request(ConnectionHandler, self(),
+ ChannelId, "subsystem",
+ true, [?string(SubSystem)], TimeOut) of
+ success -> success;
+ failure -> failure;
+ {error,timeout} -> {error,timeout};
+ _ -> failure
+ end.
+
%%--------------------------------------------------------------------
-spec send(pid(), channel_id(), iodata()) ->
ok | {error, closed}.
diff --git a/lib/ssh/src/ssh_connection_handler.erl b/lib/ssh/src/ssh_connection_handler.erl
index 4fbc5d0ae2..fa107be1b1 100644
--- a/lib/ssh/src/ssh_connection_handler.erl
+++ b/lib/ssh/src/ssh_connection_handler.erl
@@ -41,11 +41,13 @@
global_request/4, send/5, send_eof/2, info/1, info/2,
connection_info/2, channel_info/3,
adjust_window/3, close/2, stop/1, renegotiate/1, renegotiate_data/1,
- start_connection/4]).
+ start_connection/4,
+ get_print_info/1]).
%% gen_fsm callbacks
-export([hello/2, kexinit/2, key_exchange/2, new_keys/2,
- userauth/2, connected/2]).
+ userauth/2, connected/2,
+ error/2]).
-export([init/1, handle_event/3,
handle_sync_event/4, handle_info/3, terminate/3, format_status/2, code_change/4]).
@@ -171,9 +173,23 @@ init([Role, Socket, SshOpts]) ->
State#state{ssh_params = Ssh})
catch
_:Error ->
- gen_fsm:enter_loop(?MODULE, [], error, {Error, State0})
+ gen_fsm:enter_loop(?MODULE, [], error, {Error, State})
end.
+%% Temporary fix for the Nessus error. SYN-> <-SYNACK ACK-> RST-> ?
+error(_Event, {Error,State=#state{}}) ->
+ case Error of
+ {badmatch,{error,enotconn}} ->
+ %% {error,enotconn} probably from inet:peername in
+ %% init_ssh(server,..)/5 called from init/1
+ {stop, {shutdown,"TCP connenction to server was prematurely closed by the client"}, State};
+ _ ->
+ {stop, {shutdown,{init,Error}}, State}
+ end;
+error(Event, State) ->
+ %% State deliberately not checked beeing #state. This is a panic-clause...
+ {stop, {shutdown,{init,{spurious_error,Event}}}, State}.
+
%%--------------------------------------------------------------------
-spec open_channel(pid(), string(), iodata(), integer(), integer(),
timeout()) -> {open, channel_id()} | {error, term()}.
@@ -240,6 +256,9 @@ send_eof(ConnectionHandler, ChannelId) ->
%%--------------------------------------------------------------------
-spec connection_info(pid(), [atom()]) -> proplists:proplist().
%%--------------------------------------------------------------------
+get_print_info(ConnectionHandler) ->
+ sync_send_all_state_event(ConnectionHandler, get_print_info, 1000).
+
connection_info(ConnectionHandler, Options) ->
sync_send_all_state_event(ConnectionHandler, {connection_info, Options}).
@@ -550,7 +569,7 @@ connected({#ssh_msg_kexinit{}, _Payload} = Event, State) ->
%%--------------------------------------------------------------------
handle_event(#ssh_msg_disconnect{description = Desc} = DisconnectMsg, _StateName, #state{} = State) ->
- handle_disconnect(DisconnectMsg, State),
+ handle_disconnect(peer, DisconnectMsg, State),
{stop, {shutdown, Desc}, State};
handle_event(#ssh_msg_ignore{}, StateName, State) ->
@@ -758,6 +777,20 @@ handle_sync_event({recv_window, ChannelId}, _From, StateName,
end,
{reply, Reply, StateName, next_packet(State)};
+handle_sync_event(get_print_info, _From, StateName, State) ->
+ Reply =
+ try
+ {inet:sockname(State#state.socket),
+ inet:peername(State#state.socket)
+ }
+ of
+ {{ok,Local}, {ok,Remote}} -> {{Local,Remote},io_lib:format("statename=~p",[StateName])};
+ _ -> {{"-",0},"-"}
+ catch
+ _:_ -> {{"?",0},"?"}
+ end,
+ {reply, Reply, StateName, State};
+
handle_sync_event({connection_info, Options}, _From, StateName, State) ->
Info = ssh_info(Options, State, []),
{reply, Info, StateName, State};
@@ -936,6 +969,10 @@ terminate(normal, _, #state{transport_cb = Transport,
(catch Transport:close(Socket)),
ok;
+terminate({shutdown,{init,Reason}}, StateName, State) ->
+ error_logger:info_report(io_lib:format("Erlang ssh in connection handler init: ~p~n",[Reason])),
+ terminate(normal, StateName, State);
+
%% Terminated by supervisor
terminate(shutdown, StateName, #state{ssh_params = Ssh0} = State) ->
DisconnectMsg =
@@ -951,8 +988,10 @@ terminate({shutdown, #ssh_msg_disconnect{} = Msg}, StateName,
{SshPacket, Ssh} = ssh_transport:ssh_packet(Msg, Ssh0),
send_msg(SshPacket, State),
terminate(normal, StateName, State#state{ssh_params = Ssh});
+
terminate({shutdown, _}, StateName, State) ->
terminate(normal, StateName, State);
+
terminate(Reason, StateName, #state{ssh_params = Ssh0, starter = _Pid,
connection_state = Connection} = State) ->
terminate_subsytem(Connection),
@@ -965,6 +1004,7 @@ terminate(Reason, StateName, #state{ssh_params = Ssh0, starter = _Pid,
send_msg(SshPacket, State),
terminate(normal, StateName, State#state{ssh_params = Ssh}).
+
terminate_subsytem(#connection{system_supervisor = SysSup,
sub_system_supervisor = SubSysSup}) when is_pid(SubSysSup) ->
ssh_system_sup:stop_subsystem(SysSup, SubSysSup);
@@ -1161,7 +1201,10 @@ send_all_state_event(FsmPid, Event) ->
gen_fsm:send_all_state_event(FsmPid, Event).
sync_send_all_state_event(FsmPid, Event) ->
- try gen_fsm:sync_send_all_state_event(FsmPid, Event, infinity)
+ sync_send_all_state_event(FsmPid, Event, infinity).
+
+sync_send_all_state_event(FsmPid, Event, Timeout) ->
+ try gen_fsm:sync_send_all_state_event(FsmPid, Event, Timeout)
catch
exit:{noproc, _} ->
{error, closed};
@@ -1258,13 +1301,23 @@ generate_event(<<?BYTE(Byte), _/binary>> = Msg, StateName,
generate_event(Msg, StateName, State0, EncData) ->
Event = ssh_message:decode(Msg),
State = generate_event_new_state(State0, EncData),
- case Event of
- #ssh_msg_kexinit{} ->
- %% We need payload for verification later.
- event({Event, Msg}, StateName, State);
- _ ->
- event(Event, StateName, State)
- end.
+ try
+ case Event of
+ #ssh_msg_kexinit{} ->
+ %% We need payload for verification later.
+ event({Event, Msg}, StateName, State);
+ _ ->
+ event(Event, StateName, State)
+ end
+ catch
+ _:_ ->
+ DisconnectMsg =
+ #ssh_msg_disconnect{code = ?SSH_DISCONNECT_PROTOCOL_ERROR,
+ description = "Encountered unexpected input",
+ language = "en"},
+ handle_disconnect(DisconnectMsg, State)
+ end.
+
handle_request(ChannelPid, ChannelId, Type, Data, WantReply, From,
@@ -1442,17 +1495,27 @@ handle_ssh_packet(Length, StateName, #state{decoded_data_buffer = DecData0,
handle_disconnect(DisconnectMsg, State0)
end.
-handle_disconnect(#ssh_msg_disconnect{description = Desc} = Msg, #state{connection_state = Connection0,
- role = Role} = State0) ->
+handle_disconnect(DisconnectMsg, State) ->
+ handle_disconnect(own, DisconnectMsg, State).
+
+handle_disconnect(#ssh_msg_disconnect{} = DisconnectMsg, State, Error) ->
+ handle_disconnect(own, DisconnectMsg, State, Error);
+handle_disconnect(Type, #ssh_msg_disconnect{description = Desc} = Msg, #state{connection_state = Connection0, role = Role} = State0) ->
{disconnect, _, {{replies, Replies}, Connection}} = ssh_connection:handle_msg(Msg, Connection0, Role),
- State = send_replies(Replies, State0),
+ State = send_replies(disconnect_replies(Type, Msg, Replies), State0),
{stop, {shutdown, Desc}, State#state{connection_state = Connection}}.
-handle_disconnect(#ssh_msg_disconnect{description = Desc} = Msg, #state{connection_state = Connection0,
- role = Role} = State0, ErrorMsg) ->
+
+handle_disconnect(Type, #ssh_msg_disconnect{description = Desc} = Msg, #state{connection_state = Connection0,
+ role = Role} = State0, ErrorMsg) ->
{disconnect, _, {{replies, Replies}, Connection}} = ssh_connection:handle_msg(Msg, Connection0, Role),
- State = send_replies(Replies, State0),
+ State = send_replies(disconnect_replies(Type, Msg, Replies), State0),
{stop, {shutdown, {Desc, ErrorMsg}}, State#state{connection_state = Connection}}.
+disconnect_replies(own, Msg, Replies) ->
+ [{connection_reply, Msg} | Replies];
+disconnect_replies(peer, _, Replies) ->
+ Replies.
+
counterpart_versions(NumVsn, StrVsn, #ssh{role = server} = Ssh) ->
Ssh#ssh{c_vsn = NumVsn , c_version = StrVsn};
counterpart_versions(NumVsn, StrVsn, #ssh{role = client} = Ssh) ->
diff --git a/lib/ssh/src/ssh_info.erl b/lib/ssh/src/ssh_info.erl
new file mode 100644
index 0000000000..9ed598b3ab
--- /dev/null
+++ b/lib/ssh/src/ssh_info.erl
@@ -0,0 +1,193 @@
+%%
+%% %CopyrightBegin%
+%%
+%% Copyright Ericsson AB 2008-2014. All Rights Reserved.
+%%
+%% The contents of this file are subject to the Erlang Public License,
+%% Version 1.1, (the "License"); you may not use this file except in
+%% compliance with the License. You should have received a copy of the
+%% Erlang Public License along with this software. If not, it can be
+%% retrieved online at http://www.erlang.org/.
+%%
+%% Software distributed under the License is distributed on an "AS IS"
+%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
+%% the License for the specific language governing rights and limitations
+%% under the License.
+%%
+%% %CopyrightEnd%
+%%
+
+%%
+%%----------------------------------------------------------------------
+%% Purpose: Print some info of a running ssh aplication.
+%%----------------------------------------------------------------------
+
+-module(ssh_info).
+
+-compile(export_all).
+
+print() ->
+ try supervisor:which_children(ssh_sup)
+ of
+ _ ->
+ io:nl(),
+ print_general(),
+ io:nl(),
+ underline("Client part", $=),
+ print_clients(),
+ io:nl(),
+ underline("Server part", $=),
+ print_servers(),
+ io:nl(),
+ %% case os:type() of
+ %% {unix,_} ->
+ %% io:nl(),
+ %% underline("Linux part", $=),
+ %% underline("Listening"),
+ %% catch io:format(os:cmd("netstat -tpln")),
+ %% io:nl(),
+ %% underline("Other"),
+ %% catch io:format(os:cmd("netstat -tpn"));
+ %% _ -> ok
+ %% end,
+ underline("Supervisors", $=),
+ walk_sups(ssh_sup),
+ io:nl()
+ catch
+ _:_ ->
+ io:format("Ssh not found~n",[])
+ end.
+
+%%%================================================================
+print_general() ->
+ {_Name, Slogan, Ver} = lists:keyfind(ssh,1,application:which_applications()),
+ underline(io_lib:format("~s ~s", [Slogan, Ver]), $=),
+ io:format('This printout is generated ~s. ~n',[datetime()]).
+
+%%%================================================================
+print_clients() ->
+ try
+ lists:foreach(fun print_client/1, supervisor:which_children(sshc_sup))
+ catch
+ C:E ->
+ io:format('***FAILED: ~p:~p~n',[C,E])
+ end.
+
+print_client({undefined,Pid,supervisor,[ssh_connection_handler]}) ->
+ {{Local,Remote},_Str} = ssh_connection_handler:get_print_info(Pid),
+ io:format(" Local=~s Remote=~s~n",[fmt_host_port(Local),fmt_host_port(Remote)]);
+print_client(Other) ->
+ io:format(" [[Other 1: ~p]]~n",[Other]).
+
+
+%%%================================================================
+print_servers() ->
+ try
+ lists:foreach(fun print_server/1, supervisor:which_children(sshd_sup))
+ catch
+ C:E ->
+ io:format('***FAILED: ~p:~p~n',[C,E])
+ end.
+
+print_server({{server,ssh_system_sup,LocalHost,LocalPort},Pid,supervisor,[ssh_system_sup]}) when is_pid(Pid) ->
+ io:format('Local=~s (~p children)~n',[fmt_host_port({LocalHost,LocalPort}),
+ ssh_acceptor:number_of_connections(Pid)]),
+ lists:foreach(fun print_system_sup/1, supervisor:which_children(Pid));
+print_server(Other) ->
+ io:format(" [[Other 2: ~p]]~n",[Other]).
+
+print_system_sup({Ref,Pid,supervisor,[ssh_subsystem_sup]}) when is_reference(Ref),
+ is_pid(Pid) ->
+ lists:foreach(fun print_channels/1, supervisor:which_children(Pid));
+print_system_sup({{ssh_acceptor_sup,LocalHost,LocalPort}, Pid,supervisor, [ssh_acceptor_sup]}) when is_pid(Pid) ->
+ io:format(" [Acceptor for ~s]~n",[fmt_host_port({LocalHost,LocalPort})]);
+print_system_sup(Other) ->
+ io:format(" [[Other 3: ~p]]~n",[Other]).
+
+print_channels({{server,ssh_channel_sup,_,_},Pid,supervisor,[ssh_channel_sup]}) when is_pid(Pid) ->
+ lists:foreach(fun print_channel/1, supervisor:which_children(Pid));
+print_channels(Other) ->
+ io:format(" [[Other 4: ~p]]~n",[Other]).
+
+
+print_channel({Ref,Pid,worker,[ssh_channel]}) when is_reference(Ref),
+ is_pid(Pid) ->
+ {{ConnManager,ChannelID}, Str} = ssh_channel:get_print_info(Pid),
+ {{Local,Remote},StrM} = ssh_connection_handler:get_print_info(ConnManager),
+ io:format(' ch ~p: ~s ~s',[ChannelID, StrM, Str]),
+ io:format(" Local=~s Remote=~s~n",[fmt_host_port(Local),fmt_host_port(Remote)]);
+print_channel(Other) ->
+ io:format(" [[Other 5: ~p]]~n",[Other]).
+
+%%%================================================================
+-define(inc(N), (N+4)).
+
+walk_sups(StartPid) ->
+ io:format("Start at ~p, ~s.~n",[StartPid,dead_or_alive(StartPid)]),
+ walk_sups(children(StartPid), _Indent=?inc(0)).
+
+walk_sups([H={_,Pid,SupOrWorker,_}|T], Indent) ->
+ indent(Indent), io:format('~200p ~p is ~s~n',[H,Pid,dead_or_alive(Pid)]),
+ case SupOrWorker of
+ supervisor -> walk_sups(children(Pid), ?inc(Indent));
+ _ -> ok
+ end,
+ walk_sups(T, Indent);
+walk_sups([], _) ->
+ ok.
+
+dead_or_alive(Name) when is_atom(Name) ->
+ case whereis(Name) of
+ undefined ->
+ "**UNDEFINED**";
+ Pid ->
+ dead_or_alive(Pid)
+ end;
+dead_or_alive(Pid) when is_pid(Pid) ->
+ case process_info(Pid) of
+ undefined -> "**DEAD**";
+ _ -> "alive"
+ end.
+
+indent(I) -> io:format('~*c',[I,$ ]).
+
+children(Pid) ->
+ Parent = self(),
+ Helper = spawn(fun() ->
+ Parent ! {self(),supervisor:which_children(Pid)}
+ end),
+ receive
+ {Helper,L} when is_list(L) ->
+ L
+ after
+ 2000 ->
+ catch exit(Helper, kill),
+ []
+ end.
+
+%%%================================================================
+underline(Str) ->
+ underline(Str, $-).
+
+underline(Str, LineChar) ->
+ Len = lists:flatlength(Str),
+ io:format('~s~n',[Str]),
+ line(Len,LineChar).
+
+line(Len, Char) ->
+ io:format('~*c~n', [Len,Char]).
+
+
+datetime() ->
+ {{YYYY,MM,DD}, {H,M,S}} = calendar:now_to_universal_time(now()),
+ lists:flatten(io_lib:format('~4w-~2..0w-~2..0w ~2..0w:~2..0w:~2..0w UTC',[YYYY,MM,DD, H,M,S])).
+
+
+fmt_host_port({{A,B,C,D},Port}) -> io_lib:format('~p.~p.~p.~p:~p',[A,B,C,D,Port]);
+fmt_host_port({Host,Port}) -> io_lib:format('~s:~p',[Host,Port]).
+
+
+
+nyi() ->
+ io:format('Not yet implemented~n',[]),
+ nyi.
diff --git a/lib/ssh/src/ssh_io.erl b/lib/ssh/src/ssh_io.erl
index 35336bce8b..97e2dee27a 100644
--- a/lib/ssh/src/ssh_io.erl
+++ b/lib/ssh/src/ssh_io.erl
@@ -1,7 +1,7 @@
%%
%% %CopyrightBegin%
%%
-%% Copyright Ericsson AB 2005-2013. All Rights Reserved.
+%% Copyright Ericsson AB 2005-2014. All Rights Reserved.
%%
%% The contents of this file are subject to the Erlang Public License,
%% Version 1.1, (the "License"); you may not use this file except in
@@ -73,7 +73,9 @@ read_password(Prompt, Ssh) ->
listify(A) when is_atom(A) ->
atom_to_list(A);
listify(L) when is_list(L) ->
- L.
+ L;
+listify(B) when is_binary(B) ->
+ binary_to_list(B).
format(Fmt, Args) ->
io:format(Fmt, Args).
diff --git a/lib/ssh/src/ssh_message.erl b/lib/ssh/src/ssh_message.erl
index 76b57cb995..66e7717095 100644
--- a/lib/ssh/src/ssh_message.erl
+++ b/lib/ssh/src/ssh_message.erl
@@ -1,7 +1,7 @@
%%
%% %CopyrightBegin%
%%
-%% Copyright Ericsson AB 2013-2013. All Rights Reserved.
+%% Copyright Ericsson AB 2013-2014. All Rights Reserved.
%%
%% The contents of this file are subject to the Erlang Public License,
%% Version 1.1, (the "License"); you may not use this file except in
@@ -162,8 +162,15 @@ encode(#ssh_msg_userauth_info_request{
encode(#ssh_msg_userauth_info_response{
num_responses = Num,
data = Data}) ->
- ssh_bits:encode([?SSH_MSG_USERAUTH_INFO_RESPONSE, Num, Data],
- [byte, uint32, '...']);
+ Responses = lists:map(fun("") ->
+ <<>>;
+ (Response) ->
+ ssh_bits:encode([Response], [string])
+ end, Data),
+ Start = ssh_bits:encode([?SSH_MSG_USERAUTH_INFO_RESPONSE, Num],
+ [byte, uint32]),
+ iolist_to_binary([Start, Responses]);
+
encode(#ssh_msg_disconnect{
code = Code,
description = Desc,
@@ -498,6 +505,11 @@ erl_boolean(1) ->
decode_kex_init(<<?BYTE(Bool), ?UINT32(X)>>, Acc, 0) ->
list_to_tuple(lists:reverse([X, erl_boolean(Bool) | Acc]));
+decode_kex_init(<<?BYTE(Bool)>>, Acc, 0) ->
+ %% The mandatory trailing UINT32 is missing. Assume the value it anyhow must have
+ %% See rfc 4253 7.1
+ X = 0,
+ list_to_tuple(lists:reverse([X, erl_boolean(Bool) | Acc]));
decode_kex_init(<<?UINT32(Len), Data:Len/binary, Rest/binary>>, Acc, N) ->
Names = string:tokens(unicode:characters_to_list(Data), ","),
decode_kex_init(Rest, [Names | Acc], N -1).
diff --git a/lib/ssh/src/ssh_sftp.erl b/lib/ssh/src/ssh_sftp.erl
index 0ea2366ac7..721146c509 100644
--- a/lib/ssh/src/ssh_sftp.erl
+++ b/lib/ssh/src/ssh_sftp.erl
@@ -1,7 +1,7 @@
%%
%% %CopyrightBegin%
%%
-%% Copyright Ericsson AB 2005-2013. All Rights Reserved.
+%% Copyright Ericsson AB 2005-2014. All Rights Reserved.
%%
%% The contents of this file are subject to the Erlang Public License,
%% Version 1.1, (the "License"); you may not use this file except in
@@ -57,7 +57,8 @@
rep_buf = <<>>,
req_id,
req_list = [], %% {ReqId, Fun}
- inf %% list of fileinf
+ inf, %% list of fileinf,
+ opts
}).
-record(fileinf,
@@ -85,10 +86,11 @@ start_channel(Host) when is_list(Host) ->
start_channel(Host, []).
start_channel(Cm, Opts) when is_pid(Cm) ->
Timeout = proplists:get_value(timeout, Opts, infinity),
+ {_, SftpOpts} = handle_options(Opts, [], []),
case ssh_xfer:attach(Cm, []) of
{ok, ChannelId, Cm} ->
case ssh_channel:start(Cm, ChannelId,
- ?MODULE, [Cm, ChannelId, Timeout]) of
+ ?MODULE, [Cm, ChannelId, SftpOpts]) of
{ok, Pid} ->
case wait_for_version_negotiation(Pid, Timeout) of
ok ->
@@ -108,11 +110,12 @@ start_channel(Cm, Opts) when is_pid(Cm) ->
start_channel(Host, Opts) ->
start_channel(Host, 22, Opts).
start_channel(Host, Port, Opts) ->
- Timeout = proplists:get_value(timeout, Opts, infinity),
- case ssh_xfer:connect(Host, Port, proplists:delete(timeout, Opts)) of
+ {SshOpts, SftpOpts} = handle_options(Opts, [], []),
+ Timeout = proplists:get_value(timeout, SftpOpts, infinity),
+ case ssh_xfer:connect(Host, Port, SshOpts) of
{ok, ChannelId, Cm} ->
case ssh_channel:start(Cm, ChannelId, ?MODULE, [Cm,
- ChannelId, Timeout]) of
+ ChannelId, SftpOpts]) of
{ok, Pid} ->
case wait_for_version_negotiation(Pid, Timeout) of
ok ->
@@ -392,7 +395,8 @@ write_file_loop(Pid, Handle, Pos, Bin, Remain, PacketSz, FileOpTimeout) ->
%%
%% Description:
%%--------------------------------------------------------------------
-init([Cm, ChannelId, Timeout]) ->
+init([Cm, ChannelId, Options]) ->
+ Timeout = proplists:get_value(timeout, Options, infinity),
erlang:monitor(process, Cm),
case ssh_connection:subsystem(Cm, ChannelId, "sftp", Timeout) of
success ->
@@ -401,7 +405,8 @@ init([Cm, ChannelId, Timeout]) ->
{ok, #state{xf = Xf,
req_id = 0,
rep_buf = <<>>,
- inf = new_inf()}};
+ inf = new_inf(),
+ opts = Options}};
failure ->
{stop, "server failed to start sftp subsystem"};
Error ->
@@ -707,8 +712,9 @@ handle_ssh_msg({ssh_cm, _, {exit_status, ChannelId, Status}}, State0) ->
%%
%% Description: Handles channel messages
%%--------------------------------------------------------------------
-handle_msg({ssh_channel_up, _, _}, #state{xf = Xf} = State) ->
- ssh_xfer:protocol_version_request(Xf),
+handle_msg({ssh_channel_up, _, _}, #state{opts = Options, xf = Xf} = State) ->
+ Version = proplists:get_value(sftp_vsn, Options, ?SSH_SFTP_PROTOCOL_VERSION),
+ ssh_xfer:protocol_version_request(Xf, Version),
{ok, State};
%% Version negotiation timed out
@@ -754,6 +760,15 @@ terminate(_Reason, State) ->
%%====================================================================
%% Internal functions
%%====================================================================
+handle_options([], Sftp, Ssh) ->
+ {Ssh, Sftp};
+handle_options([{timeout, _} = Opt | Rest], Sftp, Ssh) ->
+ handle_options(Rest, [Opt | Sftp], Ssh);
+handle_options([{sftp_vsn, _} = Opt| Rest], Sftp, Ssh) ->
+ handle_options(Rest, [Opt | Sftp], Ssh);
+handle_options([Opt | Rest], Sftp, Ssh) ->
+ handle_options(Rest, Sftp, [Opt | Ssh]).
+
call(Pid, Msg, TimeOut) ->
ssh_channel:call(Pid, {{timeout, TimeOut}, Msg}, infinity).
diff --git a/lib/ssh/src/ssh_xfer.erl b/lib/ssh/src/ssh_xfer.erl
index 63d01fd9de..1881392db8 100644
--- a/lib/ssh/src/ssh_xfer.erl
+++ b/lib/ssh/src/ssh_xfer.erl
@@ -1,7 +1,7 @@
%%
%% %CopyrightBegin%
%%
-%% Copyright Ericsson AB 2005-2013. All Rights Reserved.
+%% Copyright Ericsson AB 2005-2014. All Rights Reserved.
%%
%% The contents of this file are subject to the Erlang Public License,
%% Version 1.1, (the "License"); you may not use this file except in
@@ -28,7 +28,7 @@
rename/5, remove/3, mkdir/4, rmdir/3, realpath/3, extended/4,
stat/4, fstat/4, lstat/4, setstat/4,
readlink/3, fsetstat/4, symlink/4,
- protocol_version_request/1,
+ protocol_version_request/2,
xf_reply/2,
xf_send_reply/3, xf_send_names/3, xf_send_name/4,
xf_send_status/3, xf_send_status/4, xf_send_status/5,
@@ -67,8 +67,8 @@ open_xfer(CM, Opts) ->
Error
end.
-protocol_version_request(XF) ->
- xf_request(XF, ?SSH_FXP_INIT, <<?UINT32(?SSH_SFTP_PROTOCOL_VERSION)>>).
+protocol_version_request(XF, Version) ->
+ xf_request(XF, ?SSH_FXP_INIT, <<?UINT32(Version)>>).
open(XF, ReqID, FileName, Access, Flags, Attrs) ->
Vsn = XF#ssh_xfer.vsn,
diff --git a/lib/ssh/test/property_test/ssh_eqc_client_server.erl b/lib/ssh/test/property_test/ssh_eqc_client_server.erl
index cf895ae85e..123b48412b 100644
--- a/lib/ssh/test/property_test/ssh_eqc_client_server.erl
+++ b/lib/ssh/test/property_test/ssh_eqc_client_server.erl
@@ -32,6 +32,10 @@
-else.
+%% Limit the testing time on CI server... this needs to be improved in % from total budget.
+-define(TESTINGTIME(Prop), eqc:testing_time(30,Prop)).
+
+
-include_lib("eqc/include/eqc.hrl").
-include_lib("eqc/include/eqc_statem.hrl").
-eqc_group_commands(true).
@@ -75,7 +79,9 @@
-define(SUBSYSTEMS, ["echo1", "echo2", "echo3", "echo4"]).
--define(SERVER_ADDRESS, { {127,1,1,1}, inet_port({127,1,1,1}) }).
+-define(SERVER_ADDRESS, { {127,1,0,choose(1,254)}, % IP
+ choose(1024,65535) % Port
+ }).
-define(SERVER_EXTRA_OPTIONS, [{parallel_login,bool()}] ).
@@ -97,7 +103,7 @@
%% To be called as eqc:quickcheck( ssh_eqc_client_server:prop_seq() ).
prop_seq() ->
- do_prop_seq(?SSH_DIR).
+ ?TESTINGTIME(do_prop_seq(?SSH_DIR)).
%% To be called from a common_test test suite
prop_seq(CT_Config) ->
@@ -105,9 +111,10 @@ prop_seq(CT_Config) ->
do_prop_seq(DataDir) ->
- ?FORALL(Cmds,commands(?MODULE, #state{data_dir=DataDir}),
+ setup_rsa(DataDir),
+ ?FORALL(Cmds,commands(?MODULE),
begin
- {H,Sf,Result} = run_commands(?MODULE,Cmds),
+ {H,Sf,Result} = run_commands(?MODULE,Cmds,[{data_dir,DataDir}]),
present_result(?MODULE, Cmds, {H,Sf,Result}, Result==ok)
end).
@@ -116,33 +123,35 @@ full_path(SSHdir, CT_Config) ->
SSHdir).
%%%----
prop_parallel() ->
- do_prop_parallel(?SSH_DIR).
+ ?TESTINGTIME(do_prop_parallel(?SSH_DIR)).
%% To be called from a common_test test suite
prop_parallel(CT_Config) ->
do_prop_parallel(full_path(?SSH_DIR, CT_Config)).
do_prop_parallel(DataDir) ->
- ?FORALL(Cmds,parallel_commands(?MODULE, #state{data_dir=DataDir}),
+ setup_rsa(DataDir),
+ ?FORALL(Cmds,parallel_commands(?MODULE),
begin
- {H,Sf,Result} = run_parallel_commands(?MODULE,Cmds),
+ {H,Sf,Result} = run_parallel_commands(?MODULE,Cmds,[{data_dir,DataDir}]),
present_result(?MODULE, Cmds, {H,Sf,Result}, Result==ok)
end).
%%%----
prop_parallel_multi() ->
- do_prop_parallel_multi(?SSH_DIR).
+ ?TESTINGTIME(do_prop_parallel_multi(?SSH_DIR)).
%% To be called from a common_test test suite
prop_parallel_multi(CT_Config) ->
do_prop_parallel_multi(full_path(?SSH_DIR, CT_Config)).
do_prop_parallel_multi(DataDir) ->
+ setup_rsa(DataDir),
?FORALL(Repetitions,?SHRINK(1,[10]),
- ?FORALL(Cmds,parallel_commands(?MODULE, #state{data_dir=DataDir}),
+ ?FORALL(Cmds,parallel_commands(?MODULE),
?ALWAYS(Repetitions,
begin
- {H,Sf,Result} = run_parallel_commands(?MODULE,Cmds),
+ {H,Sf,Result} = run_parallel_commands(?MODULE,Cmds,[{data_dir,DataDir}]),
present_result(?MODULE, Cmds, {H,Sf,Result}, Result==ok)
end))).
@@ -151,14 +160,12 @@ do_prop_parallel_multi(DataDir) ->
%%% called when using commands/1
initial_state() ->
- S = initial_state(#state{}),
- S#state{initialized=true}.
+ #state{}.
%%% called when using commands/2
-initial_state(S) ->
+initial_state(DataDir) ->
application:stop(ssh),
- ssh:start(),
- setup_rsa(S#state.data_dir).
+ ssh:start().
%%%----------------
weight(S, ssh_send) -> 5*length([C || C<-S#state.channels, has_subsyst(C)]);
@@ -172,7 +179,7 @@ weight(_S, _) -> 1.
initial_state_pre(S) -> not S#state.initialized.
-initial_state_args(S) -> [S].
+initial_state_args(_) -> [{var,data_dir}].
initial_state_next(S, _, _) -> S#state{initialized=true}.
@@ -180,10 +187,17 @@ initial_state_next(S, _, _) -> S#state{initialized=true}.
%%% Start a new daemon
%%% Precondition: not more than ?MAX_NUM_SERVERS started
+%%% This is a bit funny because we need to pick an IP address and Port to
+%%% run the server on, but there is no way to atomically select a free Port!
+%%%
+%%% Therefore we just grab one IP-Port pair randomly and try to start the ssh server
+%%% on that pair. If it fails, we just forget about it and goes on. Yes, it
+%%% is a waste of cpu cycles, but at least it works!
+
ssh_server_pre(S) -> S#state.initialized andalso
length(S#state.servers) < ?MAX_NUM_SERVERS.
-ssh_server_args(S) -> [?SERVER_ADDRESS, S#state.data_dir, ?SERVER_EXTRA_OPTIONS].
+ssh_server_args(_) -> [?SERVER_ADDRESS, {var,data_dir}, ?SERVER_EXTRA_OPTIONS].
ssh_server({IP,Port}, DataDir, ExtraOptions) ->
ok(ssh:daemon(IP, Port,
@@ -194,8 +208,10 @@ ssh_server({IP,Port}, DataDir, ExtraOptions) ->
| ExtraOptions
])).
+ssh_server_post(_S, _Args, {error,eaddrinuse}) -> true;
ssh_server_post(_S, _Args, Result) -> is_ok(Result).
+ssh_server_next(S, {error,eaddrinuse}, _) -> S;
ssh_server_next(S, Result, [{IP,Port},_,_]) ->
S#state{servers=[#srvr{ref = Result,
address = IP,
@@ -241,15 +257,16 @@ do(Pid, Fun, Timeout) when is_function(Fun,0) ->
ssh_open_connection_pre(S) -> S#state.servers /= [].
-ssh_open_connection_args(S) -> [oneof(S#state.servers), S#state.data_dir].
+ssh_open_connection_args(S) -> [oneof(S#state.servers), {var,data_dir}].
ssh_open_connection(#srvr{address=Ip, port=Port}, DataDir) ->
ok(ssh:connect(ensure_string(Ip), Port,
[
{silently_accept_hosts, true},
{user_dir, user_dir(DataDir)},
- {user_interaction, false}
- ])).
+ {user_interaction, false},
+ {connect_timeout, 2000}
+ ], 2000)).
ssh_open_connection_post(_S, _Args, Result) -> is_ok(Result).
@@ -569,12 +586,6 @@ median(_) ->
%%%================================================================
%%% The rest is taken and modified from ssh_test_lib.erl
-inet_port(IpAddress)->
- {ok, Socket} = gen_tcp:listen(0, [{ip,IpAddress},{reuseaddr,true}]),
- {ok, Port} = inet:port(Socket),
- gen_tcp:close(Socket),
- Port.
-
setup_rsa(Dir) ->
erase_dir(system_dir(Dir)),
erase_dir(user_dir(Dir)),
diff --git a/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl b/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl
index 34630bdc91..57ea2012c1 100644
--- a/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl
+++ b/lib/ssh/test/property_test/ssh_eqc_encode_decode.erl
@@ -25,8 +25,6 @@
-proptest(eqc).
-proptest([triq,proper]).
--include_lib("ct_property_test.hrl").
-
-ifndef(EQC).
-ifndef(PROPER).
-ifndef(TRIQ).
diff --git a/lib/ssh/test/ssh_connection_SUITE.erl b/lib/ssh/test/ssh_connection_SUITE.erl
index d226e5ba03..3c537d719c 100644
--- a/lib/ssh/test/ssh_connection_SUITE.erl
+++ b/lib/ssh/test/ssh_connection_SUITE.erl
@@ -37,6 +37,7 @@
all() ->
[
{group, openssh_payload},
+ start_subsystem_on_closed_channel,
interrupted_send,
start_shell,
start_shell_exec,
@@ -241,6 +242,32 @@ send_after_exit(Config) when is_list(Config) ->
end.
%%--------------------------------------------------------------------
+start_subsystem_on_closed_channel(Config) ->
+ PrivDir = ?config(priv_dir, Config),
+ UserDir = filename:join(PrivDir, nopubkey), % to make sure we don't use public-key-auth
+ file:make_dir(UserDir),
+ SysDir = ?config(data_dir, Config),
+ {Pid, Host, Port} = ssh_test_lib:daemon([{system_dir, SysDir},
+ {user_dir, UserDir},
+ {password, "morot"},
+ {subsystems, [{"echo_n", {ssh_echo_server, [4000000]}}]}]),
+
+ ConnectionRef = ssh_test_lib:connect(Host, Port, [{silently_accept_hosts, true},
+ {user, "foo"},
+ {password, "morot"},
+ {user_interaction, false},
+ {user_dir, UserDir}]),
+
+ {ok, ChannelId} = ssh_connection:session_channel(ConnectionRef, infinity),
+
+ ok = ssh_connection:close(ConnectionRef, ChannelId),
+
+ failure = ssh_connection:subsystem(ConnectionRef, ChannelId, "echo_n", infinity),
+
+ ssh:close(ConnectionRef),
+ ssh:stop_daemon(Pid).
+
+%%--------------------------------------------------------------------
interrupted_send() ->
[{doc, "Use a subsystem that echos n char and then sends eof to cause a channel exit partway through a large send."}].
diff --git a/lib/ssh/test/ssh_sftp_SUITE.erl b/lib/ssh/test/ssh_sftp_SUITE.erl
index 56b1363b7a..4c46a1b1a8 100644
--- a/lib/ssh/test/ssh_sftp_SUITE.erl
+++ b/lib/ssh/test/ssh_sftp_SUITE.erl
@@ -1,7 +1,7 @@
%%
%% %CopyrightBegin%
%%
-%% Copyright Ericsson AB 2005-2013. All Rights Reserved.
+%% Copyright Ericsson AB 2005-2014. All Rights Reserved.
%%
%% The contents of this file are subject to the Erlang Public License,
%% Version 1.1, (the "License"); you may not use this file except in
@@ -65,7 +65,7 @@ groups() ->
[{erlang_server, [], [open_close_file, open_close_dir, read_file, read_dir,
write_file, rename_file, mk_rm_dir, remove_file, links,
retrieve_attributes, set_attributes, async_read,
- async_write, position, pos_read, pos_write]},
+ async_write, position, pos_read, pos_write, version_option]},
{openssh_server, [], [open_close_file, open_close_dir, read_file, read_dir,
write_file, rename_file, mk_rm_dir, remove_file, links,
retrieve_attributes, set_attributes, async_read,
@@ -111,6 +111,21 @@ init_per_testcase(sftp_nonexistent_subsystem, Config) ->
]),
[{sftpd, Sftpd} | Config];
+init_per_testcase(version_option, Config) ->
+ prep(Config),
+ TmpConfig0 = lists:keydelete(watchdog, 1, Config),
+ TmpConfig = lists:keydelete(sftp, 1, TmpConfig0),
+ Dog = ct:timetrap(?default_timeout),
+ {_,Host, Port} = ?config(sftpd, Config),
+ {ok, ChannelPid, Connection} =
+ ssh_sftp:start_channel(Host, Port,
+ [{sftp_vsn, 3},
+ {user, ?USER},
+ {password, ?PASSWD},
+ {user_interaction, false},
+ {silently_accept_hosts, true}]),
+ Sftp = {ChannelPid, Connection},
+ [{sftp, Sftp}, {watchdog, Dog} | TmpConfig];
init_per_testcase(Case, Config) ->
prep(Config),
TmpConfig0 = lists:keydelete(watchdog, 1, Config),
@@ -447,6 +462,11 @@ sftp_nonexistent_subsystem(Config) when is_list(Config) ->
{silently_accept_hosts, true}]).
%%--------------------------------------------------------------------
+version_option() ->
+ [{doc, "Test API option sftp_vsn"}].
+version_option(Config) when is_list(Config) ->
+ open_close_dir(Config).
+%%--------------------------------------------------------------------
%% Internal functions ------------------------------------------------
%%--------------------------------------------------------------------
prep(Config) ->
diff --git a/lib/ssh/vsn.mk b/lib/ssh/vsn.mk
index 11f30e8d04..68544c1d0e 100644
--- a/lib/ssh/vsn.mk
+++ b/lib/ssh/vsn.mk
@@ -1,5 +1,5 @@
#-*-makefile-*- ; force emacs to enter makefile-mode
-SSH_VSN = 3.0.6
+SSH_VSN = 3.0.8
APP_VSN = "ssh-$(SSH_VSN)"
diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml
index 8643cd3745..62e9bd0165 100644
--- a/lib/ssl/doc/src/notes.xml
+++ b/lib/ssl/doc/src/notes.xml
@@ -25,7 +25,23 @@
<file>notes.xml</file>
</header>
<p>This document describes the changes made to the SSL application.</p>
- <section><title>SSL 5.3.6</title>
+ <section><title>SSL 5.3.7</title>
+
+ <section><title>Fixed Bugs and Malfunctions</title>
+ <list>
+ <item>
+ <p>
+ Handle the fact that servers may send an empty SNI
+ extension to the client.</p>
+ <p>
+ Own Id: OTP-12198</p>
+ </item>
+ </list>
+ </section>
+
+</section>
+
+<section><title>SSL 5.3.6</title>
<section><title>Fixed Bugs and Malfunctions</title>
<list>
diff --git a/lib/ssl/src/ssl.appup.src b/lib/ssl/src/ssl.appup.src
index 650901ef54..9d692379b4 100644
--- a/lib/ssl/src/ssl.appup.src
+++ b/lib/ssl/src/ssl.appup.src
@@ -1,6 +1,7 @@
%% -*- erlang -*-
{"%VSN%",
[
+ {"5.3.6", [{load_module, ssl_handshake, soft_purge, soft_purge, [ssl_connection]}]},
{"5.3.5", [{load_module, ssl, soft_purge, soft_purge, [ssl_connection]},
{load_module, ssl_handshake, soft_purge, soft_purge, [ssl_certificate]},
{load_module, ssl_certificate, soft_purge, soft_purge, []},
@@ -12,6 +13,7 @@
{<<"3\\..*">>, [{restart_application, ssl}]}
],
[
+ {"5.3.6", [{load_module, ssl_handshake, soft_purge, soft_purge, [ssl_connection]}]},
{"5.3.5", [{load_module, ssl, soft_purge, soft_purge,[ssl_certificate]},
{load_module, ssl_handshake, soft_purge, soft_purge,[ssl_certificate]},
{load_module, ssl_certificate, soft_purge, soft_purge,[]},
diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk
index 404b71374f..da20ed8593 100644
--- a/lib/ssl/vsn.mk
+++ b/lib/ssl/vsn.mk
@@ -1 +1 @@
-SSL_VSN = 5.3.6
+SSL_VSN = 5.3.7
diff --git a/lib/stdlib/src/dets.erl b/lib/stdlib/src/dets.erl
index 76e03bbfaa..a4bd45ea19 100644
--- a/lib/stdlib/src/dets.erl
+++ b/lib/stdlib/src/dets.erl
@@ -2839,17 +2839,22 @@ fsck_try(Fd, Tab, FH, Fname, SlotNumbers, Version) ->
tempfile(Fname) ->
Tmp = lists:concat([Fname, ".TMP"]),
- tempfile(Tmp, 10).
-
-tempfile(Tmp, 0) ->
- Tmp;
-tempfile(Tmp, N) ->
case file:delete(Tmp) of
- {error, eacces} -> % 'dets_process_died' happened anyway... (W-nd-ws)
- timer:sleep(1000),
- tempfile(Tmp, N-1);
- _ ->
- Tmp
+ {error, _Reason} -> % typically enoent
+ ok;
+ ok ->
+ assure_no_file(Tmp)
+ end,
+ Tmp.
+
+assure_no_file(File) ->
+ case file:read_file_info(File) of
+ {ok, _FileInfo} ->
+ %% Wait for some other process to close the file:
+ timer:sleep(100),
+ assure_no_file(File);
+ {error, _} ->
+ ok
end.
%% -> {ok, NewHead} | {try_again, integer()} | Error
diff --git a/lib/stdlib/src/dets_server.erl b/lib/stdlib/src/dets_server.erl
index 268c201047..3164d40f35 100644
--- a/lib/stdlib/src/dets_server.erl
+++ b/lib/stdlib/src/dets_server.erl
@@ -1,7 +1,7 @@
%%
%% %CopyrightBegin%
%%
-%% Copyright Ericsson AB 2001-2013. All Rights Reserved.
+%% Copyright Ericsson AB 2001-2014. All Rights Reserved.
%%
%% The contents of this file are subject to the Erlang Public License,
%% Version 1.1, (the "License"); you may not use this file except in
@@ -171,9 +171,15 @@ handle_info({pending_reply, {Ref, Result0}}, State) ->
link(Pid),
do_link(Store, FromPid),
true = ets:insert(Store, {FromPid, Tab}),
- true = ets:insert(?REGISTRY, {Tab, 1, Pid}),
- true = ets:insert(?OWNERS, {Pid, Tab}),
+ %% do_internal_open() has already done the following:
+ %% true = ets:insert(?REGISTRY, {Tab, 1, Pid}),
+ %% true = ets:insert(?OWNERS, {Pid, Tab}),
{ok, Tab};
+ {Reply, internal_open} ->
+ %% Clean up what do_internal_open() did:
+ true = ets:delete(?REGISTRY, Tab),
+ true = ets:delete(?OWNERS, Pid),
+ Reply;
{Reply, _} -> % ok or Error
Reply
end,
@@ -309,6 +315,12 @@ do_internal_open(State, From, Args) ->
[T, _, _] -> T;
[_, _] -> Ref
end,
+ %% Pretend the table is open. If someone else tries to
+ %% open the file it will always become a pending
+ %% 'add_user' request. If someone tries to use the table
+ %% there will be a delay, but that is OK.
+ true = ets:insert(?REGISTRY, {Tab, 1, Pid}),
+ true = ets:insert(?OWNERS, {Pid, Tab}),
pending_call(Tab, Pid, Ref, From, Args, internal_open, State);
Error ->
{Error, State}
diff --git a/lib/stdlib/test/dets_SUITE.erl b/lib/stdlib/test/dets_SUITE.erl
index 119b4dc7cb..3b08ac165e 100644
--- a/lib/stdlib/test/dets_SUITE.erl
+++ b/lib/stdlib/test/dets_SUITE.erl
@@ -223,8 +223,7 @@ open(Config, Version) ->
?format("Crashing dets server \n", []),
process_flag(trap_exit, true),
- Procs = [whereis(?DETS_SERVER) | map(fun(Tab) -> dets:info(Tab, pid) end,
- Tabs)],
+ Procs = [whereis(?DETS_SERVER) | [dets:info(Tab, pid) || Tab <- Tabs]],
foreach(fun(Pid) -> exit(Pid, kill) end, Procs),
timer:sleep(100),
c:flush(), %% flush all the EXIT sigs
@@ -235,18 +234,32 @@ open(Config, Version) ->
open_files(1, All, Version),
?format("Checking contents of repaired files \n", []),
check(Tabs, Data),
-
- close_all(Tabs),
+ close_all(Tabs),
delete_files(All),
- P1 = pps(),
+
{Ports0, Procs0} = P0,
- {Ports1, Procs1} = P1,
- true = Ports1 =:= Ports0,
- %% The dets_server process has been restarted:
- [_] = Procs0 -- Procs1,
- [_] = Procs1 -- Procs0,
- ok.
+ Test = fun() ->
+ P1 = pps(),
+ {Ports1, Procs1} = P1,
+ show("Old port", Ports0 -- Ports1),
+ show("New port", Ports1 -- Ports0),
+ show("Old procs", Procs0 -- Procs1),
+ show("New procs", Procs1 -- Procs0),
+ io:format("Remaining Dets-pids (should be nil): ~p~n",
+ [find_dets_pids()]),
+ true = Ports1 =:= Ports0,
+ %% The dets_server process has been restarted:
+ [_] = Procs0 -- Procs1,
+ [_] = Procs1 -- Procs0,
+ ok
+ end,
+ case catch Test() of
+ ok -> ok;
+ _ ->
+ timer:sleep(500),
+ ok = Test()
+ end.
check(Tabs, Data) ->
foreach(fun(Tab) ->
@@ -3275,12 +3288,22 @@ simultaneous_open(Config) ->
File = filename(Tab, Config),
ok = monit(Tab, File),
- ok = kill_while_repairing(Tab, File),
- ok = kill_while_init(Tab, File),
- ok = open_ro(Tab, File),
- ok = open_w(Tab, File, 0, Config),
- ok = open_w(Tab, File, 100, Config),
- ok.
+ case feasible() of
+ false -> {comment, "OK, but did not run all of the test"};
+ true ->
+ ok = kill_while_repairing(Tab, File),
+ ok = kill_while_init(Tab, File),
+ ok = open_ro(Tab, File),
+ ok = open_w(Tab, File, 0, Config),
+ ok = open_w(Tab, File, 100, Config)
+ end.
+
+feasible() ->
+ LP = erlang:system_info(logical_processors),
+ (is_integer(LP)
+ andalso LP >= erlang:system_info(schedulers_online)
+ andalso not erlang:system_info(debug_compiled)
+ andalso not erlang:system_info(lock_checking)).
%% One process logs and another process closes the log. Before
%% monitors were used, this would make the client never return.
@@ -3307,7 +3330,6 @@ kill_while_repairing(Tab, File) ->
Delay = 1000,
dets:start(),
Parent = self(),
- Ps = processes(),
F = fun() ->
R = (catch dets:open_file(Tab, [{file,File}])),
timer:sleep(Delay),
@@ -3318,7 +3340,7 @@ kill_while_repairing(Tab, File) ->
P1 = spawn(F),
P2 = spawn(F),
P3 = spawn(F),
- DetsPid = find_dets_pid([P1, P2, P3 | Ps]),
+ DetsPid = find_dets_pid(),
exit(DetsPid, kill),
receive {P1,R1} -> R1 end,
@@ -3342,12 +3364,6 @@ kill_while_repairing(Tab, File) ->
file:delete(File),
ok.
-find_dets_pid(P0) ->
- case lists:sort(processes() -- P0) of
- [P, _] -> P;
- _ -> timer:sleep(100), find_dets_pid(P0)
- end.
-
find_dets_pid() ->
case find_dets_pids() of
[] ->
@@ -3421,6 +3437,13 @@ open_ro(Tab, File) ->
open_w(Tab, File, Delay, Config) ->
create_opened_log(File),
+
+ Tab2 = t2,
+ File2 = filename(Tab2, Config),
+ file:delete(File2),
+ {ok,Tab2} = dets:open_file(Tab2, [{file,File2}]),
+ ok = dets:close(Tab2),
+
Parent = self(),
F = fun() ->
R = dets:open_file(Tab, [{file,File}]),
@@ -3430,16 +3453,16 @@ open_w(Tab, File, Delay, Config) ->
Pid1 = spawn(F),
Pid2 = spawn(F),
Pid3 = spawn(F),
- undefined = dets:info(Tab), % is repairing now
- 0 = qlen(),
- Tab2 = t2,
- File2 = filename(Tab2, Config),
- file:delete(File2),
+ ok = wait_for_repair_to_start(Tab),
+
+ %% It is assumed that it takes some time to repair the file.
{ok,Tab2} = dets:open_file(Tab2, [{file,File2}]),
+ %% The Dets server managed to handle to open_file request.
+ 0 = qlen(), % still repairing
+
ok = dets:close(Tab2),
file:delete(File2),
- 0 = qlen(), % still repairing
receive {Pid1,R1} -> {ok, Tab} = R1 end,
receive {Pid2,R2} -> {ok, Tab} = R2 end,
@@ -3456,6 +3479,15 @@ open_w(Tab, File, Delay, Config) ->
file:delete(File),
ok.
+wait_for_repair_to_start(Tab) ->
+ case catch dets_server:get_pid(Tab) of
+ {'EXIT', _} ->
+ timer:sleep(1),
+ wait_for_repair_to_start(Tab);
+ Pid when is_pid(Pid) ->
+ ok
+ end.
+
qlen() ->
{_, {_, N}} = lists:keysearch(message_queue_len, 1, process_info(self())),
N.
@@ -4350,6 +4382,7 @@ check_badarg({'EXIT', {badarg, [{M,F,A,_} | _]}}, M, F, Args) ->
true = test_server:is_native(M) andalso length(Args) =:= A.
check_pps({Ports0,Procs0} = P0) ->
+ ok = check_dets_tables(),
case pps() of
P0 ->
ok;
@@ -4375,13 +4408,45 @@ check_pps({Ports0,Procs0} = P0) ->
end
end.
+%% Copied from dets_server.erl:
+-define(REGISTRY, dets_registry).
+-define(OWNERS, dets_owners).
+-define(STORE, dets).
+
+check_dets_tables() ->
+ Store = [T ||
+ T <- ets:all(),
+ ets:info(T, name) =:= ?STORE,
+ owner(T) =:= dets],
+ S = case Store of
+ [Tab] -> ets:tab2list(Tab);
+ [] -> []
+ end,
+ case {ets:tab2list(?REGISTRY), ets:tab2list(?OWNERS), S} of
+ {[], [], []} -> ok;
+ {R, O, _} ->
+ io:format("Registry: ~p~n", [R]),
+ io:format("Owners: ~p~n", [O]),
+ io:format("Store: ~p~n", [S]),
+ not_ok
+ end.
+
+owner(Tab) ->
+ Owner = ets:info(Tab, owner),
+ case process_info(Owner, registered_name) of
+ {registered_name, Name} -> Name;
+ _ -> Owner
+ end.
+
show(_S, []) ->
ok;
-show(S, [Pid|Pids]) when is_pid(Pid) ->
- io:format("~s: ~p~n", [S, erlang:process_info(Pid)]),
+show(S, [{Pid, Name, InitCall}|Pids]) when is_pid(Pid) ->
+ io:format("~s: ~w (~w), ~w: ~p~n",
+ [S, Pid, proc_reg_name(Name), InitCall,
+ erlang:process_info(Pid)]),
show(S, Pids);
-show(S, [Port|Ports]) when is_port(Port)->
- io:format("~s: ~p~n", [S, erlang:port_info(Port)]),
+show(S, [{Port, _}|Ports]) when is_port(Port)->
+ io:format("~s: ~w: ~p~n", [S, Port, erlang:port_info(Port)]),
show(S, Ports).
pps() ->
@@ -4397,5 +4462,8 @@ process_list() ->
safe_second_element(process_info(P, initial_call))} ||
P <- processes()].
+proc_reg_name({registered_name, Name}) -> Name;
+proc_reg_name([]) -> no_reg_name.
+
safe_second_element({_,Info}) -> Info;
safe_second_element(Other) -> Other.
diff --git a/otp_versions.table b/otp_versions.table
index da1f7225a8..4da3e13559 100644
--- a/otp_versions.table
+++ b/otp_versions.table
@@ -1,3 +1,5 @@
+OTP-17.3.3 : ssh-3.0.8 # asn1-3.0.2 common_test-1.8.2 compiler-5.0.2 cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 crypto-3.4.1 debugger-4.0.1 dialyzer-2.7.2 diameter-1.7.1 edoc-0.7.15 eldap-1.0.4 erl_docgen-0.3.6 erl_interface-3.7.19 erts-6.2 et-1.5 eunit-2.2.8 gs-1.5.16 hipe-3.11.1 ic-4.3.6 inets-5.10.3 jinterface-1.5.11 kernel-3.0.3 megaco-3.17.2 mnesia-4.12.3 observer-2.0.2 odbc-2.10.21 orber-3.7.1 os_mon-2.3 ose-1.0.2 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 public_key-0.22.1 reltool-0.6.6 runtime_tools-1.8.14 sasl-2.4.1 snmp-5.1 ssl-5.3.7 stdlib-2.2 syntax_tools-1.6.16 test_server-3.7.1 tools-2.7 typer-0.9.8 webtool-0.8.10 wx-1.3.1 xmerl-1.3.7 :
+OTP-17.3.2 : ssh-3.0.7 ssl-5.3.7 # asn1-3.0.2 common_test-1.8.2 compiler-5.0.2 cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 crypto-3.4.1 debugger-4.0.1 dialyzer-2.7.2 diameter-1.7.1 edoc-0.7.15 eldap-1.0.4 erl_docgen-0.3.6 erl_interface-3.7.19 erts-6.2 et-1.5 eunit-2.2.8 gs-1.5.16 hipe-3.11.1 ic-4.3.6 inets-5.10.3 jinterface-1.5.11 kernel-3.0.3 megaco-3.17.2 mnesia-4.12.3 observer-2.0.2 odbc-2.10.21 orber-3.7.1 os_mon-2.3 ose-1.0.2 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 public_key-0.22.1 reltool-0.6.6 runtime_tools-1.8.14 sasl-2.4.1 snmp-5.1 stdlib-2.2 syntax_tools-1.6.16 test_server-3.7.1 tools-2.7 typer-0.9.8 webtool-0.8.10 wx-1.3.1 xmerl-1.3.7 :
OTP-17.3.1 : eldap-1.0.4 erl_interface-3.7.19 jinterface-1.5.11 orber-3.7.1 ose-1.0.2 ssh-3.0.6 # asn1-3.0.2 common_test-1.8.2 compiler-5.0.2 cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 crypto-3.4.1 debugger-4.0.1 dialyzer-2.7.2 diameter-1.7.1 edoc-0.7.15 erl_docgen-0.3.6 erts-6.2 et-1.5 eunit-2.2.8 gs-1.5.16 hipe-3.11.1 ic-4.3.6 inets-5.10.3 kernel-3.0.3 megaco-3.17.2 mnesia-4.12.3 observer-2.0.2 odbc-2.10.21 os_mon-2.3 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 public_key-0.22.1 reltool-0.6.6 runtime_tools-1.8.14 sasl-2.4.1 snmp-5.1 ssl-5.3.6 stdlib-2.2 syntax_tools-1.6.16 test_server-3.7.1 tools-2.7 typer-0.9.8 webtool-0.8.10 wx-1.3.1 xmerl-1.3.7 :
OTP-17.3 : asn1-3.0.2 common_test-1.8.2 compiler-5.0.2 crypto-3.4.1 dialyzer-2.7.2 diameter-1.7.1 edoc-0.7.15 erl_docgen-0.3.6 erl_interface-3.7.18 erts-6.2 eunit-2.2.8 hipe-3.11.1 ic-4.3.6 inets-5.10.3 jinterface-1.5.10 kernel-3.0.3 megaco-3.17.2 mnesia-4.12.3 observer-2.0.2 odbc-2.10.21 os_mon-2.3 ose-1.0.1 public_key-0.22.1 sasl-2.4.1 snmp-5.1 ssh-3.0.5 ssl-5.3.6 stdlib-2.2 tools-2.7 wx-1.3.1 # cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 debugger-4.0.1 eldap-1.0.3 et-1.5 gs-1.5.16 orber-3.7 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 reltool-0.6.6 runtime_tools-1.8.14 syntax_tools-1.6.16 test_server-3.7.1 typer-0.9.8 webtool-0.8.10 xmerl-1.3.7 :
OTP-17.2.2 : mnesia-4.12.2 # asn1-3.0.1 common_test-1.8.1 compiler-5.0.1 cosEvent-2.1.15 cosEventDomain-1.1.14 cosFileTransfer-1.1.16 cosNotification-1.1.21 cosProperty-1.1.17 cosTime-1.1.14 cosTransactions-1.2.14 crypto-3.4 debugger-4.0.1 dialyzer-2.7.1 diameter-1.7 edoc-0.7.14 eldap-1.0.3 erl_docgen-0.3.5 erl_interface-3.7.17 erts-6.1.2 et-1.5 eunit-2.2.7 gs-1.5.16 hipe-3.11 ic-4.3.5 inets-5.10.2 jinterface-1.5.9 kernel-3.0.2 megaco-3.17.1 observer-2.0.1 odbc-2.10.20 orber-3.7 os_mon-2.2.15 ose-1.0 otp_mibs-1.0.9 parsetools-2.0.11 percept-0.8.9 public_key-0.22 reltool-0.6.6 runtime_tools-1.8.14 sasl-2.4 snmp-5.0 ssh-3.0.4 ssl-5.3.5 stdlib-2.1.1 syntax_tools-1.6.16 test_server-3.7.1 tools-2.6.15 typer-0.9.8 webtool-0.8.10 wx-1.3 xmerl-1.3.7 :