diff options
131 files changed, 2962 insertions, 886 deletions
diff --git a/INSTALL-WIN32.md b/INSTALL-WIN32.md index 6481ca1b06..59b9086c39 100644 --- a/INSTALL-WIN32.md +++ b/INSTALL-WIN32.md @@ -92,7 +92,9 @@ Frequently Asked Questions A: The SMP version of Erlang needs features in the Visual Studio 2005. Can't live without them. Besides the new compiler gives the Erlang - emulator a ~40% performance boost(!) + emulator a ~40% performance boost(!). Alternatively you can build Erlang + successfully using the free (proprietary) Visual Studio 2008 Express + edition C++ compiler. * Q: Can/will I build a Cygwin binary with the procedure you describe? @@ -208,20 +210,15 @@ Frequently Asked Questions * (Buy and) Install Microsoft Visual studio 2005 and SP1 (or higher) - * Get and install Sun's JDK 1.4.2 + * Alternatively install the free MS Visual Studio 2008 Express [msvc++] + and the Windows SDK [32bit-SDK] or [64bit-SDK] depending on the Windows + platform you are running. - * Get and install NSIS 2.01 or higher (up to 2.30 tried and working) + * Get and install Sun's JDK 1.4.2 - * Get and install OpenSSL 0.9.7c or higher + * Get and install NSIS 2.01 or higher (up to 2.46 tried and working) - * Get and unpack wxWidgets-2.8.9 or higher to `/opt/local/pgm` inside - cygwin. - * Open `/cygwin/opt/local/pgm/wxWidgets-2.8.9/build/msw/wx.dsw` - * Enable `wxUSE_GLCANVAS`, `wxUSE_POSTSCRIPT` and - `wxUSE_GRAPHICS_CONTEXT` in `include/wx/msw/setup.h` - * Build all unicode release (and unicode debug) packages - * Open `/cygwin/opt/local/pgm/wxWidgets-2.8.9/contrib/build/stc/stc.dsw` - * Build the unicode release (and unicode debug) packages + * Get and install OpenSSL 0.9.7c or higher (up to 1.0.0a tried & working) * Get the Erlang source distribution (from <http://www.erlang.org/download.html>) and unpack with Cygwin's `tar`. @@ -363,6 +360,15 @@ Well' here's the list: your `PATH` to allow the environment to find mc.exe. The next Visual Studio (2010) is expected to include this tool. + Alternatively install the free MS Visual Studio 2008 Express [msvc++] and + the Windows SDK [32bit-SDK] or [64bit-SDK] depending on the Windows + platform you are running, which includes the missing mc.exe message + compiler. + +[msvc++]: http://download.microsoft.com/download/E/8/E/E8EEB394-7F42-4963-A2D8-29559B738298/VS2008ExpressWithSP1ENUX1504728.iso +[32bit-SDK]: http://download.microsoft.com/download/2/E/9/2E911956-F90F-4BFB-8231-E292A7B6F287/GRMSDK_EN_DVD.iso +[64bit-SDK]: http://download.microsoft.com/download/2/E/9/2E911956-F90F-4BFB-8231-E292A7B6F287/GRMSDKX_EN_DVD.iso + * Sun's Java JDK 1.5.0 or higher. Our Java code (jinterface, ic) is written for JDK 1.5.0. Get it for Windows and install it, the JRE is not enough. If you don't care about Java, you can skip this step, the @@ -401,9 +407,10 @@ Well' here's the list: on the `Related` link and then on the `Binaries` link (upper right corner of the page last time I looked), you can then reach the "Shining Lights Productions" Web site for Windows binaries - distributions. Get the latest or 0.9.7c if you get trouble with the - latest. It's a nifty installer. The rest should be handled by - `configure`, you needn't put anything in the path or anything. + distributions. Get the latest 32-bit installer, or use 0.9.7c if you get + trouble with the latest, and install to C:\OpenSSL which is where the + Makefiles are expecting to find it. It's a nifty installer. The rest should + be handled by `configure`, you needn't put anything in the path or anything. If you want to build openssl for windows yourself (which might be possible, as you wouldn't be reading this if you weren't a @@ -422,18 +429,51 @@ Well' here's the list: release (2.9.\* is a developer release which currently does not work with wxErlang). - Install or unpack it to `DRIVE:/PATH/cygwin/opt/local/pgm` + Install or unpack it to `DRIVE:/PATH/cygwin/opt/local/pgm`. Open from explorer (i.e. by double clicking the file) - `C:\cygwin\opt\local\pgm\wxMSW-2.8.10\build\msw\wx.dsw` + `C:\cygwin\opt\local\pgm\wxMSW-2.8.11\build\msw\wx.dsw` In Microsoft Visual Studio, click File/Open/File, locate and - open: `C:\cygwin\opt\local\pgm\wxMSW-2.8.10\include\wx\msw\setup.h` + open: `C:\cygwin\opt\local\pgm\wxMSW-2.8.11\include\wx\msw\setup.h` enable `wxUSE_GLCANVAS`, `wxUSE_POSTSCRIPT` and `wxUSE_GRAPHICS_CONTEXT` Build it by clicking Build/Batch Build and select all unicode release (and unicode debug) packages. - Open `C:\cygwin\opt\local\pgm\wxMSW-2.8.10\contrib/build/stc/stc.dsw` + Open `C:\cygwin\opt\local\pgm\wxMSW-2.8.11\contrib/build/stc/stc.dsw` and batch build all unicode packages. + If you are using Visual C++ 9.0 or higher (Visual Studio 2008 onwards) you + will also need to convert and re-create the project dependencies in the new + .sln "Solution" format. + + * Open VSC++ & the project `wxMSW-2.8.11\build\msw\wx.dsw`, accepting the + automatic conversion to the newer VC++ format and save as + `\wxMSW-2.8.11\build\msw\wx.sln` + + * right-click on the project, and set up the project dependencies for + `wx.dsw` to achieve the below build order + + jpeg, png, tiff, zlib, regex, expat, base, net, odbc, core, + gl, html, media, qa, adv, dbgrid, xrc, aui, richtext, xml + + Build all unicode release (and unicode debug) packages either from the + GUI or alternatively launch a new prompt from somewhere like Start -> + Programs -> Microsoft Visual C++ -> Visual Studio Tools -> VS2008 Cmd Prompt + and cd to where you unpacked wxMSW + + pushd c:\wxMSW*\build\msw + vcbuild /useenv /platform:Win32 /M4 wx.sln "Unicode Release|Win32" + vcbuild /useenv /platform:Win32 /M4 wx.sln "Unicode Debug|Win32" + + Open VSC++ & convert `C:\wxMSW-2.8.11\contrib\build\stc\stc.dsw` to + `C:\wxMSW-2.8.11\contrib\build\stc\stc.sln` + + * build the unicode release (and unicode debug) packages from the GUI or + alternatively open a VS2008 Cmd Prompt and cd to where you unpacked wxMSW + + pushd c:\wxMSW*\contrib\build\stc + vcbuild /useenv /platform:Win32 /M4 stc.sln "Unicode Release|Win32" + vcbuild /useenv /platform:Win32 /M4 stc.sln "Unicode Debug|Win32" + * The Erlang source distribution (from <http://www.erlang.org/download.html>). The same as for Unix platforms. Preferably use tar from within Cygwin to unpack the source tar.gz (`tar zxf otp_src_%OTP-REL%.tar.gz`). diff --git a/bootstrap/bin/start.script b/bootstrap/bin/start.script index 3f472b5e8c..6ca908f22e 100644 --- a/bootstrap/bin/start.script +++ b/bootstrap/bin/start.script @@ -1,4 +1,4 @@ -%% script generated at {2010,10,18} {14,19,26} +%% script generated at {2010,12,2} {17,24,15} {script, {"OTP APN 181 01","R14B01"}, [{preLoaded, diff --git a/bootstrap/bin/start_clean.script b/bootstrap/bin/start_clean.script index 3f472b5e8c..6d3a393d56 100644 --- a/bootstrap/bin/start_clean.script +++ b/bootstrap/bin/start_clean.script @@ -1,4 +1,4 @@ -%% script generated at {2010,10,18} {14,19,26} +%% script generated at {2010,12,2} {17,24,16} {script, {"OTP APN 181 01","R14B01"}, [{preLoaded, diff --git a/bootstrap/lib/compiler/ebin/beam_utils.beam b/bootstrap/lib/compiler/ebin/beam_utils.beam Binary files differindex 7b9c08439e..c335748e8a 100644 --- a/bootstrap/lib/compiler/ebin/beam_utils.beam +++ b/bootstrap/lib/compiler/ebin/beam_utils.beam diff --git a/bootstrap/lib/kernel/ebin/inet.beam b/bootstrap/lib/kernel/ebin/inet.beam Binary files differindex 63088b2e1b..622110bcdb 100644 --- a/bootstrap/lib/kernel/ebin/inet.beam +++ b/bootstrap/lib/kernel/ebin/inet.beam diff --git a/bootstrap/lib/kernel/ebin/kernel.beam b/bootstrap/lib/kernel/ebin/kernel.beam Binary files differindex 471ab154e8..e1db5986dc 100644 --- a/bootstrap/lib/kernel/ebin/kernel.beam +++ b/bootstrap/lib/kernel/ebin/kernel.beam diff --git a/bootstrap/lib/stdlib/ebin/dets.beam b/bootstrap/lib/stdlib/ebin/dets.beam Binary files differindex 3e133f4ff2..4b6e3756e9 100644 --- a/bootstrap/lib/stdlib/ebin/dets.beam +++ b/bootstrap/lib/stdlib/ebin/dets.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_v8.beam b/bootstrap/lib/stdlib/ebin/dets_v8.beam Binary files differindex 2ad134d371..968b9bcb28 100644 --- a/bootstrap/lib/stdlib/ebin/dets_v8.beam +++ b/bootstrap/lib/stdlib/ebin/dets_v8.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_v9.beam b/bootstrap/lib/stdlib/ebin/dets_v9.beam Binary files differindex 2eabd08ed4..355c5819cf 100644 --- a/bootstrap/lib/stdlib/ebin/dets_v9.beam +++ b/bootstrap/lib/stdlib/ebin/dets_v9.beam diff --git a/bootstrap/lib/stdlib/ebin/epp.beam b/bootstrap/lib/stdlib/ebin/epp.beam Binary files differindex 8115d9c474..0483b561d4 100644 --- a/bootstrap/lib/stdlib/ebin/epp.beam +++ b/bootstrap/lib/stdlib/ebin/epp.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_lint.beam b/bootstrap/lib/stdlib/ebin/erl_lint.beam Binary files differindex 5ebd228586..9b54fff03b 100644 --- a/bootstrap/lib/stdlib/ebin/erl_lint.beam +++ b/bootstrap/lib/stdlib/ebin/erl_lint.beam diff --git a/bootstrap/lib/stdlib/ebin/ets.beam b/bootstrap/lib/stdlib/ebin/ets.beam Binary files differindex 6b25af9b1b..9ade9e2edd 100644 --- a/bootstrap/lib/stdlib/ebin/ets.beam +++ b/bootstrap/lib/stdlib/ebin/ets.beam diff --git a/bootstrap/lib/stdlib/ebin/ordsets.beam b/bootstrap/lib/stdlib/ebin/ordsets.beam Binary files differindex c3f2c3b7b1..48effb764e 100644 --- a/bootstrap/lib/stdlib/ebin/ordsets.beam +++ b/bootstrap/lib/stdlib/ebin/ordsets.beam diff --git a/bootstrap/lib/stdlib/ebin/string.beam b/bootstrap/lib/stdlib/ebin/string.beam Binary files differindex 3c3eaf6ed2..0de0b44cb8 100644 --- a/bootstrap/lib/stdlib/ebin/string.beam +++ b/bootstrap/lib/stdlib/ebin/string.beam diff --git a/bootstrap/lib/stdlib/ebin/supervisor.beam b/bootstrap/lib/stdlib/ebin/supervisor.beam Binary files differindex a9e30b4d5d..13cb6032a9 100644 --- a/bootstrap/lib/stdlib/ebin/supervisor.beam +++ b/bootstrap/lib/stdlib/ebin/supervisor.beam diff --git a/bootstrap/lib/stdlib/ebin/unicode.beam b/bootstrap/lib/stdlib/ebin/unicode.beam Binary files differindex 4ca769b9a2..cc3a3c1859 100644 --- a/bootstrap/lib/stdlib/ebin/unicode.beam +++ b/bootstrap/lib/stdlib/ebin/unicode.beam diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index f252c2cbe2..fc950af8ce 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -1842,6 +1842,9 @@ do { \ static void check_balance(ErtsRunQueue *c_rq) { +#if ERTS_MAX_PROCESSES >= (1 << 27) +# error check_balance() assumes ERTS_MAX_PROCESS < (1 << 27) +#endif ErtsRunQueueBalance avg = {0}; Sint64 scheds_reds, full_scheds_reds; int forced, active, current_active, oowc, half_full_scheds, full_scheds, @@ -1965,12 +1968,14 @@ check_balance(ErtsRunQueue *c_rq) run_queue_info[qix].prio[pix].avail = 0; } else { - int xreds = 0; - int procreds = treds; - procreds -= run_queue_info[qix].prio[ERTS_PORT_PRIO_LEVEL].reds; + Sint64 xreds = 0; + Sint64 procreds = treds; + procreds -= + ((Sint64) + run_queue_info[qix].prio[ERTS_PORT_PRIO_LEVEL].reds); for (pix = 0; pix < ERTS_NO_PROC_PRIO_LEVELS; pix++) { - int av; + Sint64 av; if (xreds == 0) av = 100; @@ -1981,9 +1986,10 @@ check_balance(ErtsRunQueue *c_rq) if (av == 0) av = 1; } - run_queue_info[qix].prio[pix].avail = av; + run_queue_info[qix].prio[pix].avail = (int) av; + ASSERT(run_queue_info[qix].prio[pix].avail >= 0); if (pix < PRIORITY_NORMAL) /* ie., max or high */ - xreds += run_queue_info[qix].prio[pix].reds; + xreds += (Sint64) run_queue_info[qix].prio[pix].reds; } run_queue_info[qix].prio[ERTS_PORT_PRIO_LEVEL].avail = 100; } @@ -2088,7 +2094,8 @@ check_balance(ErtsRunQueue *c_rq) if (max_len != 0) { int avail = avg.prio[pix].avail; if (avail != 0) { - max_len = ((100*max_len - 1) / avail) + 1; + max_len = (int) ((100*((Sint64) max_len) - 1) + / ((Sint64) avail)) + 1; avg.prio[pix].max_len = max_len; ASSERT(max_len >= 0); } @@ -2105,9 +2112,10 @@ check_balance(ErtsRunQueue *c_rq) || run_queue_info[qix].prio[pix].avail == 0) limit = 0; else - limit = (((avg.prio[pix].max_len - * run_queue_info[qix].prio[pix].avail) - 1) - / 100 + 1); + limit = (int) (((((Sint64) avg.prio[pix].max_len) + * ((Sint64) run_queue_info[qix].prio[pix].avail)) + - 1) + / 100 + 1); run_queue_info[qix].prio[pix].migration_limit = limit; } } diff --git a/erts/emulator/test/Makefile b/erts/emulator/test/Makefile index a4c02da626..7259e1b84d 100644 --- a/erts/emulator/test/Makefile +++ b/erts/emulator/test/Makefile @@ -83,6 +83,7 @@ MODULES= \ receive_SUITE \ ref_SUITE \ register_SUITE \ + mtx_SUITE \ save_calls_SUITE \ send_term_SUITE \ sensitive_SUITE \ diff --git a/erts/emulator/test/mtx_SUITE.erl b/erts/emulator/test/mtx_SUITE.erl new file mode 100644 index 0000000000..ae77fe4d89 --- /dev/null +++ b/erts/emulator/test/mtx_SUITE.erl @@ -0,0 +1,473 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%% +%% Stress tests of rwmutex implementation. +%% +%% Author: Rickard Green +%% +-module(mtx_SUITE). + +%%-define(line_trace,true). + +-include("test_server.hrl"). + +-export([all/1, init_per_suite/1, end_per_suite/1, init_per_testcase/2, fin_per_testcase/2]). + +-export([long_rwlock/1, + hammer_ets_rwlock/1, + hammer_rwlock/1, + hammer_rwlock_check/1, + hammer_tryrwlock/1, + hammer_tryrwlock_check/1, + hammer_sched_long_rwlock/1, + hammer_sched_long_rwlock_check/1, + hammer_sched_long_freqread_rwlock/1, + hammer_sched_long_freqread_rwlock_check/1, + hammer_sched_long_tryrwlock/1, + hammer_sched_long_tryrwlock_check/1, + hammer_sched_long_freqread_tryrwlock/1, + hammer_sched_long_freqread_tryrwlock_check/1, + hammer_sched_rwlock/1, + hammer_sched_rwlock_check/1, + hammer_sched_freqread_rwlock/1, + hammer_sched_freqread_rwlock_check/1, + hammer_sched_tryrwlock/1, + hammer_sched_tryrwlock_check/1, + hammer_sched_freqread_tryrwlock/1, + hammer_sched_freqread_tryrwlock_check/1]). + +init_per_suite(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + Lib = filename:join([DataDir, atom_to_list(?MODULE)]), + ok = erlang:load_nif(Lib, none), + Config. + +end_per_suite(Config) when is_list(Config) -> + Config. + +init_per_testcase(_Case, Config) -> + Dog = ?t:timetrap(?t:minutes(15)), + [{watchdog, Dog}|Config]. + +fin_per_testcase(_Func, Config) -> + Dog = ?config(watchdog, Config), + ?t:timetrap_cancel(Dog). + +all(suite) -> + [ + long_rwlock, + hammer_rwlock_check, + hammer_rwlock, + hammer_tryrwlock_check, + hammer_tryrwlock, + hammer_ets_rwlock, + hammer_sched_long_rwlock_check, + hammer_sched_long_rwlock, + hammer_sched_long_freqread_rwlock_check, + hammer_sched_long_freqread_rwlock, + hammer_sched_long_tryrwlock_check, + hammer_sched_long_tryrwlock, + hammer_sched_long_freqread_tryrwlock_check, + hammer_sched_long_freqread_tryrwlock, + hammer_sched_rwlock_check, + hammer_sched_rwlock, + hammer_sched_freqread_rwlock_check, + hammer_sched_freqread_rwlock, + hammer_sched_tryrwlock_check, + hammer_sched_tryrwlock, + hammer_sched_freqread_tryrwlock_check, + hammer_sched_freqread_tryrwlock + ]. + +long_rwlock(Config) when is_list(Config) -> + statistics(runtime), + LLRes = long_rw_test(), + {_, RunTime} = statistics(runtime), + %% A very short run time is expected, since + %% threads in the test mostly wait + ?t:format("RunTime=~p~n", [RunTime]), + ?line true = RunTime < 100, + ?line RunTimeStr = "Run-time during test was "++integer_to_list(RunTime)++" ms.", + case LLRes of + ok -> + {comment, RunTimeStr}; + {comment, Comment} -> + {comment, Comment ++ " " ++ RunTimeStr} + end. + +hammer_rwlock(Config) when is_list(Config) -> + hammer_rw_test(false). + +hammer_rwlock_check(Config) when is_list(Config) -> + hammer_rw_test(true). + +hammer_tryrwlock(Config) when is_list(Config) -> + hammer_tryrw_test(false). + +hammer_tryrwlock_check(Config) when is_list(Config) -> + hammer_tryrw_test(true). + +hammer_sched_rwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, false, true, 0, 0). + +hammer_sched_rwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, true, true, 0, 0). + +hammer_sched_freqread_rwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, false, true, 0, 0). + +hammer_sched_freqread_rwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, true, true, 0, 0). + +hammer_sched_tryrwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, false, false, 0, 100). + +hammer_sched_tryrwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, true, false, 0, 100). + +hammer_sched_freqread_tryrwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, false, false, 0, 100). + +hammer_sched_freqread_tryrwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, true, false, 0, 100). + +hammer_sched_long_rwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, false, true, 100, 0). + +hammer_sched_long_rwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, true, true, 100, 0). + +hammer_sched_long_freqread_rwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, false, true, 100, 0). + +hammer_sched_long_freqread_rwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, true, true, 100, 0). + +hammer_sched_long_tryrwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, false, false, 100, 100). + +hammer_sched_long_tryrwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, true, false, 100, 100). + +hammer_sched_long_freqread_tryrwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, false, false, 100, 100). + +hammer_sched_long_freqread_tryrwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, true, false, 100, 100). + +hammer_sched_rwlock_test(FreqRead, LockCheck, Blocking, WaitLocked, WaitUnlocked) -> + case create_rwlock(FreqRead, LockCheck) of + enotsup -> + {skipped, "Not supported."}; + RWLock -> + Onln = erlang:system_info(schedulers_online), + NWPs = case Onln div 3 of + 1 -> case Onln < 4 of + true -> 1; + false -> 2 + end; + X -> X + end, + NRPs = Onln - NWPs, + NoLockOps = ((((50000000 div Onln) + div case {Blocking, WaitLocked} of + {false, 0} -> 1; + _ -> 10 + end) + div (case WaitLocked == 0 of + true -> 1; + false -> WaitLocked*250 + end)) + div handicap()), + ?t:format("NoLockOps=~p~n", [NoLockOps]), + Sleep = case Blocking of + true -> NoLockOps; + false -> NoLockOps div 10 + end, + WPs = lists:map( + fun (Sched) -> + spawn_opt( + fun () -> + io:format("Writer on scheduler ~p.~n", + [Sched]), + Sched = erlang:system_info(scheduler_id), + receive go -> gone end, + hammer_sched_rwlock_proc(RWLock, + Blocking, + true, + WaitLocked, + WaitUnlocked, + NoLockOps, + Sleep), + Sched = erlang:system_info(scheduler_id) + end, + [link, {scheduler, Sched}]) + end, + lists:seq(1, NWPs)), + RPs = lists:map( + fun (Sched) -> + spawn_opt( + fun () -> + io:format("Reader on scheduler ~p.~n", + [Sched]), + Sched = erlang:system_info(scheduler_id), + receive go -> gone end, + hammer_sched_rwlock_proc(RWLock, + Blocking, + false, + WaitLocked, + WaitUnlocked, + NoLockOps, + Sleep), + Sched = erlang:system_info(scheduler_id) + end, + [link, {scheduler, Sched}]) + end, + lists:seq(NWPs + 1, NWPs + NRPs)), + Procs = WPs ++ RPs, + case {Blocking, WaitLocked} of + {_, 0} -> ok; + {false, _} -> ok; + _ -> statistics(runtime) + end, + lists:foreach(fun (P) -> P ! go end, Procs), + lists:foreach(fun (P) -> + M = erlang:monitor(process, P), + receive + {'DOWN', M, process, P, _} -> + ok + end + end, + Procs), + case {Blocking, WaitLocked} of + {_, 0} -> ok; + {false, _} -> ok; + _ -> + {_, RunTime} = statistics(runtime), + ?t:format("RunTime=~p~n", [RunTime]), + ?line true = RunTime < 500, + {comment, + "Run-time during test was " + ++ integer_to_list(RunTime) + ++ " ms."} + end + end. + +hammer_sched_rwlock_proc(_RWLock, + _Blocking, + _WriteOp, + _WaitLocked, + _WaitUnlocked, + 0, + _Sleep) -> + ok; +hammer_sched_rwlock_proc(RWLock, + Blocking, + WriteOp, + WaitLocked, + WaitUnlocked, + Times, + Sleep) when Times rem Sleep == 0 -> + rwlock_op(RWLock, Blocking, WriteOp, WaitLocked, WaitUnlocked), + hammer_sched_rwlock_proc(RWLock, + Blocking, + WriteOp, + WaitLocked, + WaitUnlocked, + Times - 1, + Sleep); +hammer_sched_rwlock_proc(RWLock, + Blocking, + WriteOp, + WaitLocked, + WaitUnlocked, + Times, + Sleep) -> + rwlock_op(RWLock, Blocking, WriteOp, WaitLocked, 0), + hammer_sched_rwlock_proc(RWLock, + Blocking, + WriteOp, + WaitLocked, + WaitUnlocked, + Times - 1, + Sleep). + +-define(HAMMER_ETS_RWLOCK_REPEAT_TIMES, 1). +-define(HAMMER_ETS_RWLOCK_TSIZE, 500). + +hammer_ets_rwlock(Config) when is_list(Config) -> + {Ops, Procs} = case handicap() of + 1 -> {20000, 500}; + 2 -> {20000, 50}; + 3 -> {2000, 50}; + _ -> {200, 50} + end, + ?t:format("Procs=~p~nOps=~p~n", [Procs, Ops]), + lists:foreach(fun (XOpts) -> + ?t:format("Running with extra opts: ~p", [XOpts]), + hammer_ets_rwlock_test(XOpts, true, 2, Ops, + Procs, false) + end, + [[], + [{read_concurrency, true}], + [{write_concurrency, true}], + [{read_concurrency, true},{write_concurrency, true}]]), + ok. + +%% Aux funcs + +long_rw_test() -> + exit(no_nif_implementation). + +hammer_rw_test(_Arg) -> + exit(no_nif_implementation). + +hammer_tryrw_test(_Arg) -> + exit(no_nif_implementation). + +create_rwlock(_FreqRead, _LockCheck) -> + exit(no_nif_implementation). + +rwlock_op(_RWLock, _Blocking, _WriteOp, _WaitLocked, _WaitUnlocked) -> + exit(no_nif_implementation). + +hammer_ets_rwlock_put_data() -> + put(?MODULE, {"here are some", data, "to store", make_ref()}). + +hammer_ets_rwlock_get_data() -> + get(?MODULE). + +hammer_ets_rwlock_ops(_T, _UW, _N, _C, _SC, 0) -> + ok; +hammer_ets_rwlock_ops(T, UW, N, C, SC, Tot) when N >= ?HAMMER_ETS_RWLOCK_TSIZE -> + hammer_ets_rwlock_ops(T, UW, 0, C, SC, Tot); +hammer_ets_rwlock_ops(T, UW, N, 0, SC, Tot) -> + case UW of + true -> + true = ets:insert(T, {N, Tot, hammer_ets_rwlock_get_data()}); + false -> + [{N, _, _}] = ets:lookup(T, N) + end, + hammer_ets_rwlock_ops(T, UW, N+1, SC, SC, Tot-1); +hammer_ets_rwlock_ops(T, UW, N, C, SC, Tot) -> + case UW of + false -> + true = ets:insert(T, {N, Tot, hammer_ets_rwlock_get_data()}); + true -> + [{N, _, _}] = ets:lookup(T, N) + end, + hammer_ets_rwlock_ops(T, UW, N+1, C-1, SC, Tot-1). + +hammer_ets_rwlock_init(T, N) when N < ?HAMMER_ETS_RWLOCK_TSIZE -> + ets:insert(T, {N, N, N}), + hammer_ets_rwlock_init(T, N+1); +hammer_ets_rwlock_init(_T, _N) -> + ok. + +hammer_ets_rwlock_test(XOpts, UW, C, N, NP, SC) -> + receive after 100 -> ok end, + {TP, TM} = spawn_monitor( + fun () -> + _L = repeat_list( + fun () -> + Caller = self(), + T = fun () -> + Parent = self(), + hammer_ets_rwlock_put_data(), + T=ets:new(x, [public | XOpts]), + hammer_ets_rwlock_init(T, 0), + Ps0 = repeat_list( + fun () -> + spawn_link( + fun () -> + hammer_ets_rwlock_put_data(), + receive go -> ok end, + hammer_ets_rwlock_ops(T, UW, N, C, C, N), + Parent ! {done, self()}, + receive after infinity -> ok end + end) + end, + NP - case SC of + false -> 0; + _ -> 1 + end), + Ps = case SC of + false -> Ps0; + _ -> [spawn_link(fun () -> + hammer_ets_rwlock_put_data(), + receive go -> ok end, + hammer_ets_rwlock_ops(T, UW, N, SC, SC, N), + Parent ! {done, self()}, + receive after infinity -> ok end + end) | Ps0] + end, + Start = now(), + lists:foreach(fun (P) -> P ! go end, Ps), + lists:foreach(fun (P) -> receive {done, P} -> ok end end, Ps), + Stop = now(), + lists:foreach(fun (P) -> + unlink(P), + exit(P, bang), + M = erlang:monitor(process, P), + receive + {'DOWN', M, process, P, _} -> ok + end + end, Ps), + Res = timer:now_diff(Stop, Start)/1000000, + Caller ! {?MODULE, self(), Res} + end, + TP = spawn_link(T), + receive + {?MODULE, TP, Res} -> + Res + end + end, + ?HAMMER_ETS_RWLOCK_REPEAT_TIMES) + end), + receive + {'DOWN', TM, process, TP, _} -> ok + end. + +repeat_list(Fun, N) -> + repeat_list(Fun, N, []). + +repeat_list(_Fun, 0, Acc) -> + Acc; +repeat_list(Fun, N, Acc) -> + repeat_list(Fun, N-1, [Fun()|Acc]). + + +handicap() -> + X0 = case catch (erlang:system_info(logical_processors_available) >= + erlang:system_info(schedulers_online)) of + true -> 1; + _ -> 2 + end, + case erlang:system_info(build_type) of + opt -> + X0; + ReallySlow when ReallySlow == debug; + ReallySlow == valgrind; + ReallySlow == purify -> + X0*3; + _Slow -> + X0*2 + end. + diff --git a/erts/emulator/test/mtx_SUITE_data/Makefile.src b/erts/emulator/test/mtx_SUITE_data/Makefile.src new file mode 100644 index 0000000000..b6c843269c --- /dev/null +++ b/erts/emulator/test/mtx_SUITE_data/Makefile.src @@ -0,0 +1,30 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2010. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# + +include @erts_lib_include_internal_generated@@[email protected] +include @erts_lib_include_internal_generated@@DS@erts_internal.mk + +NIF_LIBS = mtx_SUITE@dll@ + +SHLIB_EXTRA_CFLAGS = $(ETHR_DEFS) -I@erts_lib_include_internal@ -I@erts_lib_include_internal_generated@ +LIBS = @ERTS_LIBS@ + +all: $(NIF_LIBS) + +@SHLIB_RULES@ diff --git a/erts/emulator/test/mtx_SUITE_data/mtx_SUITE.c b/erts/emulator/test/mtx_SUITE_data/mtx_SUITE.c new file mode 100644 index 0000000000..818023211c --- /dev/null +++ b/erts/emulator/test/mtx_SUITE_data/mtx_SUITE.c @@ -0,0 +1,692 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2010. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +/* + * Stress tests of rwmutex implementation. + * + * Author: Rickard Green + */ + +#include "erl_nif.h" + +#ifdef __WIN32__ +# ifndef WIN32_LEAN_AND_MEAN +# define WIN32_LEAN_AND_MEAN +# endif +# include <windows.h> +#else +# include "ethread.h" +# include "erl_misc_utils.h" +# include <unistd.h> +#endif + +#include <errno.h> +#include <stdio.h> + +static int +fail(const char *file, int line, const char *function, const char *assertion); + +#undef ASSERT +#define ASSERT(X) ((void) ((X) ? 1 : fail(__FILE__, __LINE__, __func__, #X))) + +#ifdef __WIN32__ +/* + * We cannot access the ethread symbols directly; test + * what we got in the nif api instead... + */ +#define HAVE_FREQREAD_SUPPORT 0 +#define RWMUTEX_T ErlNifRWLock +#define RWMUTEX_CREATE(FR) enif_rwlock_create("dummy") +#define RWMUTEX_DESTROY enif_rwlock_destroy +#define RWMUTEX_WLOCK enif_rwlock_rwlock +#define RWMUTEX_TRYWLOCK enif_rwlock_tryrwlock +#define RWMUTEX_WUNLOCK enif_rwlock_rwunlock +#define RWMUTEX_TRYRLOCK enif_rwlock_tryrlock +#define RWMUTEX_RLOCK enif_rwlock_rlock +#define RWMUTEX_RUNLOCK enif_rwlock_runlock +#define THR_ID ErlNifTid +#define THR_CREATE(A, B, C, D) enif_thread_create("dummy", (A), (B), (C), (D)) +#define THR_JOIN enif_thread_join +#define ATOMIC_T volatile LONG +#define ATOMIC_INIT(VarP, Val) (*(VarP) = (Val)) +#define ATOMIC_SET(VarP, Val) (*(VarP) = (Val)) +#define ATOMIC_READ(VarP) (*(VarP)) +#define ATOMIC_INC InterlockedIncrement +#define ATOMIC_DEC InterlockedDecrement + +#else + +#ifdef ETHR_USE_OWN_RWMTX_IMPL__ +# define HAVE_FREQREAD_SUPPORT 1 +#else +# define HAVE_FREQREAD_SUPPORT 0 +#endif + +#define RWMUTEX_T ethr_rwmutex +static ethr_rwmutex * +RWMUTEX_CREATE(int freqread) +{ + ethr_rwmutex *rwmtx = enif_alloc(sizeof(ethr_rwmutex)); + ethr_rwmutex_opt rwmtx_opt = ETHR_RWMUTEX_OPT_DEFAULT_INITER; + if (freqread) + rwmtx_opt.type = ETHR_RWMUTEX_TYPE_FREQUENT_READ; + ASSERT(rwmtx); + ASSERT(ethr_rwmutex_init_opt(rwmtx, &rwmtx_opt) == 0); + return rwmtx; +} +static void +RWMUTEX_DESTROY(ethr_rwmutex *rwmtx) +{ + ASSERT(ethr_rwmutex_destroy(rwmtx) == 0); + enif_free(rwmtx); +} +#define RWMUTEX_TRYWLOCK ethr_rwmutex_tryrwlock +#define RWMUTEX_WLOCK ethr_rwmutex_rwlock +#define RWMUTEX_WUNLOCK ethr_rwmutex_rwunlock +#define RWMUTEX_TRYRLOCK ethr_rwmutex_tryrlock +#define RWMUTEX_RLOCK ethr_rwmutex_rlock +#define RWMUTEX_RUNLOCK ethr_rwmutex_runlock +#define THR_ID ethr_tid +#define THR_CREATE ethr_thr_create +#define THR_JOIN ethr_thr_join +#define ATOMIC_T ethr_atomic_t +#define ATOMIC_INIT ethr_atomic_init +#define ATOMIC_SET ethr_atomic_set +#define ATOMIC_READ ethr_atomic_read +#define ATOMIC_INC ethr_atomic_inc +#define ATOMIC_DEC ethr_atomic_dec + +#endif + + +#if !defined(__func__) +# if !defined(__STDC_VERSION__) || __STDC_VERSION__ < 199901L +# if !defined(__GNUC__) || __GNUC__ < 2 +# define __func__ "[unknown_function]" +# else +# define __func__ __FUNCTION__ +# endif +# endif +#endif + +static void milli_sleep(int ms); +static int get_bool(ErlNifEnv* env, ERL_NIF_TERM term); + +/* + * Long rwlock testcase + */ + +#define LONG_RW_NO_W_THREADS 6 +#define LONG_RW_NO_THREADS 20 +#define LONG_RW_NO_WLOCK_COUNT 100 + +typedef struct { + RWMUTEX_T *rwlock; + ATOMIC_T *is_wlocked; + ATOMIC_T *is_rlocked; + int *stop; + int *count; + int sleep; +} long_rw_t; + +static void * +long_rw_w(void *varg) +{ + long_rw_t *arg = varg; + int stop = 0; + do { + RWMUTEX_WLOCK(arg->rwlock); + ASSERT(!ATOMIC_READ(arg->is_wlocked)); + ATOMIC_SET(arg->is_wlocked, 1); + ASSERT(!ATOMIC_READ(arg->is_rlocked)); + milli_sleep(arg->sleep); + if (++(*arg->count) > LONG_RW_NO_WLOCK_COUNT) + stop = *arg->stop = 1; + ATOMIC_SET(arg->is_wlocked, 0); + ASSERT(!ATOMIC_READ(arg->is_rlocked)); + RWMUTEX_WUNLOCK(arg->rwlock); + } while (!stop); + return NULL; +} + +static void * +long_rw_r(void *varg) +{ + long_rw_t *arg = varg; + int stop; + do { + RWMUTEX_RLOCK(arg->rwlock); + ASSERT(!ATOMIC_READ(arg->is_wlocked)); + ATOMIC_INC(arg->is_rlocked); + milli_sleep(arg->sleep); + stop = *arg->stop; + ATOMIC_DEC(arg->is_rlocked); + ASSERT(!ATOMIC_READ(arg->is_wlocked)); + RWMUTEX_RUNLOCK(arg->rwlock); + } while (!stop); + return NULL; +} + + +static ERL_NIF_TERM long_rw_test(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) +{ + int res, freqread, i, count, stop; + ATOMIC_T is_wlocked, is_rlocked; + THR_ID tid[LONG_RW_NO_THREADS]; + long_rw_t arg; + long_rw_t targ[LONG_RW_NO_THREADS]; + + ATOMIC_INIT(&is_wlocked, 0); + ATOMIC_INIT(&is_rlocked, 0); + + freqread = 0; + + arg.is_wlocked = &is_wlocked; + arg.is_rlocked = &is_rlocked; + arg.count = &count; + arg.stop = &stop; + + restart: + + stop = 0; + count = 0; + + arg.rwlock = RWMUTEX_CREATE(freqread); + + ASSERT(arg.rwlock); + + for (i = 0; i < LONG_RW_NO_W_THREADS; i++) { + targ[i] = arg; + targ[i].sleep = 100 + i*10; + ASSERT(THR_CREATE(&tid[i], long_rw_w, &targ[i], NULL) == 0); + } + for (; i < LONG_RW_NO_THREADS; i++) { + targ[i] = arg; + targ[i].sleep = 100; + ASSERT(THR_CREATE(&tid[i], long_rw_r, &targ[i], NULL) == 0); + } + for (i = 0; i < LONG_RW_NO_THREADS; i++) + ASSERT(THR_JOIN(tid[i], NULL) == 0); + + ASSERT(!ATOMIC_READ(arg.is_wlocked)); + ASSERT(!ATOMIC_READ(arg.is_rlocked)); + + RWMUTEX_DESTROY(arg.rwlock); + + if (HAVE_FREQREAD_SUPPORT && !freqread) { + freqread = 1; + goto restart; + } + + if (freqread) + return enif_make_atom(env, "ok"); + else + return enif_make_tuple2(env, + enif_make_atom(env, + "comment"), + enif_make_string(env, + "No frequent read test made.", + ERL_NIF_LATIN1)); +} + +/* + * Hammer rwlock testcase + */ + +#define HAMMER_RW_NO_W_THREADS 6 +#define HAMMER_RW_NO_THREADS 20 +#define HAMMER_RW_NO_WLOCK_COUNT 1000000 + +typedef struct { + RWMUTEX_T *rwlock; + ATOMIC_T is_locked; + int lock_check; + int stop; + int count; +} hammer_rw_t; + +static void * +hammer_rw_w(void *varg) +{ + hammer_rw_t *arg = varg; + int stop = 0; + do { + RWMUTEX_WLOCK(arg->rwlock); + if (arg->lock_check) { + ASSERT(!ATOMIC_READ(&arg->is_locked)); + ATOMIC_SET(&arg->is_locked, -1); + } + if (++arg->count > HAMMER_RW_NO_WLOCK_COUNT) + stop = arg->stop = 1; + if (arg->lock_check) { + ASSERT(ATOMIC_READ(&arg->is_locked) == -1); + ATOMIC_SET(&arg->is_locked, 0); + } + RWMUTEX_WUNLOCK(arg->rwlock); + } while (!stop); + return NULL; +} + +static void * +hammer_rw_r(void *varg) +{ + hammer_rw_t *arg = varg; + int stop; + do { + RWMUTEX_RLOCK(arg->rwlock); + if (arg->lock_check) { + ASSERT(ATOMIC_READ(&arg->is_locked) >= 0); + ATOMIC_INC(&arg->is_locked); + } + stop = arg->stop; + if (arg->lock_check) { + ASSERT(ATOMIC_READ(&arg->is_locked) > 0); + ATOMIC_DEC(&arg->is_locked); + } + RWMUTEX_RUNLOCK(arg->rwlock); + } while (!stop); + return NULL; +} + + +static ERL_NIF_TERM hammer_rw_test(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) +{ + hammer_rw_t arg; + char buf[10]; + int res, freqread, i; + THR_ID tid[HAMMER_RW_NO_THREADS]; + + if (argc != 1) + goto badarg; + + arg.lock_check = get_bool(env, argv[0]); + if (arg.lock_check < 0) + goto badarg; + + ATOMIC_INIT(&arg.is_locked, 0); + + freqread = 0; + + restart: + arg.stop = 0; + arg.count = 0; + + arg.rwlock = RWMUTEX_CREATE(freqread); + + ASSERT(arg.rwlock); + + for (i = 0; i < HAMMER_RW_NO_W_THREADS; i++) + ASSERT(THR_CREATE(&tid[i], hammer_rw_w, &arg, NULL) == 0); + for (; i < HAMMER_RW_NO_THREADS; i++) + ASSERT(THR_CREATE(&tid[i], hammer_rw_r, &arg, NULL) == 0); + for (i = 0; i < HAMMER_RW_NO_THREADS; i++) + ASSERT(THR_JOIN(tid[i], NULL) == 0); + + ASSERT(!ATOMIC_READ(&arg.is_locked)); + + RWMUTEX_DESTROY(arg.rwlock); + + if (HAVE_FREQREAD_SUPPORT && !freqread) { + freqread = 1; + goto restart; + } + + if (freqread) + return enif_make_atom(env, "ok"); + else + return enif_make_tuple2(env, + enif_make_atom(env, + "comment"), + enif_make_string(env, + "No frequent read test made.", + ERL_NIF_LATIN1)); + badarg: + return enif_make_badarg(env); +} + +/* + * Hammer try rwlock testcase + */ + +#define HAMMER_TRYRW_NO_W_THREADS 10 +#define HAMMER_TRYRW_NO_THREADS 20 +#define HAMMER_TRYRW_NO_WLOCK_COUNT 10000000 +#define HAMMER_TRYRW_NO_RLOCK_COUNT 10000000 +#define HAMMER_TRYRW_NO_WLOCK_WAIT_COUNT ((10*HAMMER_TRYRW_NO_WLOCK_COUNT)/8) +#define HAMMER_TRYRW_NO_RLOCK_WAIT_COUNT ((10*HAMMER_TRYRW_NO_RLOCK_COUNT)/8) + +typedef struct { + RWMUTEX_T *rwlock; + ATOMIC_T is_locked; + int lock_check; + int w_count; + ATOMIC_T r_count; +} hammer_tryrw_t; + +static void * +hammer_tryrw_w(void *varg) +{ + hammer_tryrw_t *arg = varg; + int stop = 0; + int wait = 0; + do { + while (EBUSY == RWMUTEX_TRYWLOCK(arg->rwlock)); + if (arg->lock_check) { + ASSERT(!ATOMIC_READ(&arg->is_locked)); + ATOMIC_SET(&arg->is_locked, -1); + } + if (++arg->w_count > HAMMER_TRYRW_NO_WLOCK_COUNT) + stop = 1; + else if (arg->w_count > HAMMER_TRYRW_NO_RLOCK_WAIT_COUNT) + wait = 1; + if (arg->lock_check) { + ASSERT(ATOMIC_READ(&arg->is_locked) == -1); + ATOMIC_SET(&arg->is_locked, 0); + } + RWMUTEX_WUNLOCK(arg->rwlock); + if (wait) + milli_sleep(1); + } while (!stop); + return NULL; +} + +static void * +hammer_tryrw_r(void *varg) +{ + hammer_tryrw_t *arg = varg; + long r_count; + int stop = 0; + int wait = 0; + do { + while (EBUSY == RWMUTEX_TRYRLOCK(arg->rwlock)); + if (arg->lock_check) { + ASSERT(ATOMIC_READ(&arg->is_locked) >= 0); + ATOMIC_INC(&arg->is_locked); + } + ATOMIC_INC(&arg->r_count); + r_count = ATOMIC_READ(&arg->r_count); + if (r_count > HAMMER_TRYRW_NO_RLOCK_COUNT) + stop = 1; + else if (r_count > HAMMER_TRYRW_NO_RLOCK_WAIT_COUNT) + wait = 1; + if (arg->lock_check) { + ASSERT(ATOMIC_READ(&arg->is_locked) > 0); + ATOMIC_DEC(&arg->is_locked); + } + RWMUTEX_RUNLOCK(arg->rwlock); + if (wait) + milli_sleep(1); + } while (!stop); + return NULL; +} + + +static ERL_NIF_TERM hammer_tryrw_test(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) +{ + hammer_tryrw_t arg; + char buf[10]; + int res, freqread, i; + THR_ID tid[HAMMER_TRYRW_NO_THREADS]; + + if (argc != 1) + goto badarg; + + arg.lock_check = get_bool(env, argv[0]); + if (arg.lock_check < 0) + goto badarg; + + ATOMIC_INIT(&arg.is_locked, 0); + freqread = 0; + + restart: + + arg.w_count = 0; + ATOMIC_INIT(&arg.r_count, 0); + + arg.rwlock = RWMUTEX_CREATE(freqread); + + ASSERT(arg.rwlock); + + for (i = 0; i < HAMMER_TRYRW_NO_W_THREADS; i++) + ASSERT(THR_CREATE(&tid[i], hammer_tryrw_w, &arg, NULL) == 0); + for (; i < HAMMER_TRYRW_NO_THREADS; i++) + ASSERT(THR_CREATE(&tid[i], hammer_tryrw_r, &arg, NULL) == 0); + for (i = 0; i < HAMMER_TRYRW_NO_THREADS; i++) + ASSERT(THR_JOIN(tid[i], NULL) == 0); + + ASSERT(!ATOMIC_READ(&arg.is_locked)); + + RWMUTEX_DESTROY(arg.rwlock); + + if (HAVE_FREQREAD_SUPPORT && !freqread) { + freqread = 1; + goto restart; + } + + if (freqread) + return enif_make_atom(env, "ok"); + else + return enif_make_tuple2(env, + enif_make_atom(env, + "comment"), + enif_make_string(env, + "No frequent read test made.", + ERL_NIF_LATIN1)); + badarg: + return enif_make_badarg(env); +} + +typedef struct { + int lock_check; + ATOMIC_T is_locked; + RWMUTEX_T *rwlock; +} rwlock_resource_t; + +static void +rwlock_destructor(ErlNifEnv* env, void* obj) +{ + rwlock_resource_t *rwlr = obj; + if (rwlr->lock_check) + ASSERT(!ATOMIC_READ(&rwlr->is_locked)); + RWMUTEX_DESTROY(rwlr->rwlock); +} + +/* + * create_rwlock(FreqRead, LockCheck) + */ + +static ERL_NIF_TERM +create_rwlock(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) +{ + int lock_check, freqread; + ERL_NIF_TERM rwlock_term; + rwlock_resource_t *rwlr; + char buf[100]; + + if (argc != 2) + goto badarg; + + freqread = get_bool(env, argv[0]); + if (freqread < 0) + goto badarg; + + if (!HAVE_FREQREAD_SUPPORT && freqread) + return enif_make_atom(env, "enotsup"); + + lock_check = get_bool(env, argv[1]); + if (lock_check < 0) + goto badarg; + + rwlr = enif_alloc_resource(enif_priv_data(env), sizeof(rwlock_resource_t)); + rwlr->lock_check = lock_check; + ATOMIC_INIT(&rwlr->is_locked, 0); + rwlr->rwlock = RWMUTEX_CREATE(freqread); + rwlock_term = enif_make_resource(env, rwlr); + enif_release_resource(rwlr); + return rwlock_term; + + badarg: + return enif_make_badarg(env); +} + +/* + * rwlock_op(RWLock, Blocking, WriteOp, WaitTime) + */ + +static ERL_NIF_TERM +rwlock_op(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) +{ + rwlock_resource_t *rwlr; + int blocking, write, wait_locked, wait_unlocked; + + if (argc != 5) + goto badarg; + + if (!enif_get_resource(env, argv[0], enif_priv_data(env), (void **) &rwlr)) + goto badarg; + + blocking = get_bool(env, argv[1]); + if (blocking < 0) + goto badarg; + + write = get_bool(env, argv[2]); + if (write < 0) + goto badarg; + + if (!enif_get_int(env, argv[3], &wait_locked)) + goto badarg; + if (wait_locked < 0) + goto badarg; + + if (!enif_get_int(env, argv[4], &wait_unlocked)) + goto badarg; + if (wait_unlocked < 0) + goto badarg; + + if (write) { + if (blocking) + RWMUTEX_WLOCK(rwlr->rwlock); + else + while (EBUSY == RWMUTEX_TRYWLOCK(rwlr->rwlock)); + if (rwlr->lock_check) { + ASSERT(!ATOMIC_READ(&rwlr->is_locked)); + ATOMIC_SET(&rwlr->is_locked, -1); + } + } + else { + if (blocking) + RWMUTEX_RLOCK(rwlr->rwlock); + else + while (EBUSY == RWMUTEX_TRYRLOCK(rwlr->rwlock)); + if (rwlr->lock_check) { + ASSERT(ATOMIC_READ(&rwlr->is_locked) >= 0); + ATOMIC_INC(&rwlr->is_locked); + } + } + + if (wait_locked) + milli_sleep(wait_locked); + + if (write) { + if (rwlr->lock_check) { + ASSERT(ATOMIC_READ(&rwlr->is_locked) == -1); + ATOMIC_SET(&rwlr->is_locked, 0); + } + RWMUTEX_WUNLOCK(rwlr->rwlock); + } + else { + if (rwlr->lock_check) { + ASSERT(ATOMIC_READ(&rwlr->is_locked) > 0); + ATOMIC_DEC(&rwlr->is_locked); + } + RWMUTEX_RUNLOCK(rwlr->rwlock); + } + + if (wait_unlocked) + milli_sleep(wait_unlocked); + + return enif_make_atom(env, "ok"); + badarg: + return enif_make_badarg(env); +} + +static int load_nif_lib(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) +{ + *priv_data = enif_open_resource_type(env, + NULL, + "rwlock_resource", + rwlock_destructor, + ERL_NIF_RT_CREATE, + NULL); + if (*priv_data) + return 0; + else + return -1; +} + +/* + * 0 -> false + * >0 -> true + * <0 -> error + */ + +static int +get_bool(ErlNifEnv* env, ERL_NIF_TERM term) +{ + int res; + char buf[10]; + + res = enif_get_atom(env, term, buf, sizeof(buf), ERL_NIF_LATIN1); + if (res == 0) + return -1; + if (strcmp("false", buf) == 0) + return 0; + else if (strcmp("true", buf) == 0) + return 1; + else + return -1; +} + +static int +fail(const char *file, int line, const char *function, const char *assertion) +{ + fprintf(stderr, "%s:%d: Assertion failed in %s(): %s\n", + file, line, function, assertion); + abort(); +} + +static void +milli_sleep(int ms) +{ +#ifdef __WIN32__ + Sleep(ms); +#else + while (erts_milli_sleep(ms) != 0); +#endif +} + +static ErlNifFunc nif_funcs[] = { + {"long_rw_test", 0, long_rw_test}, + {"hammer_rw_test", 1, hammer_rw_test}, + {"hammer_tryrw_test", 1, hammer_tryrw_test}, + {"create_rwlock", 2, create_rwlock}, + {"rwlock_op", 5, rwlock_op} +}; + +ERL_NIF_INIT(mtx_SUITE, nif_funcs, load_nif_lib, NULL, NULL, NULL) diff --git a/erts/etc/unix/format_man_pages b/erts/etc/unix/format_man_pages index 2c4f6eee4f..93dcdcd8fa 100644 --- a/erts/etc/unix/format_man_pages +++ b/erts/etc/unix/format_man_pages @@ -3,7 +3,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1996-2009. All Rights Reserved. +# Copyright Ericsson AB 1996-2010. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -59,34 +59,21 @@ esac # Create the 'cat' directories (probably not needed) # -cd $ERL_ROOT +cd $ERL_ROOT/man -if [ ! -d man/cat1 ] -then - mkdir man/cat1 -fi +for d in 0 1 2 3 4 5 6 7 8 9 +do + if [ ! -d cat$d ] + then + mkdir cat$d + fi -if [ ! -d man/cat3 ] -then - mkdir man/cat3 -fi - -if [ ! -d man/cat4 ] -then - mkdir man/cat4 -fi - -if [ ! -d man/cat6 ] -then - mkdir man/cat6 -fi +done # # Cleanup old formatting # -cd $ERL_ROOT/man - rm -f whatis windex # Remove old cat files diff --git a/erts/include/internal/ethr_mutex.h b/erts/include/internal/ethr_mutex.h index 8d9d5e3d08..01855864e3 100644 --- a/erts/include/internal/ethr_mutex.h +++ b/erts/include/internal/ethr_mutex.h @@ -33,6 +33,13 @@ # define ETHR_MTX_HARD_DEBUG #endif +#if 0 +# define ETHR_MTX_CHK_EXCL +#if 1 +# define ETHR_MTX_CHK_NON_EXCL +#endif +#endif + #ifdef ETHR_MTX_HARD_DEBUG # ifdef __GNUC__ # warning ETHR_MTX_HARD_DEBUG @@ -49,6 +56,15 @@ #if defined(ETHR_USE_OWN_RWMTX_IMPL__) || defined(ETHR_USE_OWN_MTX_IMPL__) +#ifdef ETHR_DEBUG +# ifndef ETHR_MTX_CHK_EXCL +# define ETHR_MTX_CHK_EXCL +# endif +# ifndef ETHR_MTX_CHK_NON_EXCL +# define ETHR_MTX_CHK_NON_EXCL +# endif +#endif + #if 0 # define ETHR_MTX_Q_LOCK_SPINLOCK__ # define ETHR_MTX_QLOCK_TYPE__ ethr_spinlock_t @@ -68,8 +84,8 @@ /* frequent read kind */ #define ETHR_RWMTX_R_FLG__ (((long) 1) << 28) -#define ETHR_RWMTX_R_PEND_UNLCK_MASK__ (ETHR_RWMTX_R_FLG__ - 1) -#define ETHR_RWMTX_R_MASK__ (ETHR_RWMTX_R_WAIT_FLG__ - 1) +#define ETHR_RWMTX_R_ABRT_UNLCK_FLG__ (((long) 1) << 27) +#define ETHR_RWMTX_R_PEND_UNLCK_MASK__ (ETHR_RWMTX_R_ABRT_UNLCK_FLG__ - 1) /* normal kind */ #define ETHR_RWMTX_RS_MASK__ (ETHR_RWMTX_R_WAIT_FLG__ - 1) @@ -79,6 +95,19 @@ #define ETHR_CND_WAIT_FLG__ ETHR_RWMTX_R_WAIT_FLG__ +#ifdef ETHR_DEBUG +#define ETHR_DBG_CHK_UNUSED_FLG_BITS(V) \ + ETHR_ASSERT(!((V) & ~(ETHR_RWMTX_W_FLG__ \ + | ETHR_RWMTX_W_WAIT_FLG__ \ + | ETHR_RWMTX_R_WAIT_FLG__ \ + | ETHR_RWMTX_RS_MASK__))) +#else +#define ETHR_DBG_CHK_UNUSED_FLG_BITS(V) +#endif + +#define ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(MTX) \ + ETHR_DBG_CHK_UNUSED_FLG_BITS(ethr_atomic_read(&(MTX)->mtxb.flgs)) + struct ethr_mutex_base_ { #ifdef ETHR_MTX_HARD_DEBUG_FENCE long pre_fence; @@ -91,6 +120,12 @@ struct ethr_mutex_base_ { #ifdef ETHR_MTX_HARD_DEBUG_WSQ int ws; #endif +#ifdef ETHR_MTX_CHK_EXCL + ethr_atomic_t exclusive; +#endif +#ifdef ETHR_MTX_CHK_NON_EXCL + ethr_atomic_t non_exclusive; +#endif #ifdef ETHR_MTX_HARD_DEBUG_LFS ethr_atomic_t hdbg_lfs; #endif @@ -344,6 +379,116 @@ do { \ #define ETHR_MTX_HARD_DEBUG_FENCE_INIT(X) #endif +#ifdef ETHR_MTX_CHK_EXCL + +#if !defined(ETHR_DEBUG) && defined(__GNUC__) +#warning "check exclusive is enabled" +#endif + +# define ETHR_MTX_CHK_EXCL_INIT__(MTXB) \ + ethr_atomic_init(&(MTXB)->exclusive, 0) + +# define ETHR_MTX_CHK_EXCL_IS_EXCL(MTXB) \ +do { \ + ETHR_COMPILER_BARRIER; \ + if (!ethr_atomic_read(&(MTXB)->exclusive)) \ + ethr_assert_failed(__FILE__, __LINE__, __func__,\ + "is exclusive"); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_IS_NOT_EXCL(MTXB) \ +do { \ + ETHR_COMPILER_BARRIER; \ + if (ethr_atomic_read(&(MTXB)->exclusive)) \ + ethr_assert_failed(__FILE__, __LINE__, __func__,\ + "is not exclusive"); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_SET_EXCL(MTXB) \ +do { \ + ETHR_MTX_CHK_EXCL_IS_NOT_EXCL((MTXB)); \ + ethr_atomic_set(&(MTXB)->exclusive, 1); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_UNSET_EXCL(MTXB) \ +do { \ + ETHR_MTX_CHK_EXCL_IS_EXCL((MTXB)); \ + ethr_atomic_set(&(MTXB)->exclusive, 0); \ + ETHR_COMPILER_BARRIER; \ +} while (0) + +#ifdef ETHR_MTX_CHK_NON_EXCL + +#if !defined(ETHR_DEBUG) && defined(__GNUC__) +#warning "check non-exclusive is enabled" +#endif + +# define ETHR_MTX_CHK_NON_EXCL_INIT__(MTXB) \ + ethr_atomic_init(&(MTXB)->non_exclusive, 0) +# define ETHR_MTX_CHK_EXCL_IS_NON_EXCL(MTXB) \ +do { \ + ETHR_COMPILER_BARRIER; \ + if (!ethr_atomic_read(&(MTXB)->non_exclusive)) \ + ethr_assert_failed(__FILE__, __LINE__, __func__,\ + "is non-exclusive"); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_IS_NOT_NON_EXCL(MTXB) \ +do { \ + ETHR_COMPILER_BARRIER; \ + if (ethr_atomic_read(&(MTXB)->non_exclusive)) \ + ethr_assert_failed(__FILE__, __LINE__, __func__,\ + "is not non-exclusive"); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_SET_NON_EXCL(MTXB) \ +do { \ + ETHR_COMPILER_BARRIER; \ + ethr_atomic_inc(&(MTXB)->non_exclusive); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_SET_NON_EXCL_NO(MTXB, NO) \ +do { \ + ETHR_COMPILER_BARRIER; \ + ethr_atomic_add(&(MTXB)->non_exclusive, (NO)); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_UNSET_NON_EXCL(MTXB) \ +do { \ + ETHR_COMPILER_BARRIER; \ + ethr_atomic_dec(&(MTXB)->non_exclusive); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +#else +# define ETHR_MTX_CHK_NON_EXCL_INIT__(MTXB) +# define ETHR_MTX_CHK_EXCL_IS_NON_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_IS_NOT_NON_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_SET_NON_EXCL_NO(MTXB, NO) +# define ETHR_MTX_CHK_EXCL_SET_NON_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_UNSET_NON_EXCL(MTXB) +#endif + +#else +# define ETHR_MTX_CHK_EXCL_INIT__(MTXB) +# define ETHR_MTX_CHK_EXCL_IS_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_IS_NOT_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_SET_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_UNSET_EXCL(MTXB) +# define ETHR_MTX_CHK_NON_EXCL_INIT__(MTXB) +# define ETHR_MTX_CHK_EXCL_IS_NON_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_IS_NOT_NON_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_SET_NON_EXCL_NO(MTXB, NO) +# define ETHR_MTX_CHK_EXCL_SET_NON_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_UNSET_NON_EXCL(MTXB) +#endif + +# define ETHR_MTX_CHK_EXCL_INIT(MTXB) \ +do { \ + ETHR_MTX_CHK_EXCL_INIT__((MTXB)); \ + ETHR_MTX_CHK_NON_EXCL_INIT__((MTXB)); \ +} while (0) + + #ifdef ETHR_USE_OWN_MTX_IMPL__ #define ETHR_MTX_DEFAULT_MAIN_SPINCOUNT_MAX 2000 @@ -365,12 +510,19 @@ ETHR_INLINE_FUNC_NAME_(ethr_mutex_trylock)(ethr_mutex *mtx) long act; int res; ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(mtx); act = ethr_atomic_cmpxchg_acqb(&mtx->mtxb.flgs, ETHR_RWMTX_W_FLG__, 0); res = (act == 0) ? 0 : EBUSY; +#ifdef ETHR_MTX_CHK_EXCL + if (res == 0) + ETHR_MTX_CHK_EXCL_SET_EXCL(&mtx->mtxb); +#endif + ETHR_MTX_HARD_DEBUG_LFS_TRYRWLOCK(&mtx->mtxb, res); ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(mtx); ETHR_COMPILER_BARRIER; return res; @@ -381,13 +533,17 @@ ETHR_INLINE_FUNC_NAME_(ethr_mutex_lock)(ethr_mutex *mtx) { long act; ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(mtx); act = ethr_atomic_cmpxchg_acqb(&mtx->mtxb.flgs, ETHR_RWMTX_W_FLG__, 0); if (act != 0) ethr_mutex_lock_wait__(mtx, act); + ETHR_MTX_CHK_EXCL_SET_EXCL(&mtx->mtxb); + ETHR_MTX_HARD_DEBUG_LFS_RWLOCK(&mtx->mtxb); ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(mtx); ETHR_COMPILER_BARRIER; } @@ -399,12 +555,16 @@ ETHR_INLINE_FUNC_NAME_(ethr_mutex_unlock)(ethr_mutex *mtx) ETHR_COMPILER_BARRIER; ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); ETHR_MTX_HARD_DEBUG_LFS_RWUNLOCK(&mtx->mtxb); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(mtx); + + ETHR_MTX_CHK_EXCL_UNSET_EXCL(&mtx->mtxb); act = ethr_atomic_cmpxchg_relb(&mtx->mtxb.flgs, 0, ETHR_RWMTX_W_FLG__); if (act != ETHR_RWMTX_W_FLG__) ethr_mutex_unlock_wake__(mtx, act); ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(mtx); } #endif /* ETHR_TRY_INLINE_FUNCS */ diff --git a/erts/include/internal/i386/atomic.h b/erts/include/internal/i386/atomic.h index f28258059f..52d01aab32 100644 --- a/erts/include/internal/i386/atomic.h +++ b/erts/include/internal/i386/atomic.h @@ -167,15 +167,52 @@ ethr_native_atomic_xchg(ethr_native_atomic_t *var, long val) * Atomic ops with at least specified barriers. */ -#define ethr_native_atomic_read_acqb ethr_native_atomic_read -#define ethr_native_atomic_inc_return_acqb ethr_native_atomic_inc_return +static ETHR_INLINE long +ethr_native_atomic_read_acqb(ethr_native_atomic_t *var) +{ + long val; +#if defined(__x86_64__) || !defined(ETHR_PRE_PENTIUM4_COMPAT) + val = var->counter; +#else + val = ethr_native_atomic_add_return(var, 0); +#endif + __asm__ __volatile__("" : : : "memory"); + return val; +} + +static ETHR_INLINE void +ethr_native_atomic_set_relb(ethr_native_atomic_t *var, long i) +{ + __asm__ __volatile__("" : : : "memory"); #if defined(__x86_64__) || !defined(ETHR_PRE_PENTIUM4_COMPAT) -#define ethr_native_atomic_set_relb ethr_native_atomic_set + var->counter = i; #else -#define ethr_native_atomic_set_relb ethr_native_atomic_xchg + (void) ethr_native_atomic_xchg(var, i); #endif -#define ethr_native_atomic_dec_relb ethr_native_atomic_dec -#define ethr_native_atomic_dec_return_relb ethr_native_atomic_dec_return +} + +static ETHR_INLINE long +ethr_native_atomic_inc_return_acqb(ethr_native_atomic_t *var) +{ + long res = ethr_native_atomic_inc_return(var); + __asm__ __volatile__("" : : : "memory"); + return res; +} + +static ETHR_INLINE void +ethr_native_atomic_dec_relb(ethr_native_atomic_t *var) +{ + __asm__ __volatile__("" : : : "memory"); + ethr_native_atomic_dec(var); +} + +static ETHR_INLINE long +ethr_native_atomic_dec_return_relb(ethr_native_atomic_t *var) +{ + __asm__ __volatile__("" : : : "memory"); + return ethr_native_atomic_dec_return(var); +} + #define ethr_native_atomic_cmpxchg_acqb ethr_native_atomic_cmpxchg #define ethr_native_atomic_cmpxchg_relb ethr_native_atomic_cmpxchg diff --git a/erts/include/internal/sparc32/atomic.h b/erts/include/internal/sparc32/atomic.h index 2a995d4465..2da6472393 100644 --- a/erts/include/internal/sparc32/atomic.h +++ b/erts/include/internal/sparc32/atomic.h @@ -176,38 +176,59 @@ ethr_native_atomic_cmpxchg(ethr_native_atomic_t *var, long new, long old) * Atomic ops with at least specified barriers. */ +/* TODO: relax acquire barriers */ + static ETHR_INLINE long ethr_native_atomic_read_acqb(ethr_native_atomic_t *var) { long res = ethr_native_atomic_read(var); - __asm__ __volatile__("membar #StoreLoad|#StoreStore"); + __asm__ __volatile__("membar #LoadLoad|#LoadStore|#StoreLoad|#StoreStore" : : : "memory"); return res; } static ETHR_INLINE void ethr_native_atomic_set_relb(ethr_native_atomic_t *var, long i) { - __asm__ __volatile__("membar #LoadStore|#StoreStore"); + __asm__ __volatile__("membar #LoadStore|#StoreStore" : : : "memory"); ethr_native_atomic_set(var, i); } +static ETHR_INLINE long +ethr_native_atomic_inc_return_acqb(ethr_native_atomic_t *var) +{ + long res = ethr_native_atomic_inc_return(var); + __asm__ __volatile__("membar #LoadLoad|#LoadStore" : : : "memory"); + return res; +} + static ETHR_INLINE void ethr_native_atomic_dec_relb(ethr_native_atomic_t *var) { - __asm__ __volatile__("membar #LoadStore|#StoreStore"); + __asm__ __volatile__("membar #LoadStore|#StoreStore" : : : "memory"); ethr_native_atomic_dec(var); } static ETHR_INLINE long ethr_native_atomic_dec_return_relb(ethr_native_atomic_t *var) { - __asm__ __volatile__("membar #LoadStore|#StoreStore"); + __asm__ __volatile__("membar #LoadStore|#StoreStore" : : : "memory"); return ethr_native_atomic_dec_return(var); } -#define ethr_native_atomic_inc_return_acqb ethr_native_atomic_inc_return -#define ethr_native_atomic_cmpxchg_acqb ethr_native_atomic_cmpxchg -#define ethr_native_atomic_cmpxchg_relb ethr_native_atomic_cmpxchg +static ETHR_INLINE long +ethr_native_atomic_cmpxchg_acqb(ethr_native_atomic_t *var, long new, long old) +{ + long res = ethr_native_atomic_cmpxchg(var, new, old); + __asm__ __volatile__("membar #LoadLoad|#LoadStore" : : : "memory"); + return res; +} + +static ETHR_INLINE long +ethr_native_atomic_cmpxchg_relb(ethr_native_atomic_t *var, long new, long old) +{ + __asm__ __volatile__("membar #LoadStore|#StoreStore" : : : "memory"); + return ethr_native_atomic_cmpxchg(var, new, old); +} #endif /* ETHR_TRY_INLINE_FUNCS */ diff --git a/erts/lib_src/common/ethr_mutex.c b/erts/lib_src/common/ethr_mutex.c index 78323b62a3..95fedc0ba2 100644 --- a/erts/lib_src/common/ethr_mutex.c +++ b/erts/lib_src/common/ethr_mutex.c @@ -205,9 +205,14 @@ static void hard_debug_chk_q__(struct ethr_mutex_base_ *, int); #ifdef ETHR_USE_OWN_RWMTX_IMPL__ static void +rwmutex_transfer_read_lock(ethr_rwmutex *rwmtx, + long initial, + int q_locked); +static void rwmutex_unlock_wake(ethr_rwmutex *rwmtx, int have_w, - long initial); + long initial, + int transfer_read_lock); static int rwmutex_try_complete_runlock(ethr_rwmutex *rwmtx, long initial, @@ -265,6 +270,8 @@ rwmutex_freqread_rdrs_inc(ethr_rwmutex *rwmtx, ethr_ts_event *tse) } } +#if 0 /* Not used */ + static ETHR_INLINE void rwmutex_freqread_rdrs_dec(ethr_rwmutex *rwmtx, ethr_ts_event *tse) { @@ -284,6 +291,8 @@ rwmutex_freqread_rdrs_dec(ethr_rwmutex *rwmtx, ethr_ts_event *tse) } } +#endif + static ETHR_INLINE long rwmutex_freqread_rdrs_dec_read(ethr_rwmutex *rwmtx, ethr_ts_event *tse) { @@ -334,7 +343,7 @@ rwmutex_freqread_rdrs_read(ethr_rwmutex *rwmtx, int ix) ETHR_ASSERT(res >= 0); break; case ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ: - ETHR_ASSERT(res == 0 || res == 1); + ETHR_ASSERT(ix == 0 ? res >= 0 : (res == 0 || res == 1)); break; default: ETHR_ASSERT(0); @@ -400,6 +409,7 @@ event_wait(struct ethr_mutex_base_ *mtxb, int locked = 0; long act; int need_try_complete_runlock = 0; + int transfer_read_lock = 0; /* Need to enqueue and wait... */ @@ -444,8 +454,8 @@ event_wait(struct ethr_mutex_base_ *mtxb, while (1) { long new, exp = act; - int freqread_tryrlock = 0; need_try_complete_runlock = 0; + transfer_read_lock = 0; if (type == ETHR_RWMTX_W_WAIT_FLG__) { if (is_freq_read && act == ETHR_RWMTX_R_FLG__) @@ -465,14 +475,11 @@ event_wait(struct ethr_mutex_base_ *mtxb, new = act + 1; /* Try to get it */ } else { - if (act & ~ETHR_RWMTX_R_FLG__) - new = act | ETHR_RWMTX_R_WAIT_FLG__; - else { /* Try to get it */ - ethr_rwmutex *rwmtx = (ethr_rwmutex *) mtxb; - rwmutex_freqread_rdrs_inc(rwmtx, tse); - ETHR_MEMORY_BARRIER; - new = act | ETHR_RWMTX_R_FLG__; - freqread_tryrlock = 1; + new = act | ETHR_RWMTX_R_WAIT_FLG__; + if ((act & (ETHR_RWMTX_W_FLG__ + | ETHR_RWMTX_W_WAIT_FLG__)) == 0) { + /* Transfer read lock to this thread. */ + transfer_read_lock = 1; } } } @@ -488,24 +495,6 @@ event_wait(struct ethr_mutex_base_ *mtxb, goto done; } } - - if (freqread_tryrlock) { - ethr_rwmutex *rwmtx = (ethr_rwmutex *) mtxb; - - /* We didn't set ETHR_RWMTX_R_FLG__, however someone - else might have */ - if (act == ETHR_RWMTX_R_FLG__) - goto done; /* Got it by help from someone else */ - - ETHR_ASSERT((act & ETHR_RWMTX_WAIT_FLGS__) == 0); - /* - * We know that no waiter flags have been set, i.e., - * we cannot get into a situation where we need to wake - * someone up here. Just restore the readers counter - * and do it over again... - */ - rwmutex_freqread_rdrs_dec(rwmtx, tse); - } } /* Enqueue */ @@ -535,20 +524,36 @@ event_wait(struct ethr_mutex_base_ *mtxb, /* Wait */ locked = 0; - ETHR_MTX_Q_UNLOCK(&mtxb->qlck); - if (need_try_complete_runlock) { + ETHR_ASSERT(!(transfer_read_lock && need_try_complete_runlock)); + + if (transfer_read_lock) { ETHR_ASSERT(((ethr_rwmutex *) mtxb)->type != ETHR_RWMUTEX_TYPE_NORMAL); /* - * We were the only one in queue when we enqueued, and it - * was seemingly read locked. We need to try to complete a - * runlock otherwise we might be hanging forever. If the - * runlock could be completed we will be dequeued and - * woken by ourselves. + * We are the only one in the queue and we are not write + * locked; rwmutex_transfer_read_lock() will: + * - transfer a read lock to us (since we're first in q) + * - unlock the Q-lock */ - rwmutex_try_complete_runlock((ethr_rwmutex *) mtxb, - act, tse, 0, 1, 0); + rwmutex_transfer_read_lock(((ethr_rwmutex *) mtxb), act, 1); + } + else { + ETHR_MTX_Q_UNLOCK(&mtxb->qlck); + + if (need_try_complete_runlock) { + ETHR_ASSERT(((ethr_rwmutex *) mtxb)->type + != ETHR_RWMUTEX_TYPE_NORMAL); + /* + * We were the only one in queue when we enqueued, and it + * was seemingly read locked. We need to try to complete a + * runlock otherwise we might be hanging forever. If the + * runlock could be completed we will be dequeued and + * woken by ourselves. + */ + rwmutex_try_complete_runlock((ethr_rwmutex *) mtxb, + act, tse, 0, 1, 0); + } } while (1) { @@ -653,8 +658,6 @@ write_lock_wait(struct ethr_mutex_base_ *mtxb, ethr_ts_event *tse = NULL; int until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; int res; - int freq_read_size = -1; - int freq_read_start_ix = -1; ETHR_ASSERT(!is_freq_read || is_rwmtx); @@ -666,44 +669,23 @@ write_lock_wait(struct ethr_mutex_base_ *mtxb, */ while (1) { - long exp; - while (act != 0) { if (is_freq_read && act == ETHR_RWMTX_R_FLG__) { ethr_rwmutex *rwmtx = (ethr_rwmutex *) mtxb; + scnt--; if (!tse) tse = ethr_get_ts_event(); - if (freq_read_size < 0) { - if (rwmtx->type == ETHR_RWMUTEX_TYPE_FREQUENT_READ) { - freq_read_size = reader_groups_array_size; - freq_read_start_ix = tse->rgix; - } - else { - freq_read_size = main_threads_array_size; - freq_read_start_ix = tse->mtix; - } - } - res = check_readers_array(rwmtx, - freq_read_start_ix, - freq_read_size); - scnt--; - if (res == 0) { - act = ethr_atomic_read(&mtxb->flgs); - if (act & ETHR_RWMTX_R_MASK__) { - res = rwmutex_try_complete_runlock(rwmtx, act, - tse, 0, 0, - 1); - if (res != EBUSY) - goto done; /* Got it */ - } - if (scnt <= 0) - goto chk_spin; - if (--until_yield == 0) { - until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; - ETHR_YIELD(); - } - continue; + res = rwmutex_try_complete_runlock(rwmtx, act, + tse, 0, 0, + 1); + if (res != EBUSY) + goto done; /* Got it */ + if (scnt <= 0) + goto chk_spin; + if (--until_yield == 0) { + until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; + ETHR_YIELD(); } } @@ -728,11 +710,9 @@ write_lock_wait(struct ethr_mutex_base_ *mtxb, scnt--; } - exp = act; - act = ethr_atomic_cmpxchg_acqb(&mtxb->flgs, ETHR_RWMTX_W_FLG__, - exp); + 0); if (act == 0) goto done; /* Got it */ } @@ -753,6 +733,7 @@ mtxb_init(struct ethr_mutex_base_ *mtxb, #ifdef ETHR_MTX_HARD_DEBUG_WSQ mtxb->ws = 0; #endif + ETHR_MTX_CHK_EXCL_INIT(mtxb); if (no_spin) { mtxb->main_scnt = 0; mtxb->aux_scnt = 0; @@ -1254,7 +1235,7 @@ ethr_cond_wait(ethr_cond *cnd, ethr_mutex *mtx) ETHR_MTX_HARD_DEBUG_LFS_RWLOCK(&mtx->mtxb); ETHR_MTX_HARD_DEBUG_FENCE_CHK(cnd); ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); - + ETHR_MTX_CHK_EXCL_SET_EXCL(&mtx->mtxb); tse->udata = udata; ethr_leave_ts_event(tse); return 0; @@ -1499,7 +1480,62 @@ int check_readers_array(ethr_rwmutex *rwmtx, return 0; } -static ETHR_INLINE void +static void +rwmutex_freqread_rdrs_dec_chk_wakeup(ethr_rwmutex *rwmtx, + ethr_ts_event *tse, + long initial) +{ + long act = initial; + + if ((act & (ETHR_RWMTX_W_FLG__| + ETHR_RWMTX_R_ABRT_UNLCK_FLG__)) == 0) { + if ((act & ETHR_RWMTX_WAIT_FLGS__) == 0) { + if (act & ETHR_RWMTX_R_PEND_UNLCK_MASK__) { + /* + * We *need* to try to complete the runlock. + * A writer that just enqueued (not seen by us + * in flag field) may depend on someone else + * completing the runlock. We just took over + * that responsibilty since we modified reader + * groups. + */ + rwmutex_try_complete_runlock(rwmtx, act, tse, 1, 0, 0); + } + } + else if ((act & ETHR_RWMTX_WAIT_FLGS__) == ETHR_RWMTX_R_WAIT_FLG__) + rwmutex_transfer_read_lock(rwmtx, act, 0); + else if ((act & ETHR_RWMTX_WAIT_FLGS__) == ETHR_RWMTX_W_WAIT_FLG__) + rwmutex_try_complete_runlock(rwmtx, act, tse, 1, 0, 0); + else { + /* + * Don't know if we got readers or writers + * first in queue; need to peek + */ + ETHR_MTX_Q_LOCK(&rwmtx->mtxb.qlck); + if (!rwmtx->mtxb.q) + ETHR_MTX_Q_UNLOCK(&rwmtx->mtxb.qlck); + else if (is_w_waiter(rwmtx->mtxb.q)) { + act = ethr_atomic_read(&rwmtx->mtxb.flgs); + ETHR_MTX_Q_UNLOCK(&rwmtx->mtxb.qlck); + if ((act & ETHR_RWMTX_W_FLG__) == 0) + rwmutex_try_complete_runlock(rwmtx, act, tse, 1, 0, 0); + } + else { + /* + * rwmutex_transfer_read_lock() will + * unlock Q lock. + */ + act = ethr_atomic_read(&rwmtx->mtxb.flgs); + if (act & ETHR_RWMTX_W_FLG__) + ETHR_MTX_Q_UNLOCK(&rwmtx->mtxb.qlck); + else + rwmutex_transfer_read_lock(rwmtx, act, 1); + } + } + } +} + +static void rwmutex_freqread_restore_failed_tryrlock(ethr_rwmutex *rwmtx, ethr_ts_event *tse) { @@ -1509,24 +1545,11 @@ rwmutex_freqread_restore_failed_tryrlock(ethr_rwmutex *rwmtx, */ act = rwmutex_freqread_rdrs_dec_read(rwmtx, tse); - ETHR_WRITE_MEMORY_BARRIER; + ETHR_MEMORY_BARRIER; if (act == 0) { - -#ifndef ETHR_WRITE_MEMORY_BARRIER_IS_FULL - ETHR_READ_MEMORY_BARRIER; -#endif - act = ethr_atomic_read(&rwmtx->mtxb.flgs); - - if ((act & ETHR_RWMTX_W_FLG__) == 0 - && act & (ETHR_RWMTX_WAIT_FLGS__|ETHR_RWMTX_R_PEND_UNLCK_MASK__)) { - /* - * We either got waiters, or someone else trying - * to read unlock which we might have to help. - */ - rwmutex_try_complete_runlock(rwmtx, act, tse, 1, 1, 0); - } + rwmutex_freqread_rdrs_dec_chk_wakeup(rwmtx, tse, act); } } @@ -1542,12 +1565,16 @@ rwmutex_try_complete_runlock(ethr_rwmutex *rwmtx, long act = initial; int six, res, length; + ETHR_ASSERT((act & ETHR_RWMTX_W_FLG__) == 0); + + if (act & ETHR_RWMTX_R_ABRT_UNLCK_FLG__) + return try_write_lock ? EBUSY : 0; + tse_tmp = tse; if (!tse_tmp) tse_tmp = ethr_get_ts_event(); - if ((act & ETHR_RWMTX_WAIT_FLGS__) - && (act & ~ETHR_RWMTX_WAIT_FLGS__) == 0) + if ((act & ETHR_RWMTX_WAIT_FLGS__) && (act & ~ETHR_RWMTX_WAIT_FLGS__) == 0) goto check_waiters; if (rwmtx->type == ETHR_RWMUTEX_TYPE_FREQUENT_READ) { @@ -1569,14 +1596,21 @@ rwmutex_try_complete_runlock(ethr_rwmutex *rwmtx, if (check_before_try) { res = check_readers_array(rwmtx, six, length); + + ETHR_MEMORY_BARRIER; + if (res == EBUSY) return try_write_lock ? EBUSY : 0; } + restart: + while (1) { long exp = act; long new = act+1; + ETHR_ASSERT((act & ETHR_RWMTX_R_ABRT_UNLCK_FLG__) == 0); + ETHR_ASSERT((act & ETHR_RWMTX_R_PEND_UNLCK_MASK__) < ETHR_RWMTX_R_PEND_UNLCK_MASK__); @@ -1585,8 +1619,10 @@ rwmutex_try_complete_runlock(ethr_rwmutex *rwmtx, act = new; break; } + if (!try_write_lock) { - if (act == ETHR_RWMTX_W_FLG__ || act == 0) + if (act == 0 || (act & (ETHR_RWMTX_W_FLG__ + | ETHR_RWMTX_R_ABRT_UNLCK_FLG__))) return 0; if ((act & ETHR_RWMTX_WAIT_FLGS__) == 0) { if ((act & ETHR_RWMTX_R_FLG__) == 0) @@ -1601,33 +1637,50 @@ rwmutex_try_complete_runlock(ethr_rwmutex *rwmtx, else { if (act == 0) goto tryrwlock; - if (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_WAIT_FLGS__)) + if (act & (ETHR_RWMTX_W_FLG__ + | ETHR_RWMTX_R_ABRT_UNLCK_FLG__)) return EBUSY; } } res = check_readers_array(rwmtx, six, length); - if (res == EBUSY) { - act = ethr_atomic_dec_read(&rwmtx->mtxb.flgs); - if (act & ETHR_RWMTX_R_MASK__) - return try_write_lock ? EBUSY : 0; - } - else { - while (1) { - long exp = act; - long new = act; - new &= ~ETHR_RWMTX_R_FLG__; - new--; - ETHR_ASSERT(act & ETHR_RWMTX_R_PEND_UNLCK_MASK__); + ETHR_MEMORY_BARRIER; - act = ethr_atomic_cmpxchg(&rwmtx->mtxb.flgs, new, exp); - if (exp == act) { - if (new & ETHR_RWMTX_R_PEND_UNLCK_MASK__) - return try_write_lock ? EBUSY : 0; - act = new; - break; + ETHR_ASSERT((act & ETHR_RWMTX_W_FLG__) == 0); + + while (1) { + int finished_abort = 0; + long exp = act; + long new = act; + + new--; + if (act & ETHR_RWMTX_R_ABRT_UNLCK_FLG__) { + if ((new & ETHR_RWMTX_R_PEND_UNLCK_MASK__) == 0) { + new &= ~ETHR_RWMTX_R_ABRT_UNLCK_FLG__; + finished_abort = 1; } + ETHR_ASSERT(act & ETHR_RWMTX_R_FLG__); + } + else if ((act & ETHR_RWMTX_R_FLG__) && res != EBUSY) { + new &= ~ETHR_RWMTX_R_FLG__; + } + + ETHR_ASSERT(act & ETHR_RWMTX_R_PEND_UNLCK_MASK__); + + act = ethr_atomic_cmpxchg(&rwmtx->mtxb.flgs, new, exp); + if (exp == act) { + act = new; + if (act & ETHR_RWMTX_W_FLG__) + return try_write_lock ? EBUSY : 0; + if (finished_abort && (act & ETHR_RWMTX_WAIT_FLGS__)) + goto restart; + if (act & (ETHR_RWMTX_R_FLG__ + | ETHR_RWMTX_R_ABRT_UNLCK_FLG__ + | ETHR_RWMTX_R_PEND_UNLCK_MASK__)) + return try_write_lock ? EBUSY : 0; + /* Read unlock completed */ + break; } } @@ -1637,12 +1690,9 @@ rwmutex_try_complete_runlock(ethr_rwmutex *rwmtx, * to write lock it). */ - if (act & ETHR_RWMTX_W_FLG__) - return try_write_lock ? EBUSY : 0; - if (act & ETHR_RWMTX_WAIT_FLGS__) { check_waiters: - rwmutex_unlock_wake(rwmtx, 0, act); + rwmutex_unlock_wake(rwmtx, 0, act, 0); return try_write_lock ? EBUSY : 0; } @@ -1670,7 +1720,7 @@ rwmutex_incdec_restore_failed_tryrlock(ethr_rwmutex *rwmtx) act = ethr_atomic_dec_read(&rwmtx->mtxb.flgs); if ((act & ETHR_RWMTX_WAIT_FLGS__) && (act & ~ETHR_RWMTX_WAIT_FLGS__) == 0) { - rwmutex_unlock_wake(rwmtx, 0, act); + rwmutex_unlock_wake(rwmtx, 0, act, 0); } } @@ -1700,7 +1750,7 @@ rwmutex_normal_rlock_wait(ethr_rwmutex *rwmtx, #endif while (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)) { - if (scnt >= 0) { + if (scnt <= 0) { tse = ethr_get_ts_event(); if (update_spincount(&rwmtx->mtxb, tse, &start_scnt, &scnt)) { event_wait(&rwmtx->mtxb, tse, scnt, @@ -1736,10 +1786,83 @@ rwmutex_normal_rlock_wait(ethr_rwmutex *rwmtx, static void rwmutex_freqread_rlock_wait(ethr_rwmutex *rwmtx, - ethr_ts_event *tse, - long initial) + ethr_ts_event *tse); + +static int +rwmutex_freqread_rlock(ethr_rwmutex *rwmtx, ethr_ts_event *tse, int trylock) { - long act = initial; + int res = 0; + long act; + + rwmutex_freqread_rdrs_inc(rwmtx, tse); + + ETHR_MEMORY_BARRIER; + + act = ethr_atomic_read_acqb(&rwmtx->mtxb.flgs); + + if (act != ETHR_RWMTX_R_FLG__) { + int wake_other_readers; + + while (1) { + long exp, new; + + wake_other_readers = 0; + + if (act == 0) + new = act | ETHR_RWMTX_R_FLG__; + else if (act == ETHR_RWMTX_R_FLG__) + break; /* Got it */ + else if (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)) { + rwmutex_freqread_restore_failed_tryrlock(rwmtx, tse); + if (trylock) + res = EBUSY; + else + rwmutex_freqread_rlock_wait(rwmtx, tse); + break; + } + else if (act & ETHR_RWMTX_R_ABRT_UNLCK_FLG__) { + if ((act & ETHR_RWMTX_R_FLG__) == 0) + ETHR_FATAL_ERROR__(EFAULT); + /* + * An aborted runlock, not write locked, and no write + * waiters, i.e., we got it... + */ + if (act & ETHR_RWMTX_R_WAIT_FLG__) + wake_other_readers = 1; + break; + } + else { + new = act | ETHR_RWMTX_R_FLG__; + if (act & ETHR_RWMTX_R_PEND_UNLCK_MASK__) { + /* + * Someone is doing tryrwlock (no writer and no + * write waiters); we will try to abort that... + */ + new |= ETHR_RWMTX_R_ABRT_UNLCK_FLG__; + } + + if (act & ETHR_RWMTX_R_WAIT_FLG__) + wake_other_readers = 1; + } + + exp = act; + act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, new, exp); + if (act == exp) + break; + } + + if (wake_other_readers) + rwmutex_transfer_read_lock(rwmtx, act, 0); + } + + return res; +} + +static void +rwmutex_freqread_rlock_wait(ethr_rwmutex *rwmtx, + ethr_ts_event *tse) +{ + long act; int scnt, start_scnt; int until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; @@ -1752,12 +1875,10 @@ rwmutex_freqread_rlock_wait(ethr_rwmutex *rwmtx, while (1) { - rwmutex_freqread_restore_failed_tryrlock(rwmtx, tse); - act = ethr_atomic_read(&rwmtx->mtxb.flgs); - while (act & ~(ETHR_RWMTX_R_FLG__|ETHR_RWMTX_R_WAIT_FLG__)) { - if (scnt >= 0) { + while (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)) { + if (scnt <= 0) { if (update_spincount(&rwmtx->mtxb, tse, &start_scnt, &scnt)) { event_wait(&rwmtx->mtxb, tse, scnt, ETHR_RWMTX_R_WAIT_FLG__, 1, 1); @@ -1773,30 +1894,8 @@ rwmutex_freqread_rlock_wait(ethr_rwmutex *rwmtx, scnt--; } - rwmutex_freqread_rdrs_inc(rwmtx, tse); - - ETHR_MEMORY_BARRIER; - - act = ethr_atomic_read(&rwmtx->mtxb.flgs); - - if (act == ETHR_RWMTX_R_FLG__) - return; /* Got it */ - - while (1) { - long exp, new; - - if (act & ~(ETHR_RWMTX_R_FLG__|ETHR_RWMTX_R_WAIT_FLG__)) - break; /* Busy (need to restore inc) */ - - if (act & ETHR_RWMTX_R_FLG__) - return; /* Got it */ - - exp = act; - new = act | ETHR_RWMTX_R_FLG__; - act = ethr_atomic_cmpxchg(&rwmtx->mtxb.flgs, new, exp); - if (act == exp) - return; /* Got it */ - } + if (rwmutex_freqread_rlock(rwmtx, tse, 1) != EBUSY) + break; /* Got it */ } } @@ -1813,17 +1912,26 @@ rwmutex_freqread_rwlock_wait(ethr_rwmutex *rwmtx, long initial) } static ETHR_INLINE void -rwlock_wake_set_flags(ethr_rwmutex *rwmtx, long new_initial, int act_initial) +rwlock_wake_set_flags(ethr_rwmutex *rwmtx, long new_initial, long act_initial) { long act, act_mask; + int chk_abrt_flg; + + ETHR_MEMORY_BARRIER; + if (rwmtx->type != ETHR_RWMUTEX_TYPE_NORMAL) { /* r pend unlock mask may vary and must be retained */ act_mask = ETHR_RWMTX_R_PEND_UNLCK_MASK__; + if (new_initial & ETHR_RWMTX_R_FLG__) + chk_abrt_flg = 1; + else + chk_abrt_flg = 0; } else { #ifdef ETHR_RLOCK_WITH_INC_DEC /* rs mask may vary and must be retained */ act_mask = ETHR_RWMTX_RS_MASK__; + chk_abrt_flg = 0; #else /* rs mask always zero */ ETHR_ASSERT((act_initial & ETHR_RWMTX_RS_MASK__) == 0); @@ -1836,6 +1944,8 @@ rwlock_wake_set_flags(ethr_rwmutex *rwmtx, long new_initial, int act_initial) while (1) { long exp = act; long new = new_initial + (act & act_mask); + if (chk_abrt_flg && (act & act_mask)) + new |= ETHR_RWMTX_R_ABRT_UNLCK_FLG__; act = ethr_atomic_cmpxchg(&rwmtx->mtxb.flgs, new, exp); if (act == exp) break; @@ -1883,6 +1993,14 @@ dbg_unlock_wake(ethr_rwmutex *rwmtx, exp |= ETHR_RWMTX_R_WAIT_FLG__; if (rwmtx->rq_end->next != rwmtx->mtxb.q) exp |= ETHR_RWMTX_W_WAIT_FLG__; + else if (exp == ETHR_RWMTX_R_WAIT_FLG__) { + if (!have_w) { + if (rwmtx->type != ETHR_RWMUTEX_TYPE_NORMAL) + imask |= ETHR_RWMTX_R_FLG__; + else + imask |= ETHR_RWMTX_RS_MASK__; + } + } act = ethr_atomic_read(&rwmtx->mtxb.flgs); ETHR_ASSERT((exp & ~imask) == (act & ~imask)); @@ -1894,41 +2012,83 @@ dbg_unlock_wake(ethr_rwmutex *rwmtx, #endif static void -rwmutex_unlock_wake(ethr_rwmutex *rwmtx, int have_w, long initial) +rwmutex_transfer_read_lock(ethr_rwmutex *rwmtx, long initial, int q_locked) +{ + long act = initial; + + if (!q_locked) { + ethr_ts_event *tse; + ETHR_ASSERT(initial & ETHR_RWMTX_R_WAIT_FLG__); + ETHR_ASSERT((initial & ETHR_RWMTX_W_FLG__) == 0); + ETHR_MTX_Q_LOCK(&rwmtx->mtxb.qlck); + + act = ethr_atomic_read(&rwmtx->mtxb.flgs); + tse = rwmtx->mtxb.q; + if ((act & ETHR_RWMTX_W_FLG__) || !tse || is_w_waiter(tse)) { + /* Someone else woke the readers up... */ + ETHR_MTX_Q_UNLOCK(&rwmtx->mtxb.qlck); + return; + } + } + + rwmutex_unlock_wake(rwmtx, 0, initial, 1); +} + +static void +rwmutex_unlock_wake(ethr_rwmutex *rwmtx, int have_w, long initial, + int transfer_read_lock) { long new, act = initial; ethr_ts_event *tse; - if ((act & ETHR_RWMTX_WAIT_FLGS__) == 0) { - if (!have_w) - return; - else { - while ((act & ETHR_RWMTX_WAIT_FLGS__) == 0) { - long exp = act; - new = exp & ~ETHR_RWMTX_W_FLG__; - act = ethr_atomic_cmpxchg(&rwmtx->mtxb.flgs, new, exp); - if (act == exp) - return; + if (transfer_read_lock) { + /* + * - Q already locked + * - Got R waiters first in Q + * - Not W locked + */ + tse = rwmtx->mtxb.q; + + ETHR_ASSERT(act & ETHR_RWMTX_R_WAIT_FLG__); + ETHR_ASSERT((act & (ETHR_RWMTX_W_FLG__)) == 0); + ETHR_ASSERT(tse && !is_w_waiter(tse)); + } + else { + + if ((act & ETHR_RWMTX_WAIT_FLGS__) == 0) { + if (!have_w) + return; + else { + while ((act & ETHR_RWMTX_WAIT_FLGS__) == 0) { + long exp = act; + new = exp & ~ETHR_RWMTX_W_FLG__; + act = ethr_atomic_cmpxchg(&rwmtx->mtxb.flgs, new, exp); + if (act == exp) + return; + } } } - } - ETHR_MTX_Q_LOCK(&rwmtx->mtxb.qlck); - tse = rwmtx->mtxb.q; + ETHR_MTX_Q_LOCK(&rwmtx->mtxb.qlck); + tse = rwmtx->mtxb.q; - if (!have_w) { - if (!tse) { + if (!have_w) { + if (!tse) { #ifdef ETHR_DEBUG - act = ethr_atomic_read(&rwmtx->mtxb.flgs); - ETHR_ASSERT((act & ETHR_RWMTX_WAIT_FLGS__) == 0); + act = ethr_atomic_read(&rwmtx->mtxb.flgs); + ETHR_ASSERT((act & ETHR_RWMTX_WAIT_FLGS__) == 0); #endif - goto already_served; - } - act = ethr_atomic_read(&rwmtx->mtxb.flgs); - if (act & ~ETHR_RWMTX_WAIT_FLGS__) { - already_served: - ETHR_MTX_Q_UNLOCK(&rwmtx->mtxb.qlck); - return; + goto already_served; + } + act = ethr_atomic_read(&rwmtx->mtxb.flgs); + if (act == (ETHR_RWMTX_R_WAIT_FLG__|ETHR_RWMTX_R_FLG__)) { + ETHR_ASSERT(tse && !is_w_waiter(tse)); + } + else if (act & ~ETHR_RWMTX_WAIT_FLGS__) { + already_served: + ETHR_MTX_Q_UNLOCK(&rwmtx->mtxb.qlck); + return; + } } } @@ -1988,6 +2148,7 @@ rwmutex_unlock_wake(ethr_rwmutex *rwmtx, int have_w, long initial) rwmutex_freqread_rdrs_add(rwmtx, type, ix, wrs); } } + new = ETHR_RWMTX_R_FLG__; } @@ -1995,6 +2156,7 @@ rwmutex_unlock_wake(ethr_rwmutex *rwmtx, int have_w, long initial) new |= ETHR_RWMTX_W_WAIT_FLG__; rwlock_wake_set_flags(rwmtx, new, act); + wake_readers(rwmtx, rs); } } @@ -2157,6 +2319,7 @@ ethr_rwmutex_destroy(ethr_rwmutex *rwmtx) return EINVAL; } #endif + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); if (rwmtx->type != ETHR_RWMUTEX_TYPE_NORMAL) { long act = ethr_atomic_read(&rwmtx->mtxb.flgs); if (act == ETHR_RWMTX_R_FLG__) @@ -2187,6 +2350,8 @@ ethr_rwmutex_tryrlock(ethr_rwmutex *rwmtx) ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + switch (rwmtx->type) { case ETHR_RWMUTEX_TYPE_NORMAL: { #ifdef ETHR_RLOCK_WITH_INC_DEC @@ -2225,42 +2390,23 @@ ethr_rwmutex_tryrlock(ethr_rwmutex *rwmtx) case ETHR_RWMUTEX_TYPE_FREQUENT_READ: case ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ: { ethr_ts_event *tse = ethr_get_ts_event(); - - rwmutex_freqread_rdrs_inc(rwmtx, tse); - - ETHR_MEMORY_BARRIER; - - act = ethr_atomic_read_acqb(&rwmtx->mtxb.flgs); - - if (act != ETHR_RWMTX_R_FLG__) { - while (1) { - long exp, new; - - if (act & ~(ETHR_RWMTX_R_FLG__|ETHR_RWMTX_R_WAIT_FLG__)) { - rwmutex_freqread_restore_failed_tryrlock(rwmtx, tse); - res = EBUSY; - break; - } - - if (act & ETHR_RWMTX_R_FLG__) - break; - - exp = act; - new = act | ETHR_RWMTX_R_FLG__; - act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, new, exp); - if (act == exp) - break; - } - } - + res = rwmutex_freqread_rlock(rwmtx, tse, 1); ethr_leave_ts_event(tse); break; } } +#ifdef ETHR_MTX_CHK_EXCL + if (res == 0) { + ETHR_MTX_CHK_EXCL_SET_NON_EXCL(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_IS_NOT_EXCL(&rwmtx->mtxb); + } +#endif + ETHR_MTX_HARD_DEBUG_LFS_TRYRLOCK(&rwmtx->mtxb, res); ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); return res; } @@ -2275,6 +2421,8 @@ ethr_rwmutex_rlock(ethr_rwmutex *rwmtx) ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + switch (rwmtx->type) { case ETHR_RWMUTEX_TYPE_NORMAL: { #ifdef ETHR_RLOCK_WITH_INC_DEC @@ -2286,9 +2434,8 @@ ethr_rwmutex_rlock(ethr_rwmutex *rwmtx) while (1) { act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, exp+1, exp); - if (act == exp) { + if (act == exp) break; - } if (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)) { rwmutex_normal_rlock_wait(rwmtx, act); @@ -2303,38 +2450,15 @@ ethr_rwmutex_rlock(ethr_rwmutex *rwmtx) case ETHR_RWMUTEX_TYPE_FREQUENT_READ: case ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ: { ethr_ts_event *tse = ethr_get_ts_event(); - - rwmutex_freqread_rdrs_inc(rwmtx, tse); - - ETHR_MEMORY_BARRIER; - - act = ethr_atomic_read_acqb(&rwmtx->mtxb.flgs); - - if (act != ETHR_RWMTX_R_FLG__) { - while (1) { - long exp, new; - - if (act & ~(ETHR_RWMTX_R_FLG__|ETHR_RWMTX_R_WAIT_FLG__)) { - rwmutex_freqread_rlock_wait(rwmtx, tse, act); - break; - } - - if (act & ETHR_RWMTX_R_FLG__) - break; - - exp = act; - new = act | ETHR_RWMTX_R_FLG__; - act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, new, exp); - if (act == exp) - break; - } - } - + rwmutex_freqread_rlock(rwmtx, tse, 0); ethr_leave_ts_event(tse); break; } } + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + ETHR_MTX_CHK_EXCL_SET_NON_EXCL(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_IS_NOT_EXCL(&rwmtx->mtxb); ETHR_MTX_HARD_DEBUG_LFS_RLOCK(&rwmtx->mtxb); ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); } @@ -2344,6 +2468,8 @@ ethr_rwmutex_runlock(ethr_rwmutex *rwmtx) { long act; + ETHR_MTX_CHK_EXCL_IS_NOT_EXCL(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_UNSET_NON_EXCL(&rwmtx->mtxb); ETHR_ASSERT(!ethr_not_inited__); ETHR_ASSERT(rwmtx); ETHR_ASSERT(rwmtx->initialized == ETHR_RWMUTEX_INITIALIZED); @@ -2351,13 +2477,15 @@ ethr_rwmutex_runlock(ethr_rwmutex *rwmtx) ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); ETHR_MTX_HARD_DEBUG_LFS_RUNLOCK(&rwmtx->mtxb); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + switch (rwmtx->type) { case ETHR_RWMUTEX_TYPE_NORMAL: act = ethr_atomic_dec_read_relb(&rwmtx->mtxb.flgs); if ((act & ETHR_RWMTX_WAIT_FLGS__) && (act & ~ETHR_RWMTX_WAIT_FLGS__) == 0) { ETHR_ASSERT((act & ETHR_RWMTX_W_FLG__) == 0); - rwmutex_unlock_wake(rwmtx, 0, act); + rwmutex_unlock_wake(rwmtx, 0, act, 0); } break; @@ -2369,21 +2497,12 @@ ethr_rwmutex_runlock(ethr_rwmutex *rwmtx) ETHR_ASSERT(act >= 0); - ETHR_WRITE_MEMORY_BARRIER; + ETHR_MEMORY_BARRIER; if (act == 0) { - -#ifndef ETHR_WRITE_MEMORY_BARRIER_IS_FULL - ETHR_READ_MEMORY_BARRIER; -#endif act = ethr_atomic_read(&rwmtx->mtxb.flgs); - - if ((act & ETHR_RWMTX_W_FLG__) == 0 - && (act & (ETHR_RWMTX_WAIT_FLGS__ - | ETHR_RWMTX_R_PEND_UNLCK_MASK__))) { - rwmutex_try_complete_runlock(rwmtx, act, tse, 1, 0, 0); - } - + if (act != ETHR_RWMTX_R_FLG__) + rwmutex_freqread_rdrs_dec_chk_wakeup(rwmtx, tse, act); } ethr_leave_ts_event(tse); @@ -2391,6 +2510,7 @@ ethr_rwmutex_runlock(ethr_rwmutex *rwmtx) } } + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); } @@ -2406,6 +2526,8 @@ ethr_rwmutex_tryrwlock(ethr_rwmutex *rwmtx) ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + switch (rwmtx->type) { case ETHR_RWMUTEX_TYPE_NORMAL: act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, @@ -2422,25 +2544,32 @@ ethr_rwmutex_tryrwlock(ethr_rwmutex *rwmtx) do { - if (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_WAIT_FLGS__)) { - res = EBUSY; - break; - } - - if (act & ETHR_RWMTX_R_MASK__) { + if (act == 0) + act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, + ETHR_RWMTX_W_FLG__, 0); + else if (act == ETHR_RWMTX_R_FLG__) { res = rwmutex_try_complete_runlock(rwmtx, act, NULL, 0, 1, 1); break; } - - act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, - ETHR_RWMTX_W_FLG__, 0); + else { + res = EBUSY; + break; + } } while (act != 0); break; } +#ifdef ETHR_MTX_CHK_EXCL + if (res == 0) { + ETHR_MTX_CHK_EXCL_SET_EXCL(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_IS_NOT_NON_EXCL(&rwmtx->mtxb); + } +#endif + + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); ETHR_MTX_HARD_DEBUG_LFS_TRYRWLOCK(&rwmtx->mtxb, res); ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); @@ -2457,6 +2586,8 @@ ethr_rwmutex_rwlock(ethr_rwmutex *rwmtx) ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + switch (rwmtx->type) { case ETHR_RWMUTEX_TYPE_NORMAL: act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, @@ -2485,8 +2616,11 @@ ethr_rwmutex_rwlock(ethr_rwmutex *rwmtx) break; } + ETHR_MTX_CHK_EXCL_SET_EXCL(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_IS_NOT_NON_EXCL(&rwmtx->mtxb); ETHR_MTX_HARD_DEBUG_LFS_RWLOCK(&rwmtx->mtxb); ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); } @@ -2501,12 +2635,17 @@ ethr_rwmutex_rwunlock(ethr_rwmutex *rwmtx) ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); ETHR_MTX_HARD_DEBUG_LFS_RWUNLOCK(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_IS_NOT_NON_EXCL(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_UNSET_EXCL(&rwmtx->mtxb); + + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + switch (rwmtx->type) { case ETHR_RWMUTEX_TYPE_NORMAL: act = ethr_atomic_cmpxchg_relb(&rwmtx->mtxb.flgs, 0, ETHR_RWMTX_W_FLG__); if (act != ETHR_RWMTX_W_FLG__) - rwmutex_unlock_wake(rwmtx, 1, act); + rwmutex_unlock_wake(rwmtx, 1, act, 0); break; case ETHR_RWMUTEX_TYPE_FREQUENT_READ: @@ -2514,11 +2653,12 @@ ethr_rwmutex_rwunlock(ethr_rwmutex *rwmtx) act = ethr_atomic_cmpxchg_relb(&rwmtx->mtxb.flgs, 0, ETHR_RWMTX_W_FLG__); if (act != ETHR_RWMTX_W_FLG__) - rwmutex_unlock_wake(rwmtx, 1, act); + rwmutex_unlock_wake(rwmtx, 1, act, 0); break; } ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); } #else diff --git a/erts/test/ethread_SUITE.erl b/erts/test/ethread_SUITE.erl index 0cc315e9be..93e27fa8d3 100644 --- a/erts/test/ethread_SUITE.erl +++ b/erts/test/ethread_SUITE.erl @@ -171,7 +171,15 @@ max_threads(doc) -> max_threads(suite) -> []; max_threads(Config) -> - run_case(Config, "max_threads", ""). + case {os:type(), os:version()} of + {{unix,darwin}, {9, _, _}} -> + %% For some reason pthread_create() crashes when more + %% threads cannot be created, instead of returning an + %% error code on our MacOS X Leopard machine... + {skipped, "MacOS X Leopard cannot cope with this test..."}; + _ -> + run_case(Config, "max_threads", "") + end. tsd(doc) -> ["Tests thread specific data."]; diff --git a/lib/asn1/src/asn1rt_driver_handler.erl b/lib/asn1/src/asn1rt_driver_handler.erl index c95b243ae0..cc2b501e16 100644 --- a/lib/asn1/src/asn1rt_driver_handler.erl +++ b/lib/asn1/src/asn1rt_driver_handler.erl @@ -71,7 +71,10 @@ load_driver(Reason) -> end. init(FromPid,FromRef) -> - register(asn1_driver_owner,self()), + case catch register(asn1_driver_owner,self()) of + true -> true; + _Other -> exit(normal) + end, Dir = filename:join([code:priv_dir(asn1),"lib"]), case catch erl_ddll:load_driver(Dir,asn1_erl_drv) of ok -> diff --git a/lib/compiler/src/beam_utils.erl b/lib/compiler/src/beam_utils.erl index 761d4ffec0..45cdf8a659 100644 --- a/lib/compiler/src/beam_utils.erl +++ b/lib/compiler/src/beam_utils.erl @@ -407,16 +407,23 @@ check_liveness(R, [{bif,Op,{f,Fail},Ss,D}|Is], St0) -> Other -> Other end; -check_liveness(R, [{gc_bif,Op,{f,Fail},_,Ss,D}|Is], St0) -> - case check_liveness_fail(R, Op, Ss, Fail, St0) of - {killed,St} = Killed -> - case member(R, Ss) of - true -> {used,St}; - false when R =:= D -> Killed; - false -> check_liveness(R, Is, St) - end; - Other -> - Other +check_liveness(R, [{gc_bif,Op,{f,Fail},Live,Ss,D}|Is], St0) -> + case R of + {x,X} when X >= Live -> + {killed,St0}; + {x,_} -> + {used,St0}; + _ -> + case check_liveness_fail(R, Op, Ss, Fail, St0) of + {killed,St}=Killed -> + case member(R, Ss) of + true -> {used,St}; + false when R =:= D -> Killed; + false -> check_liveness(R, Is, St) + end; + Other -> + Other + end end; check_liveness(R, [{bs_add,{f,0},Ss,D}|Is], St) -> case member(R, Ss) of @@ -482,10 +489,13 @@ check_liveness(R, [{bs_context_to_binary,S}|Is], St) -> S -> {used,St}; _ -> check_liveness(R, Is, St) end; -check_liveness(R, [{loop_rec,{f,_},{x,0}}|Is], St) -> +check_liveness(R, [{loop_rec,{f,_},{x,0}}|_], St) -> case R of - {x,_} -> {killed,St}; - _ -> check_liveness(R, Is, St) + {x,_} -> + {killed,St}; + _ -> + %% y register. Rarely happens. Be very conversative. + {unknown,St} end; check_liveness(R, [{loop_rec_end,{f,Fail}}|_], St) -> check_liveness_at(R, Fail, St); diff --git a/lib/compiler/test/compilation_SUITE.erl b/lib/compiler/test/compilation_SUITE.erl index 9c06740816..935e384d2d 100644 --- a/lib/compiler/test/compilation_SUITE.erl +++ b/lib/compiler/test/compilation_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -46,7 +46,7 @@ all(suite) -> trycatch_4, opt_crash, otp_5404,otp_5436,otp_5481,otp_5553,otp_5632, otp_5714,otp_5872,otp_6121,otp_6121a,otp_6121b, - otp_7202,otp_7345,on_load,string_table + otp_7202,otp_7345,on_load,string_table,otp_8949_a,otp_8949_a ]. -define(comp(N), @@ -606,5 +606,45 @@ string_table(Config) when is_list(Config) -> ?line {"StrT", <<"stringabletringtable">>} = StringTableChunk, ok. +otp_8949_a(Config) when is_list(Config) -> + value = otp_8949_a(), + ok. + +-record(cs, {exs,keys = [],flags = 1}). +-record(exs, {children = []}). + +otp_8949_a() -> + case id([#cs{}]) of + [#cs{}=Cs] -> + SomeVar = id(value), + if + Cs#cs.flags band 1 =/= 0 -> + id(SomeVar); + (((Cs#cs.exs)#exs.children /= []) + and + (Cs#cs.flags band (1 bsl 0 bor (1 bsl 22)) == 0)); + Cs#cs.flags band (1 bsl 22) =/= 0 -> + ok + end + end. + +otp_8949_b(Config) when is_list(Config) -> + self() ! something, + ?line value = otp_8949_b([], false), + ?line {'EXIT',_} = (catch otp_8949_b([], true)), + ok. + +%% Would cause an endless loop in beam_utils. +otp_8949_b(A, B) -> + Var = id(value), + if + A == [], B == false -> + ok + end, + receive + something -> + id(Var) + end. + id(I) -> I. diff --git a/lib/debugger/test/int_eval_SUITE.erl b/lib/debugger/test/int_eval_SUITE.erl index 19b006e750..1628ca69b1 100644 --- a/lib/debugger/test/int_eval_SUITE.erl +++ b/lib/debugger/test/int_eval_SUITE.erl @@ -65,10 +65,7 @@ bifs_outside_erlang(Config) when is_list(Config) -> Self = self(), ok = io:format("Self: ~p", [Self]), Info = ets:info(Id), - {owner,Self} = lists:nth(2, Info), - %% Was - %% {owner,Self} = element(2, Info), - %% in R10B. + Self = proplists:get_value(owner, Info), ?IM:ets_delete(Id), ok end, diff --git a/lib/dialyzer/doc/manual.txt b/lib/dialyzer/doc/manual.txt index 470ddd6c73..d9cb52f722 100644 --- a/lib/dialyzer/doc/manual.txt +++ b/lib/dialyzer/doc/manual.txt @@ -123,9 +123,10 @@ The exit status of the command line version is: Usage: dialyzer [--help] [--version] [--shell] [--quiet] [--verbose] - [-pa dir]* [--plt plt] [-Ddefine]* [-I include_dir]* - [--output_plt file] [-Wwarn]* [--src] [--gui | --wx] - [files_or_dirs] [-r dirs] [--apps applications] [-o outfile] + [-pa dir]* [--plt plt] [--plts plts] [-Ddefine]* + [-I include_dir]* [--output_plt file] [-Wwarn]* + [--src] [--gui | --wx] [files_or_dirs] [-r dirs] + [--apps applications] [-o outfile] [--build_plt] [--add_to_plt] [--remove_from_plt] [--check_plt] [--no_check_plt] [--plt_info] [--get_warnings] [--no_native] @@ -167,6 +168,10 @@ Options: --plt plt Use the specified plt as the initial plt (if the plt was built during setup the files will be checked for consistency) + --plts plts + Merges the specified plts to create the initial plt -- requires + that the plts are disjoint (i.e., do not have any module + appearing in more than one plt) -Wwarn A family of options which selectively turn on/off warnings (for help on the names of warnings use dialyzer -Whelp) @@ -294,6 +299,7 @@ Option :: {files, [Filename :: string()]} | {defines, [{Macro :: atom(), Value :: term()}]} | {from, src_code | byte_code} %% Defaults to byte_code | {init_plt, FileName :: string()} %% If changed from default + | {plts, [FileName :: string()]} %% If changed from default | {include_dirs, [DirName :: string()]} | {output_file, FileName :: string()} | {output_plt, FileName :: string()} diff --git a/lib/dialyzer/src/dialyzer.erl b/lib/dialyzer/src/dialyzer.erl index b4161ea194..471f9fccd2 100644 --- a/lib/dialyzer/src/dialyzer.erl +++ b/lib/dialyzer/src/dialyzer.erl @@ -106,27 +106,35 @@ cl_print_plt_info(Opts) -> end, doit(F). -print_plt_info(#options{init_plt = PLT, output_file = OutputFile}) -> +print_plt_info(#options{init_plts = PLTs, output_file = OutputFile}) -> + PLTInfo = get_plt_info(PLTs), + do_print_plt_info(PLTInfo, OutputFile). + +get_plt_info([PLT|PLTs]) -> String = case dialyzer_plt:included_files(PLT) of {ok, Files} -> - io_lib:format("The PLT ~s includes the following files:\n~p\n", + io_lib:format("The PLT ~s includes the following files:\n~p\n\n", [PLT, Files]); {error, read_error} -> - Msg = io_lib:format("Could not read the PLT file ~p\n", [PLT]), + Msg = io_lib:format("Could not read the PLT file ~p\n\n", [PLT]), throw({dialyzer_error, Msg}); {error, no_such_file} -> - Msg = io_lib:format("The PLT file ~p does not exist\n", [PLT]), + Msg = io_lib:format("The PLT file ~p does not exist\n\n", [PLT]), throw({dialyzer_error, Msg}) end, + String ++ get_plt_info(PLTs); +get_plt_info([]) -> "". + +do_print_plt_info(PLTInfo, OutputFile) -> case OutputFile =:= none of true -> - io:format("~s", [String]), + io:format("~s", [PLTInfo]), ?RET_NOTHING_SUSPICIOUS; false -> case file:open(OutputFile, [write]) of {ok, FileDesc} -> - io:format(FileDesc, "~s", [String]), + io:format(FileDesc, "~s", [PLTInfo]), ok = file:close(FileDesc), ?RET_NOTHING_SUSPICIOUS; {error, Reason} -> diff --git a/lib/dialyzer/src/dialyzer.hrl b/lib/dialyzer/src/dialyzer.hrl index 2da8ed2e5d..1d98574585 100644 --- a/lib/dialyzer/src/dialyzer.hrl +++ b/lib/dialyzer/src/dialyzer.hrl @@ -129,7 +129,7 @@ defines = [] :: [dial_define()], from = byte_code :: start_from(), get_warnings = maybe :: boolean() | 'maybe', - init_plt = none :: 'none' | file:filename(), + init_plts = [] :: [file:filename()], include_dirs = [] :: [file:filename()], output_plt = none :: 'none' | file:filename(), legal_warnings = ordsets:new() :: ordset(dial_warn_tag()), diff --git a/lib/dialyzer/src/dialyzer_cl.erl b/lib/dialyzer/src/dialyzer_cl.erl index 0250c47ad0..1987c1732c 100644 --- a/lib/dialyzer/src/dialyzer_cl.erl +++ b/lib/dialyzer/src/dialyzer_cl.erl @@ -81,11 +81,15 @@ build_plt(Opts) -> init_opts_for_build(Opts) -> case Opts#options.output_plt =:= none of true -> - case Opts#options.init_plt of - none -> Opts#options{init_plt = none, output_plt = get_default_plt()}; - Plt -> Opts#options{init_plt = none, output_plt = Plt} + case Opts#options.init_plts of + [] -> Opts#options{output_plt = get_default_output_plt()}; + [Plt] -> Opts#options{init_plts = [], output_plt = Plt}; + Plts -> + Msg = io_lib:format("Could not build multiple PLT files: ~s\n", + [format_plts(Plts)]), + error(Msg) end; - false -> Opts#options{init_plt = none} + false -> Opts#options{init_plts = []} end. %%-------------------------------------------------------------------- @@ -98,39 +102,58 @@ add_to_plt(Opts) -> init_opts_for_add(Opts) -> case Opts#options.output_plt =:= none of true -> - case Opts#options.init_plt of - none -> Opts#options{output_plt = get_default_plt(), - init_plt = get_default_plt()}; - Plt -> Opts#options{output_plt = Plt} + case Opts#options.init_plts of + [] -> Opts#options{output_plt = get_default_output_plt(), + init_plts = get_default_init_plt()}; + [Plt] -> Opts#options{output_plt = Plt}; + Plts -> + Msg = io_lib:format("Could not add to multiple PLT files: ~s\n", + [format_plts(Plts)]), + error(Msg) end; false -> - case Opts#options.init_plt =:= none of - true -> Opts#options{init_plt = get_default_plt()}; + case Opts#options.init_plts =:= [] of + true -> Opts#options{init_plts = get_default_init_plt()}; false -> Opts end end. %%-------------------------------------------------------------------- -check_plt(Opts) -> +check_plt(#options{init_plts = []} = Opts) -> Opts1 = init_opts_for_check(Opts), - report_check(Opts), - plt_common(Opts1, [], []). + report_check(Opts1), + plt_common(Opts1, [], []); +check_plt(#options{init_plts = Plts} = Opts) -> + check_plt_aux(Plts, Opts). + +check_plt_aux([_] = Plt, Opts) -> + Opts1 = Opts#options{init_plts = Plt}, + Opts2 = init_opts_for_check(Opts1), + report_check(Opts2), + plt_common(Opts2, [], []); +check_plt_aux([Plt|Plts], Opts) -> + Opts1 = Opts#options{init_plts = [Plt]}, + Opts2 = init_opts_for_check(Opts1), + report_check(Opts2), + plt_common(Opts2, [], []), + check_plt_aux(Plts, Opts). init_opts_for_check(Opts) -> - Plt = - case Opts#options.init_plt of - none -> get_default_plt(); - Plt0 -> Plt0 + InitPlt = + case Opts#options.init_plts of + []-> get_default_init_plt(); + Plt -> Plt end, + [OutputPlt] = InitPlt, Opts#options{files = [], files_rec = [], analysis_type = plt_check, defines = [], from = byte_code, - init_plt = Plt, + init_plts = InitPlt, include_dirs = [], - output_plt = Plt, + output_plt = OutputPlt, use_contracts = true }. @@ -144,21 +167,25 @@ remove_from_plt(Opts) -> init_opts_for_remove(Opts) -> case Opts#options.output_plt =:= none of true -> - case Opts#options.init_plt of - none -> Opts#options{output_plt = get_default_plt(), - init_plt = get_default_plt()}; - Plt -> Opts#options{output_plt = Plt} + case Opts#options.init_plts of + [] -> Opts#options{output_plt = get_default_output_plt(), + init_plts = get_default_init_plt()}; + [Plt] -> Opts#options{output_plt = Plt}; + Plts -> + Msg = io_lib:format("Could not remove from multiple PLT files: ~s\n", + [format_plts(Plts)]), + error(Msg) end; false -> - case Opts#options.init_plt =:= none of - true -> Opts#options{init_plt = get_default_plt()}; + case Opts#options.init_plts =:= [] of + true -> Opts#options{init_plts = get_default_init_plt()}; false -> Opts end end. %%-------------------------------------------------------------------- -plt_common(Opts, RemoveFiles, AddFiles) -> +plt_common(#options{init_plts = [InitPlt]} = Opts, RemoveFiles, AddFiles) -> case check_plt(Opts, RemoveFiles, AddFiles) of ok -> case Opts#options.report_mode of @@ -174,7 +201,7 @@ plt_common(Opts, RemoveFiles, AddFiles) -> report_failed_plt_check(Opts, DiffMd5), {AnalFiles, RemovedMods, ModDeps1} = expand_dependent_modules(Md5, DiffMd5, ModDeps), - Plt = clean_plt(Opts#options.init_plt, RemovedMods), + Plt = clean_plt(InitPlt, RemovedMods), case AnalFiles =:= [] of true -> %% Only removed stuff. Just write the PLT. @@ -186,19 +213,19 @@ plt_common(Opts, RemoveFiles, AddFiles) -> end; {error, no_such_file} -> Msg = io_lib:format("Could not find the PLT: ~s\n~s", - [Opts#options.init_plt, default_plt_error_msg()]), + [InitPlt, default_plt_error_msg()]), error(Msg); {error, not_valid} -> Msg = io_lib:format("The file: ~s is not a valid PLT file\n~s", - [Opts#options.init_plt, default_plt_error_msg()]), + [InitPlt, default_plt_error_msg()]), error(Msg); {error, read_error} -> Msg = io_lib:format("Could not read the PLT: ~s\n~s", - [Opts#options.init_plt, default_plt_error_msg()]), + [InitPlt, default_plt_error_msg()]), error(Msg); {error, {no_file_to_remove, F}} -> Msg = io_lib:format("Could not remove the file ~s from the PLT: ~s\n", - [F, Opts#options.init_plt]), + [F, InitPlt]), error(Msg) end. @@ -218,8 +245,7 @@ default_plt_error_msg() -> %%-------------------------------------------------------------------- -check_plt(Opts, RemoveFiles, AddFiles) -> - Plt = Opts#options.init_plt, +check_plt(#options{init_plts = [Plt]} = Opts, RemoveFiles, AddFiles) -> case dialyzer_plt:check_plt(Plt, RemoveFiles, AddFiles) of {old_version, _MD5} = OldVersion -> report_old_version(Opts), @@ -234,14 +260,14 @@ check_plt(Opts, RemoveFiles, AddFiles) -> %%-------------------------------------------------------------------- -report_check(#options{report_mode = ReportMode, init_plt = InitPlt}) -> +report_check(#options{report_mode = ReportMode, init_plts = [InitPlt]}) -> case ReportMode of quiet -> ok; _ -> io:format(" Checking whether the PLT ~s is up-to-date...", [InitPlt]) end. -report_old_version(#options{report_mode = ReportMode, init_plt = InitPlt}) -> +report_old_version(#options{report_mode = ReportMode, init_plts = [InitPlt]}) -> case ReportMode of quiet -> ok; _ -> @@ -264,7 +290,7 @@ report_failed_plt_check(#options{analysis_type = AnalType, report_analysis_start(#options{analysis_type = Type, report_mode = ReportMode, - init_plt = InitPlt, + init_plts = InitPlts, output_plt = OutputPlt}) -> case ReportMode of quiet -> ok; @@ -272,6 +298,7 @@ report_analysis_start(#options{analysis_type = Type, io:format(" "), case Type of plt_add -> + [InitPlt] = InitPlts, case InitPlt =:= OutputPlt of true -> io:format("Adding information to ~s...", [OutputPlt]); false -> io:format("Adding information from ~s to ~s...", @@ -282,6 +309,7 @@ report_analysis_start(#options{analysis_type = Type, plt_check -> io:format("Rebuilding the information in ~s...", [OutputPlt]); plt_remove -> + [InitPlt] = InitPlts, case InitPlt =:= OutputPlt of true -> io:format("Removing information from ~s...", [OutputPlt]); false -> io:format("Removing information from ~s to ~s...", @@ -320,16 +348,28 @@ report_md5_diff(List) -> %%-------------------------------------------------------------------- -get_default_plt() -> +get_default_init_plt() -> + [dialyzer_plt:get_default_plt()]. + +get_default_output_plt() -> dialyzer_plt:get_default_plt(). %%-------------------------------------------------------------------- +format_plts([Plt]) -> Plt; +format_plts([Plt|Plts]) -> + Plt ++ ", " ++ format_plts(Plts). + +%%-------------------------------------------------------------------- + do_analysis(Options) -> Files = get_files_from_opts(Options), - case Options#options.init_plt of - none -> do_analysis(Files, Options, dialyzer_plt:new(), none); - File -> do_analysis(Files, Options, dialyzer_plt:from_file(File), none) + case Options#options.init_plts of + [] -> do_analysis(Files, Options, dialyzer_plt:new(), none); + PltFiles -> + Plts = [dialyzer_plt:from_file(F) || F <- PltFiles], + Plt = dialyzer_plt:merge_plts_or_report_conflicts(PltFiles, Plts), + do_analysis(Files, Options, Plt, none) end. do_analysis(Files, Options, Plt, PltInfo) -> diff --git a/lib/dialyzer/src/dialyzer_cl_parse.erl b/lib/dialyzer/src/dialyzer_cl_parse.erl index 0160b84abc..2f9e577544 100644 --- a/lib/dialyzer/src/dialyzer_cl_parse.erl +++ b/lib/dialyzer/src/dialyzer_cl_parse.erl @@ -138,11 +138,17 @@ cl(["-pa", Path|T]) -> true -> cl(T); {error, _} -> error("Bad directory for -pa: "++Path) end; -cl(["--plt", PLT|T]) -> - put(dialyzer_init_plt, PLT), - cl(T); cl(["--plt"]) -> error("No plt specified for --plt"); +cl(["--plt", PLT|T]) -> + put(dialyzer_init_plts, [PLT]), + cl(T); +cl(["--plts"]) -> + error("No plts specified for --plts"); +cl(["--plts"|T]) -> + {PLTs, NewT} = get_plts(T, []), + put(dialyzer_init_plts, PLTs), + cl(NewT); cl(["-q"|T]) -> put(dialyzer_options_report_mode, quiet), cl(T); @@ -284,7 +290,7 @@ common_options() -> [{defines, get(dialyzer_options_defines)}, {from, get(dialyzer_options_from)}, {include_dirs, get(dialyzer_include)}, - {init_plt, get(dialyzer_init_plt)}, + {plts, get(dialyzer_init_plts)}, {output_plt, get(dialyzer_output_plt)}, {report_mode, get(dialyzer_options_report_mode)}, {use_spec, get(dialyzer_options_use_contracts)}, @@ -309,6 +315,13 @@ get_lib_dir([], Acc) -> %%----------------------------------------------------------------------- +get_plts(["--"|T], Acc) -> {lists:reverse(Acc), T}; +get_plts(["-"++_Opt = H|T], Acc) -> {lists:reverse(Acc), [H|T]}; +get_plts([H|T], Acc) -> get_plts(T, [H|Acc]); +get_plts([], Acc) -> {lists:reverse(Acc), []}. + +%%----------------------------------------------------------------------- + help_warnings() -> S = warning_options_msg(), io:put_chars(S), @@ -316,9 +329,10 @@ help_warnings() -> help_message() -> S = "Usage: dialyzer [--help] [--version] [--shell] [--quiet] [--verbose] - [-pa dir]* [--plt plt] [-Ddefine]* [-I include_dir]* - [--output_plt file] [-Wwarn]* [--src] [--gui | --wx] - [files_or_dirs] [-r dirs] [--apps applications] [-o outfile] + [-pa dir]* [--plt plt] [--plts plts] [-Ddefine]* + [-I include_dir]* [--output_plt file] [-Wwarn]* + [--src] [--gui | --wx] [files_or_dirs] [-r dirs] + [--apps applications] [-o outfile] [--build_plt] [--add_to_plt] [--remove_from_plt] [--check_plt] [--no_check_plt] [--plt_info] [--get_warnings] [--no_native] @@ -362,6 +376,10 @@ Options: --plt plt Use the specified plt as the initial plt (if the plt was built during setup the files will be checked for consistency) + --plts plts + Merges the specified plts to create the initial plt -- requires + that the plts are disjoint (i.e., do not have any module + appearing in more than one plt) -Wwarn A family of options which selectively turn on/off warnings (for help on the names of warnings use dialyzer -Whelp) @@ -378,12 +396,12 @@ Options: --build_plt The analysis starts from an empty plt and creates a new one from the files specified with -c and -r. Only works for beam files. - Use --plt or --output_plt to override the default plt location. + Use --plt(s) or --output_plt to override the default plt location. --add_to_plt The plt is extended to also include the files specified with -c and -r. - Use --plt to specify wich plt to start from, and --output_plt to - specify where to put the plt. Note that the analysis might include - files from the plt if they depend on the new files. + Use --plt(s) to specify wich plt to start from, and --output_plt to + specify where to put the plt. Note that the analysis might include + files from the plt if they depend on the new files. This option only works with beam files. --remove_from_plt The information from the files specified with -c and -r is removed @@ -396,8 +414,8 @@ Options: Skip the plt check when running Dialyzer. Useful when working with installed plts that never change. --plt_info - Makes Dialyzer print information about the plt and then quit. The plt - can be specified with --plt. + Makes Dialyzer print information about the plt and then quit. The plt + can be specified with --plt(s). --get_warnings Makes Dialyzer emit warnings even when manipulating the plt. Only emits warnings for files that are actually analyzed. diff --git a/lib/dialyzer/src/dialyzer_gui.erl b/lib/dialyzer/src/dialyzer_gui.erl index f353638cdf..4436330f7f 100644 --- a/lib/dialyzer/src/dialyzer_gui.erl +++ b/lib/dialyzer/src/dialyzer_gui.erl @@ -88,8 +88,8 @@ -spec start(#options{}) -> ?RET_NOTHING_SUSPICIOUS. -start(DialyzerOptions = #options{from = From, init_plt = InitPltFile, - legal_warnings = LegalWarnings}) -> +start(#options{from = From, init_plts = InitPltFiles, + legal_warnings = LegalWarnings} = DialyzerOptions) -> process_flag(trap_exit, true), GS = gs:start(), @@ -336,9 +336,13 @@ start(DialyzerOptions = #options{from = From, init_plt = InitPltFile, gs:config(Packer, WH), {ok, CWD} = file:get_cwd(), - InitPlt = try dialyzer_plt:from_file(InitPltFile) - catch throw:{dialyzer_error, _} -> dialyzer_plt:new() - end, + InitPlt = + case InitPltFiles of + [] -> dialyzer_plt:new(); + _ -> + Plts = [dialyzer_plt:from_file(F) || F <- InitPltFiles], + dialyzer_plt:merge_plts_or_report_conflicts(InitPltFiles, Plts) + end, State = #gui_state{add_all = AddAll, add_file = AddFile, diff --git a/lib/dialyzer/src/dialyzer_gui_wx.erl b/lib/dialyzer/src/dialyzer_gui_wx.erl index 2e309d7ec1..e711c15ea7 100644 --- a/lib/dialyzer/src/dialyzer_gui_wx.erl +++ b/lib/dialyzer/src/dialyzer_gui_wx.erl @@ -88,7 +88,7 @@ start(DialyzerOptions) -> State = wx:batch(fun() -> create_window(Wx, DialyzerOptions) end), gui_loop(State). -create_window(Wx, DialyzerOptions) -> +create_window(Wx, #options{init_plts = InitPltFiles} = DialyzerOptions) -> {ok, Host} = inet:gethostname(), %%---------- initializing frame --------- @@ -258,11 +258,15 @@ create_window(Wx, DialyzerOptions) -> plt = PltMenu, options =OptionsMenu, help = HelpMenu}, - - InitPlt = try dialyzer_plt:from_file(DialyzerOptions#options.init_plt) - catch throw:{dialyzer_error, _} -> dialyzer_plt:new() - end, + InitPlt = + case InitPltFiles of + [] -> dialyzer_plt:new(); + _ -> + Plts = [dialyzer_plt:from_file(F) || F <- InitPltFiles], + dialyzer_plt:merge_plts_or_report_conflicts(InitPltFiles, Plts) + end, + #gui_state{add = AddButton, add_dir = AddDirButton, add_rec = AddRecButton, diff --git a/lib/dialyzer/src/dialyzer_options.erl b/lib/dialyzer/src/dialyzer_options.erl index 010625b7bd..2c0afa6e2b 100644 --- a/lib/dialyzer/src/dialyzer_options.erl +++ b/lib/dialyzer/src/dialyzer_options.erl @@ -53,14 +53,21 @@ build(Opts) -> InitPlt = dialyzer_plt:get_default_plt(), DefaultOpts = #options{}, DefaultOpts1 = DefaultOpts#options{legal_warnings = DefaultWarns1, - init_plt = InitPlt}, - try - NewOpts = build_options(Opts, DefaultOpts1), + init_plts = [InitPlt]}, + try + Opts1 = preprocess_opts(Opts), + NewOpts = build_options(Opts1, DefaultOpts1), postprocess_opts(NewOpts) catch throw:{dialyzer_options_error, Msg} -> {error, Msg} end. +preprocess_opts([]) -> []; +preprocess_opts([{init_plt, File}|Opts]) -> + [{plts, [File]}|preprocess_opts(Opts)]; +preprocess_opts([Opt|Opts]) -> + [Opt|preprocess_opts(Opts)]. + postprocess_opts(Opts = #options{}) -> Opts1 = check_output_plt(Opts), adapt_get_warnings(Opts1). @@ -144,9 +151,9 @@ build_options([{OptionName, Value} = Term|Rest], Options) -> build_options(Rest, Options#options{from = Value}); get_warnings -> build_options(Rest, Options#options{get_warnings = Value}); - init_plt -> - assert_filenames([Term], [Value]), - build_options(Rest, Options#options{init_plt = Value}); + plts -> + assert_filenames(Term, Value), + build_options(Rest, Options#options{init_plts = Value}); include_dirs -> assert_filenames(Term, Value), OldVal = Options#options.include_dirs, diff --git a/lib/dialyzer/src/dialyzer_plt.erl b/lib/dialyzer/src/dialyzer_plt.erl index 08d0b318b5..a7ba270c41 100644 --- a/lib/dialyzer/src/dialyzer_plt.erl +++ b/lib/dialyzer/src/dialyzer_plt.erl @@ -51,6 +51,7 @@ lookup_contract/2, lookup_module/2, merge_plts/1, + merge_plts_or_report_conflicts/2, new/0, plt_and_info_from_file/1, get_specs/1, @@ -292,6 +293,38 @@ merge_plts(List) -> exported_types = sets_merge(ExpTypesList), contracts = table_merge(ContractsList)}. +-spec merge_disj_plts([plt()]) -> plt(). + +merge_disj_plts(List) -> + InfoList = [Info || #plt{info = Info} <- List], + TypesList = [Types || #plt{types = Types} <- List], + ExpTypesList = [ExpTypes || #plt{exported_types = ExpTypes} <- List], + ContractsList = [Contracts || #plt{contracts = Contracts} <- List], + #plt{info = table_disj_merge(InfoList), + types = table_disj_merge(TypesList), + exported_types = sets_disj_merge(ExpTypesList), + contracts = table_disj_merge(ContractsList)}. + +-spec merge_plts_or_report_conflicts([file:filename()], [plt()]) -> plt(). + +merge_plts_or_report_conflicts(PltFiles, Plts) -> + try + merge_disj_plts(Plts) + catch throw:{dialyzer_error, not_disjoint_plts} -> + IncFiles = lists:append([begin {ok, Fs} = included_files(F), Fs end + || F <- PltFiles]), + ConfFiles = find_duplicates(IncFiles), + Msg = io_lib:format("Could not merge PLTs since they are not disjoint\n" + "The following files are included in more than one " + "PLTs:\n~p\n", [ConfFiles]), + error(Msg) + end. + +find_duplicates(List) -> + ModList = [filename:basename(E) || E <- List], + SortedList = lists:usort(ModList), + lists:usort(ModList -- SortedList). + -spec to_file(file:filename(), plt(), mod_deps(), {[file_md5()], mod_deps()}) -> 'ok'. to_file(FileName, @@ -556,6 +589,25 @@ table_merge([Plt|Plts], Acc) -> NewAcc = dict:merge(fun(_Key, Val, Val) -> Val end, Plt, Acc), table_merge(Plts, NewAcc). +table_disj_merge([H|T]) -> + table_disj_merge(T, H). + +table_disj_merge([], Acc) -> + Acc; +table_disj_merge([Plt|Plts], Acc) -> + case table_is_disjoint(Plt, Acc) of + true -> + NewAcc = dict:merge(fun(_Key, _Val1, _Val2) -> gazonk end, + Plt, Acc), + table_disj_merge(Plts, NewAcc); + false -> throw({dialyzer_error, not_disjoint_plts}) + end. + +table_is_disjoint(T1, T2) -> + K1 = dict:fetch_keys(T1), + K2 = dict:fetch_keys(T2), + lists:all(fun(E) -> not lists:member(E, K2) end, K1). + sets_merge([H|T]) -> sets_merge(T, H). @@ -565,6 +617,19 @@ sets_merge([Plt|Plts], Acc) -> NewAcc = sets:union(Plt, Acc), sets_merge(Plts, NewAcc). +sets_disj_merge([H|T]) -> + sets_disj_merge(T, H). + +sets_disj_merge([], Acc) -> + Acc; +sets_disj_merge([Plt|Plts], Acc) -> + case sets:is_disjoint(Plt, Acc) of + true -> + NewAcc = sets:union(Plt, Acc), + sets_disj_merge(Plts, NewAcc); + false -> throw({dialyzer_error, not_disjoint_plts}) + end. + %%--------------------------------------------------------------------------- %% Debug utilities. diff --git a/lib/docbuilder/src/docb_main.erl b/lib/docbuilder/src/docb_main.erl index 87a1401a02..4f5f035a65 100644 --- a/lib/docbuilder/src/docb_main.erl +++ b/lib/docbuilder/src/docb_main.erl @@ -34,14 +34,23 @@ %% Parses the source file File and transforms the result to html, %% latex and/or man page format. process(File, Opts) -> - - File1 = File ++ ".tmpconv", - os:cmd("sed -e 's/xi:include[ \t]*href/include file/g' -e 's/xmlns:xi=\"http:\\/\\/www.w3.org\\/2001\\/XInclude\"//g' < " ++ - File ++ ".xml > " ++ File1 ++ ".xml"), %LATH + + SrcType = docb_util:lookup_option(src_type, Opts), + + File1 = + case SrcType of + ".xml" -> + FileTmp = File ++ ".tmpconv", + os:cmd("sed -e 's/xi:include[ \t]*href/include file/g' -e 's/xmlns:xi=\"http:\\/\\/www.w3.org\\/2001\\/XInclude\"//g' < " ++ + File ++ ".xml > " ++ FileTmp ++ ".xml"), + FileTmp; + ".sgml" -> + File + end, case parse1(File1, Opts) of errors -> - file:delete(File1 ++ ".xml"), + delete_tmp_file(SrcType, File1), errors; {ok, Tree} -> From = element(1, Tree), @@ -62,15 +71,21 @@ process(File, Opts) -> Result = [transform(From, To, Opts, File, Tree)||To <- Tos], case lists:member(transformation_error,Result) of true -> - file:delete(File1 ++ ".xml"), + delete_tmp_file(SrcType, File1), errors; _ -> - file:delete(File1 ++ ".xml"), + delete_tmp_file(SrcType, File1), ok end end. + +delete_tmp_file(".xml", File) -> + file:delete(File ++ ".xml"); +delete_tmp_file(_, _) -> + ok. + %%---------------------------------------------------------------------- %% parse(File, Opts) -> {ok, Tree} | errors diff --git a/lib/erl_docgen/priv/dtd_man_entities/xhtml-lat1.ent b/lib/erl_docgen/priv/dtd_man_entities/xhtml-lat1.ent index 3df9970a43..7a07e2c406 100644 --- a/lib/erl_docgen/priv/dtd_man_entities/xhtml-lat1.ent +++ b/lib/erl_docgen/priv/dtd_man_entities/xhtml-lat1.ent @@ -21,26 +21,26 @@ <!ENTITY sect "§"> <!-- section sign, U+00A7 ISOnum --> <!ENTITY uml "¨"> <!-- diaeresis = spacing diaeresis, U+00A8 ISOdia --> -<!ENTITY copy "©"> <!-- copyright sign, U+00A9 ISOnum --> +<!ENTITY copy "(C)"> <!-- copyright sign, U+00A9 ISOnum --> <!ENTITY ordf "ª"> <!-- feminine ordinal indicator, U+00AA ISOnum --> -<!ENTITY laquo "«"> <!-- left-pointing double angle quotation mark - = left pointing guillemet, U+00AB ISOnum --> +<!ENTITY laquo """> <!-- left-pointing double angle quotation mark + = left pointing guillemetn = " in man pages, U+00AB ISOnum --> <!ENTITY not "¬"> <!-- not sign = discretionary hyphen, U+00AC ISOnum --> -<!ENTITY shy "­"> <!-- soft hyphen = discretionary hyphen, +<!ENTITY shy ""> <!-- soft hyphen = discretionary hyphen, U+00AD ISOnum --> -<!ENTITY reg "®"> <!-- registered sign = registered trade mark sign, +<!ENTITY reg "(R)"> <!-- registered sign = registered trade mark sign, U+00AE ISOnum --> <!ENTITY macr "¯"> <!-- macron = spacing macron = overline = APL overbar, U+00AF ISOdia --> <!ENTITY deg "°"> <!-- degree sign, U+00B0 ISOnum --> -<!ENTITY plusmn "±"> <!-- plus-minus sign = plus-or-minus sign, +<!ENTITY plusmn "+/-"> <!-- plus-minus sign = plus-or-minus sign, U+00B1 ISOnum --> <!ENTITY sup2 "²"> <!-- superscript two = superscript digit two = squared, U+00B2 ISOnum --> <!ENTITY sup3 "³"> <!-- superscript three = superscript digit three = cubed, U+00B3 ISOnum --> -<!ENTITY acute "´"> <!-- acute accent = spacing acute, +<!ENTITY acute "'"> <!-- acute accent = spacing acute, U+00B4 ISOdia --> <!ENTITY micro "µ"> <!-- micro sign, U+00B5 ISOnum --> <!ENTITY para "¶"> <!-- pilcrow sign = paragraph sign, @@ -62,134 +62,134 @@ = fraction three quarters, U+00BE ISOnum --> <!ENTITY iquest "¿"> <!-- inverted question mark = turned question mark, U+00BF ISOnum --> -<!ENTITY Agrave "À"> <!-- latin capital letter A with grave +<!ENTITY Agrave "A"> <!-- latin capital letter A with grave = latin capital letter A grave, U+00C0 ISOlat1 --> -<!ENTITY Aacute "Á"> <!-- latin capital letter A with acute, +<!ENTITY Aacute "A"> <!-- latin capital letter A with acute, U+00C1 ISOlat1 --> -<!ENTITY Acirc "Â"> <!-- latin capital letter A with circumflex, +<!ENTITY Acirc "A"> <!-- latin capital letter A with circumflex, U+00C2 ISOlat1 --> -<!ENTITY Atilde "Ã"> <!-- latin capital letter A with tilde, +<!ENTITY Atilde "A"> <!-- latin capital letter A with tilde, U+00C3 ISOlat1 --> -<!ENTITY Auml "Ä"> <!-- latin capital letter A with diaeresis, +<!ENTITY Auml "A"> <!-- latin capital letter A with diaeresis, U+00C4 ISOlat1 --> -<!ENTITY Aring "Å"> <!-- latin capital letter A with ring above +<!ENTITY Aring "A"> <!-- latin capital letter A with ring above = latin capital letter A ring, U+00C5 ISOlat1 --> -<!ENTITY AElig "Æ"> <!-- latin capital letter AE +<!ENTITY AElig "AE"> <!-- latin capital letter AE = latin capital ligature AE, U+00C6 ISOlat1 --> -<!ENTITY Ccedil "Ç"> <!-- latin capital letter C with cedilla, +<!ENTITY Ccedil "C"> <!-- latin capital letter C with cedilla, U+00C7 ISOlat1 --> -<!ENTITY Egrave "È"> <!-- latin capital letter E with grave, +<!ENTITY Egrave "E"> <!-- latin capital letter E with grave, U+00C8 ISOlat1 --> -<!ENTITY Eacute "É"> <!-- latin capital letter E with acute, +<!ENTITY Eacute "E"> <!-- latin capital letter E with acute, U+00C9 ISOlat1 --> -<!ENTITY Ecirc "Ê"> <!-- latin capital letter E with circumflex, +<!ENTITY Ecirc "E"> <!-- latin capital letter E with circumflex, U+00CA ISOlat1 --> -<!ENTITY Euml "Ë"> <!-- latin capital letter E with diaeresis, +<!ENTITY Euml "E"> <!-- latin capital letter E with diaeresis, U+00CB ISOlat1 --> -<!ENTITY Igrave "Ì"> <!-- latin capital letter I with grave, +<!ENTITY Igrave "I"> <!-- latin capital letter I with grave, U+00CC ISOlat1 --> -<!ENTITY Iacute "Í"> <!-- latin capital letter I with acute, +<!ENTITY Iacute "I"> <!-- latin capital letter I with acute, U+00CD ISOlat1 --> -<!ENTITY Icirc "Î"> <!-- latin capital letter I with circumflex, +<!ENTITY Icirc "I"> <!-- latin capital letter I with circumflex, U+00CE ISOlat1 --> -<!ENTITY Iuml "Ï"> <!-- latin capital letter I with diaeresis, +<!ENTITY Iuml "I"> <!-- latin capital letter I with diaeresis, U+00CF ISOlat1 --> <!ENTITY ETH "Ð"> <!-- latin capital letter ETH, U+00D0 ISOlat1 --> -<!ENTITY Ntilde "Ñ"> <!-- latin capital letter N with tilde, +<!ENTITY Ntilde "N"> <!-- latin capital letter N with tilde, U+00D1 ISOlat1 --> -<!ENTITY Ograve "Ò"> <!-- latin capital letter O with grave, +<!ENTITY Ograve "O"> <!-- latin capital letter O with grave, U+00D2 ISOlat1 --> -<!ENTITY Oacute "Ó"> <!-- latin capital letter O with acute, +<!ENTITY Oacute "O"> <!-- latin capital letter O with acute, U+00D3 ISOlat1 --> -<!ENTITY Ocirc "Ô"> <!-- latin capital letter O with circumflex, +<!ENTITY Ocirc "O"> <!-- latin capital letter O with circumflex, U+00D4 ISOlat1 --> -<!ENTITY Otilde "Õ"> <!-- latin capital letter O with tilde, +<!ENTITY Otilde "O"> <!-- latin capital letter O with tilde, U+00D5 ISOlat1 --> -<!ENTITY Ouml "Ö"> <!-- latin capital letter O with diaeresis, +<!ENTITY Ouml "O"> <!-- latin capital letter O with diaeresis, U+00D6 ISOlat1 --> -<!ENTITY times "×"> <!-- multiplication sign, U+00D7 ISOnum --> +<!ENTITY times "x"> <!-- multiplication sign, U+00D7 ISOnum --> <!ENTITY Oslash "Ø"> <!-- latin capital letter O with stroke = latin capital letter O slash, U+00D8 ISOlat1 --> -<!ENTITY Ugrave "Ù"> <!-- latin capital letter U with grave, +<!ENTITY Ugrave "U"> <!-- latin capital letter U with grave, U+00D9 ISOlat1 --> -<!ENTITY Uacute "Ú"> <!-- latin capital letter U with acute, +<!ENTITY Uacute "U"> <!-- latin capital letter U with acute, U+00DA ISOlat1 --> -<!ENTITY Ucirc "Û"> <!-- latin capital letter U with circumflex, +<!ENTITY Ucirc "U"> <!-- latin capital letter U with circumflex, U+00DB ISOlat1 --> -<!ENTITY Uuml "Ü"> <!-- latin capital letter U with diaeresis, +<!ENTITY Uuml "U"> <!-- latin capital letter U with diaeresis, U+00DC ISOlat1 --> -<!ENTITY Yacute "Ý"> <!-- latin capital letter Y with acute, +<!ENTITY Yacute "Y"> <!-- latin capital letter Y with acute, U+00DD ISOlat1 --> <!ENTITY THORN "Þ"> <!-- latin capital letter THORN, U+00DE ISOlat1 --> <!ENTITY szlig "ß"> <!-- latin small letter sharp s = ess-zed, U+00DF ISOlat1 --> -<!ENTITY agrave "à"> <!-- latin small letter a with grave +<!ENTITY agrave "a"> <!-- latin small letter a with grave = latin small letter a grave, U+00E0 ISOlat1 --> -<!ENTITY aacute "á"> <!-- latin small letter a with acute, +<!ENTITY aacute "a"> <!-- latin small letter a with acute, U+00E1 ISOlat1 --> -<!ENTITY acirc "â"> <!-- latin small letter a with circumflex, +<!ENTITY acirc "a"> <!-- latin small letter a with circumflex, U+00E2 ISOlat1 --> -<!ENTITY atilde "ã"> <!-- latin small letter a with tilde, +<!ENTITY atilde "a"> <!-- latin small letter a with tilde, U+00E3 ISOlat1 --> -<!ENTITY auml "ä"> <!-- latin small letter a with diaeresis, +<!ENTITY auml "a"> <!-- latin small letter a with diaeresis, U+00E4 ISOlat1 --> -<!ENTITY aring "å"> <!-- latin small letter a with ring above +<!ENTITY aring "a"> <!-- latin small letter a with ring above = latin small letter a ring, U+00E5 ISOlat1 --> -<!ENTITY aelig "æ"> <!-- latin small letter ae +<!ENTITY aelig "ae"> <!-- latin small letter ae = latin small ligature ae, U+00E6 ISOlat1 --> -<!ENTITY ccedil "ç"> <!-- latin small letter c with cedilla, +<!ENTITY ccedil "c"> <!-- latin small letter c with cedilla, U+00E7 ISOlat1 --> -<!ENTITY egrave "è"> <!-- latin small letter e with grave, +<!ENTITY egrave "e"> <!-- latin small letter e with grave, U+00E8 ISOlat1 --> -<!ENTITY eacute "é"> <!-- latin small letter e with acute, +<!ENTITY eacute "e"> <!-- latin small letter e with acute, U+00E9 ISOlat1 --> -<!ENTITY ecirc "ê"> <!-- latin small letter e with circumflex, +<!ENTITY ecirc "e"> <!-- latin small letter e with circumflex, U+00EA ISOlat1 --> -<!ENTITY euml "ë"> <!-- latin small letter e with diaeresis, +<!ENTITY euml "e"> <!-- latin small letter e with diaeresis, U+00EB ISOlat1 --> -<!ENTITY igrave "ì"> <!-- latin small letter i with grave, +<!ENTITY igrave "i"> <!-- latin small letter i with grave, U+00EC ISOlat1 --> -<!ENTITY iacute "í"> <!-- latin small letter i with acute, +<!ENTITY iacute "i"> <!-- latin small letter i with acute, U+00ED ISOlat1 --> -<!ENTITY icirc "î"> <!-- latin small letter i with circumflex, +<!ENTITY icirc "i"> <!-- latin small letter i with circumflex, U+00EE ISOlat1 --> -<!ENTITY iuml "ï"> <!-- latin small letter i with diaeresis, +<!ENTITY iuml "i"> <!-- latin small letter i with diaeresis, U+00EF ISOlat1 --> <!ENTITY eth "ð"> <!-- latin small letter eth, U+00F0 ISOlat1 --> -<!ENTITY ntilde "ñ"> <!-- latin small letter n with tilde, +<!ENTITY ntilde "n"> <!-- latin small letter n with tilde, U+00F1 ISOlat1 --> -<!ENTITY ograve "ò"> <!-- latin small letter o with grave, +<!ENTITY ograve "o"> <!-- latin small letter o with grave, U+00F2 ISOlat1 --> -<!ENTITY oacute "ó"> <!-- latin small letter o with acute, +<!ENTITY oacute "o"> <!-- latin small letter o with acute, U+00F3 ISOlat1 --> -<!ENTITY ocirc "ô"> <!-- latin small letter o with circumflex, +<!ENTITY ocirc "o"> <!-- latin small letter o with circumflex, U+00F4 ISOlat1 --> -<!ENTITY otilde "õ"> <!-- latin small letter o with tilde, +<!ENTITY otilde "o"> <!-- latin small letter o with tilde, U+00F5 ISOlat1 --> -<!ENTITY ouml "ö"> <!-- latin small letter o with diaeresis, +<!ENTITY ouml "o"> <!-- latin small letter o with diaeresis, U+00F6 ISOlat1 --> <!ENTITY divide "÷"> <!-- division sign, U+00F7 ISOnum --> -<!ENTITY oslash "ø"> <!-- latin small letter o with stroke, +<!ENTITY oslash "o"> <!-- latin small letter o with stroke, = latin small letter o slash, U+00F8 ISOlat1 --> -<!ENTITY ugrave "ù"> <!-- latin small letter u with grave, +<!ENTITY ugrave "u"> <!-- latin small letter u with grave, U+00F9 ISOlat1 --> -<!ENTITY uacute "ú"> <!-- latin small letter u with acute, +<!ENTITY uacute "u"> <!-- latin small letter u with acute, U+00FA ISOlat1 --> -<!ENTITY ucirc "û"> <!-- latin small letter u with circumflex, +<!ENTITY ucirc "u"> <!-- latin small letter u with circumflex, U+00FB ISOlat1 --> -<!ENTITY uuml "ü"> <!-- latin small letter u with diaeresis, +<!ENTITY uuml "u"> <!-- latin small letter u with diaeresis, U+00FC ISOlat1 --> -<!ENTITY yacute "ý"> <!-- latin small letter y with acute, +<!ENTITY yacute "y"> <!-- latin small letter y with acute, U+00FD ISOlat1 --> <!ENTITY thorn "þ"> <!-- latin small letter thorn with, U+00FE ISOlat1 --> -<!ENTITY yuml "ÿ"> <!-- latin small letter y with diaeresis, +<!ENTITY yuml "y"> <!-- latin small letter y with diaeresis, U+00FF ISOlat1 --> diff --git a/lib/erl_docgen/priv/xsl/db_man.xsl b/lib/erl_docgen/priv/xsl/db_man.xsl index 2ed0d95c6d..2a8fb9fe3e 100644 --- a/lib/erl_docgen/priv/xsl/db_man.xsl +++ b/lib/erl_docgen/priv/xsl/db_man.xsl @@ -323,7 +323,7 @@ <xsl:template match="list"> <xsl:text> .RS 2</xsl:text> <xsl:apply-templates/> - <xsl:text> .RE</xsl:text> + <xsl:text> .RE </xsl:text> </xsl:template> <xsl:template match="list/item"> @@ -336,7 +336,7 @@ <xsl:template match="taglist"> <xsl:text> .RS 2</xsl:text> <xsl:apply-templates select="tag|item"/> - <xsl:text> .RE</xsl:text> + <xsl:text> .RE </xsl:text> </xsl:template> <xsl:template match="taglist/tag"> @@ -622,6 +622,23 @@ <xsl:text>></xsl:text> </xsl:template> + <!-- Do not noramlize any text within pre and code tags. --> + <xsl:template match="pre/text()"> + <xsl:call-template name="replace-string"> + <xsl:with-param name="text" select="." /> + <xsl:with-param name="replace" select=""\"" /> + <xsl:with-param name="with" select=""\\"" /> + </xsl:call-template> + </xsl:template> + + <xsl:template match="code/text()"> + <xsl:call-template name="replace-string"> + <xsl:with-param name="text" select="." /> + <xsl:with-param name="replace" select=""\"" /> + <xsl:with-param name="with" select=""\\"" /> + </xsl:call-template> + </xsl:template> + <!-- Replace ' by \&' ans . by \&. --> <xsl:template match="text()"> <xsl:variable name="startstring"> diff --git a/lib/erl_interface/src/legacy/erl_marshal.c b/lib/erl_interface/src/legacy/erl_marshal.c index 5084c65230..a6c2f64dd0 100644 --- a/lib/erl_interface/src/legacy/erl_marshal.c +++ b/lib/erl_interface/src/legacy/erl_marshal.c @@ -511,29 +511,28 @@ static int erl_term_len_helper(ETERM *ep, int dist) case ERL_INTEGER: i = ep->uval.ival.i; - if ((i > ERL_MAX) || (i < ERL_MIN)) len = 7; - else if ((i < 256) && (i >= 0)) len = 2; + if ((i < 256) && (i >= 0)) len = 2; else len = 5; break; case ERL_U_INTEGER: u = ep->uval.uival.u; - if (u > ERL_MAX) len = 7; + if ((int)u < 0) len = 7; else if (u < 256) len = 2; else len = 5; break; case ERL_LONGLONG: l = ep->uval.llval.i; - if ((l > ((long long) ERL_MAX)) || - (l < ((long long) ERL_MIN))) len = 11; + if ((l > ((long long) INT_MAX)) || + (l < ((long long) INT_MIN))) len = 11; else if ((l < 256) && (l >= 0)) len = 2; else len = 5; break; case ERL_U_LONGLONG: ul = ep->uval.ullval.u; - if (ul > ((unsigned long long) ERL_MAX)) len = 11; + if (ul > ((unsigned long long) INT_MAX)) len = 11; else if (ul < 256) len = 2; else len = 5; break; @@ -546,12 +545,7 @@ static int erl_term_len_helper(ETERM *ep, int dist) case ERL_REF: i = strlen((char *)ERL_REF_NODE(ep)); - if (dist >= 4 && ERL_REF_LEN(ep) > 1) { - len = 1 + 2 + (i+3) + 1 + ERL_REF_LEN(ep) * 4; - } else { - /* 1 + N + 4 + 1 where N = 3 + strlen */ - len = 9 + i; - } + len = 1 + 2 + (i+3) + 1 + ERL_REF_LEN(ep) * 4; break; case ERL_PORT: diff --git a/lib/erl_interface/test/ei_decode_SUITE.erl b/lib/erl_interface/test/ei_decode_SUITE.erl index 09a37409f2..524a04a3b4 100644 --- a/lib/erl_interface/test/ei_decode_SUITE.erl +++ b/lib/erl_interface/test/ei_decode_SUITE.erl @@ -232,7 +232,7 @@ send_integers(P) -> ?line send_term_as_binary(P, 16#80000000), % SMALL_BIG_EXT new smallest pos(*) ?line send_term_as_binary(P,-16#80000001), % SMALL_BIG_EXT new largest neg (*) - case erlang:system_info(wordsize) of + case erlang:system_info({wordsize,external}) of 4 -> ?line send_term_as_binary(P, 16#80000000),% SMALL_BIG_EXT u32 ?line send_term_as_binary(P, 16#ffffffff),% SMALL_BIG_EXT largest u32 diff --git a/lib/erl_interface/test/erl_eterm_SUITE_data/eterm_test.c b/lib/erl_interface/test/erl_eterm_SUITE_data/eterm_test.c index f273efd532..80d7f69520 100644 --- a/lib/erl_interface/test/erl_eterm_SUITE_data/eterm_test.c +++ b/lib/erl_interface/test/erl_eterm_SUITE_data/eterm_test.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1997-2009. All Rights Reserved. + * Copyright Ericsson AB 1997-2010. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -98,14 +98,30 @@ static void encode_decode(ETERM* original, const char* text) { static unsigned char encoded[16*1024]; ETERM* new_terms; - int bytes = erl_encode(original, encoded); + ETERM* head; + int bytes; + int len; + + /* If a list, check the elements one by one first */ + head = erl_hd(original); + if (head != NULL) { + encode_decode(head, "CAR"); + encode_decode(erl_tl(original), "CDR"); + } + bytes = erl_encode(original, encoded); if (bytes == 0) { fail("failed to encode terms"); } else if (bytes > sizeof(encoded)) { fail("encoded terms buffer overflow"); } + else if (bytes != (len=erl_term_len(original))) { + fprintf(stderr, "bytes(%d) != len(%d) for term ", bytes, len); + erl_print_term(stderr, original); + fprintf(stderr, " [%s]\r\n", text); + fail("erl_encode and erl_term_len do not agree"); + } else if ((new_terms = erl_decode(encoded)) == NULL) { fail("failed to decode terms"); } diff --git a/lib/et/src/et_selector.erl b/lib/et/src/et_selector.erl index f39f21aa70..c8e9c907b2 100644 --- a/lib/et/src/et_selector.erl +++ b/lib/et/src/et_selector.erl @@ -115,13 +115,13 @@ change_pattern({Mod, Pattern}) when is_atom(Mod) -> old_ctp({Mod, _Fun, Args}) -> case Mod of - et -> ignore; + et -> {ok, ignore}; _ -> dbg:ctp({Mod, report_event, Args}) end. old_tp({Mod, _Fun, Args}, Pattern) -> case Mod of - et -> ignore; + et -> {ok, ignore}; _ -> dbg:tp({Mod, report_event, Args}, Pattern) end. diff --git a/lib/eunit/doc/src/Makefile b/lib/eunit/doc/src/Makefile index 19be96d763..2cdc579275 100644 --- a/lib/eunit/doc/src/Makefile +++ b/lib/eunit/doc/src/Makefile @@ -146,7 +146,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* rm -f $(MAN3DIR)/* - rm -f $(XML_CHAPTER_FILES) *.html + rm -f $(XML_REF3_FILES) $(XML_CHAPTER_FILES) *.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/eunit/src/eunit.erl b/lib/eunit/src/eunit.erl index 59084a52fb..4a86a108cf 100644 --- a/lib/eunit/src/eunit.erl +++ b/lib/eunit/src/eunit.erl @@ -16,7 +16,7 @@ %% $Id: eunit.erl 339 2009-04-05 14:10:47Z rcarlsson $ %% %% @copyright 2004-2009 Micka�l R�mond, Richard Carlsson -%% @author Micka�l R�mond <[email protected]> +%% @author Mickaël Rémond <[email protected]> %% [http://www.process-one.net/] %% @author Richard Carlsson <[email protected]> %% [http://user.it.uu.se/~richardc/] diff --git a/lib/eunit/src/eunit_surefire.erl b/lib/eunit/src/eunit_surefire.erl index aeda31d251..eb994a990a 100644 --- a/lib/eunit/src/eunit_surefire.erl +++ b/lib/eunit/src/eunit_surefire.erl @@ -15,7 +15,7 @@ %% %% $Id: $ %% -%% @author Micka�l R�mond <[email protected]> +%% @author Mickaël Rémond <[email protected]> %% @copyright 2009 Micka�l R�mond, Paul Guyot %% @see eunit %% @doc Surefire reports for EUnit (Format used by Maven and Atlassian diff --git a/lib/inets/doc/src/ftp.xml b/lib/inets/doc/src/ftp.xml index 25dfe716fc..ca902d8d9d 100644 --- a/lib/inets/doc/src/ftp.xml +++ b/lib/inets/doc/src/ftp.xml @@ -107,7 +107,7 @@ <tag>{mode, Mode}</tag> <item> <marker id="mode"></marker> - <p>Mode = <c>active | passive</c> </p>> + <p>Mode = <c>active | passive</c> </p> <p>Default is <c>passive</c>. </p> </item> diff --git a/lib/inets/doc/src/http_client.xml b/lib/inets/doc/src/http_client.xml index ea8053cafa..672ea3fa98 100644 --- a/lib/inets/doc/src/http_client.xml +++ b/lib/inets/doc/src/http_client.xml @@ -1,4 +1,4 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE chapter SYSTEM "chapter.dtd"> <chapter> @@ -57,7 +57,7 @@ [{inets, [{services, [{httpc, PropertyList}]}]}] </pre> <p>For valid properties see - <seealso marker="http">httpc(3)</seealso>. </p> + <seealso marker="httpc">httpc(3)</seealso>. </p> </section> <section> diff --git a/lib/inets/doc/src/httpc.xml b/lib/inets/doc/src/httpc.xml index 9c8df28fec..c20358178b 100644 --- a/lib/inets/doc/src/httpc.xml +++ b/lib/inets/doc/src/httpc.xml @@ -1,4 +1,4 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd"> <erlref> @@ -172,7 +172,8 @@ ssl_options() = {verify, code()} | {autoredirect, boolean()} | {proxy_auth, {userstring(), passwordstring()}} | {version, http_version()} | - {relaxed, boolean()}</v> + {relaxed, boolean()} | + {url_encode, boolean()}</v> <v>timeout() = integer() >= 0 | infinity</v> <v>Options = options()</v> <v>options() = [option()]</v> @@ -276,6 +277,11 @@ ssl_options() = {verify, code()} | <p>Defaults to <c>false</c>. </p> </item> + <tag><c><![CDATA[url_encode]]></c></tag> + <item> + <p>Will apply Percent-encoding, also known as URL encoding on the URL.</p> + <p>Defaults to <c>false</c>. </p> + </item> </taglist> <p>Option (<c>option()</c>) details: </p> @@ -342,7 +348,7 @@ ssl_options() = {verify, code()} | <p>Socket options to be used for this and subsequent request(s). </p> <p>Overrides any value set by the - <seealso marker="set_options">set_options</seealso> + <seealso marker="#set_options">set_options</seealso> function. </p> <p>Note that the validity of the options are <em>not</em> checked in any way. </p> @@ -632,4 +638,3 @@ apply(Module, Function, [ReplyInfo | Args]) </section> </erlref> - diff --git a/lib/inets/doc/src/httpd.xml b/lib/inets/doc/src/httpd.xml index 847605fe93..62f4e18f82 100644 --- a/lib/inets/doc/src/httpd.xml +++ b/lib/inets/doc/src/httpd.xml @@ -1,4 +1,4 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd"> <erlref> @@ -421,7 +421,7 @@ bytes Beware of trailing space in Replacement that will be used. If you must have a space in Re use e.g the character encoding - <code>\040</code> see <seealso marker="re">re(3)</seealso>. + <code>\040</code> see <seealso marker="stdlib:re">re(3)</seealso>. </item> <tag>{directory_index, [string()]}</tag> @@ -931,6 +931,10 @@ bytes connection }). </code> + + <p>To acess the record in your callback-module use </p> + <code> -include_lib("inets/include/httpd.hrl"). </code> + <p>The fields of the <c>mod</c> record has the following meaning: </p> <taglist> @@ -978,10 +982,10 @@ bytes <c>parsed_header</c> contains all HTTP header fields from the HTTP-request stored in a list as key-value tuples. See RFC 2616 for a listing of all header fields. For example the date field - would be stored as: <c>{"date","Wed, 15 Oct 1997 14:35:17 GMT"}. + would be stored as: <c>{"date","Wed, 15 Oct 1997 14:35:17 GMT"} </c>. RFC 2616 defines that HTTP is a case insensitive protocol and the header fields may be in lower case or upper case. Httpd will - ensure that all header field names are in lower case. </c>. + ensure that all header field names are in lower case. </item> <tag><c>entity_body</c></tag> <item>The <c>Entity-Body</c> as defined diff --git a/lib/inets/include/httpd.hrl b/lib/inets/include/httpd.hrl new file mode 100644 index 0000000000..a7e63ca670 --- /dev/null +++ b/lib/inets/include/httpd.hrl @@ -0,0 +1,41 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-ifndef(httpd_hrl). +-define(httpd_hrl, true). + +-include_lib("kernel/include/file.hrl"). + +-record(init_data,{peername,resolve}). + +-record(mod,{init_data, + data=[], + socket_type=ip_comm, + socket, + config_db, + method, + absolute_uri=[], + request_uri, + http_version, + request_line, + parsed_header=[], + entity_body, + connection}). +-endif. % -ifdef(httpd_hrl). diff --git a/lib/inets/src/http_client/Makefile b/lib/inets/src/http_client/Makefile index 575c6efaec..0397b48ab2 100644 --- a/lib/inets/src/http_client/Makefile +++ b/lib/inets/src/http_client/Makefile @@ -51,7 +51,6 @@ MODULES = \ httpc_profile_sup \ httpc_response \ httpc_request \ - http_uri \ HRL_FILES = httpc_internal.hrl diff --git a/lib/inets/src/http_client/httpc.erl b/lib/inets/src/http_client/httpc.erl index 851364001c..ae754fab94 100644 --- a/lib/inets/src/http_client/httpc.erl +++ b/lib/inets/src/http_client/httpc.erl @@ -442,18 +442,23 @@ handle_request(Method, Url, HeadersRecord = header_record(NewHeaders, Host2, HTTPOptions), Receiver = proplists:get_value(receiver, Options), SocketOpts = proplists:get_value(socket_opts, Options), + UrlEncodeBool = HTTPOptions#http_options.url_encode, + MaybeEscPath = url_encode(Path, UrlEncodeBool), + MaybeEscQuery = url_encode(Query, UrlEncodeBool), + AbsUri = url_encode(Url, UrlEncodeBool), + Request = #request{from = Receiver, scheme = Scheme, address = {Host, Port}, - path = Path, - pquery = Query, + path = MaybeEscPath, + pquery = MaybeEscQuery, method = Method, headers = HeadersRecord, content = {ContentType, Body}, settings = HTTPOptions, - abs_uri = Url, + abs_uri = AbsUri, userinfo = UserInfo, - stream = Stream, + stream = Stream, headers_as_is = headers_as_is(Headers, Options), socket_opts = SocketOpts, started = Started}, @@ -471,6 +476,10 @@ handle_request(Method, Url, Error end. +url_encode(URI, true) -> + http_uri:encode(URI); +url_encode(URI, false) -> + URI. handle_answer(RequestId, false, _) -> {ok, RequestId}; @@ -578,12 +587,8 @@ http_options_default() -> (_) -> error end, - AutoRedirectPost = fun(Value) when (Value =:= true) orelse - (Value =:= false) -> - {ok, Value}; - (_) -> - error - end, + AutoRedirectPost = boolfun(), + SslPost = fun(Value) when is_list(Value) -> {ok, {?HTTP_DEFAULT_SSL_KIND, Value}}; ({ssl, SslOptions}) when is_list(SslOptions) -> @@ -601,12 +606,8 @@ http_options_default() -> (_) -> error end, - RelaxedPost = fun(Value) when (Value =:= true) orelse - (Value =:= false) -> - {ok, Value}; - (_) -> - error - end, + RelaxedPost = boolfun(), + ConnTimeoutPost = fun(Value) when is_integer(Value) andalso (Value >= 0) -> {ok, Value}; @@ -615,6 +616,8 @@ http_options_default() -> (_) -> error end, + + UrlDecodePost = boolfun(), [ {version, {value, "HTTP/1.1"}, #http_options.version, VersionPost}, {timeout, {value, ?HTTP_REQUEST_TIMEOUT}, #http_options.timeout, TimeoutPost}, @@ -622,18 +625,21 @@ http_options_default() -> {ssl, {value, {?HTTP_DEFAULT_SSL_KIND, []}}, #http_options.ssl, SslPost}, {proxy_auth, {value, undefined}, #http_options.proxy_auth, ProxyAuthPost}, {relaxed, {value, false}, #http_options.relaxed, RelaxedPost}, + {url_encode, {value, false}, #http_options.url_encode, UrlDecodePost}, %% this field has to be *after* the timeout option (as that field is used for the default value) {connect_timeout, {field, #http_options.timeout}, #http_options.connect_timeout, ConnTimeoutPost} ]. +boolfun() -> + fun(Value) when (Value =:= true) orelse + (Value =:= false) -> + {ok, Value}; + (_) -> + error + end. request_options_defaults() -> - VerifyBoolean = - fun(Value) when ((Value =:= true) orelse (Value =:= false)) -> - ok; - (_) -> - error - end, + VerifyBoolean = boolfun(), VerifySync = VerifyBoolean, diff --git a/lib/inets/src/http_client/httpc_handler.erl b/lib/inets/src/http_client/httpc_handler.erl index 8af6613fa2..cb6f3e2841 100644 --- a/lib/inets/src/http_client/httpc_handler.erl +++ b/lib/inets/src/http_client/httpc_handler.erl @@ -713,33 +713,38 @@ terminate(normal, profile_name = ProfileName, request = Request, timers = Timers, - pipeline = Pipeline}) -> + pipeline = Pipeline, + keep_alive = KeepAlive} = State) -> ?hcrt("terminate(normal) - remote close", [{id, Id}, {profile, ProfileName}]), %% Clobber session (catch httpc_manager:delete_session(Id, ProfileName)), + maybe_retry_queue(Pipeline, State), + maybe_retry_queue(KeepAlive, State), + %% Cancel timers - #timers{request_timers = ReqTmrs, queue_timer = QTmr} = Timers, - cancel_timer(QTmr, timeout_queue), - lists:foreach(fun({_, Timer}) -> cancel_timer(Timer, timeout) end, - ReqTmrs), + cancel_timers(Timers), %% Maybe deliver answers to requests - deliver_answers([Request | queue:to_list(Pipeline)]), + deliver_answer(Request), %% And, just in case, close our side (**really** overkill) http_transport:close(SocketType, Socket); -terminate(_, #state{session = #session{id = Id, - socket = Socket, - socket_type = SocketType}, +terminate(Reason, #state{session = #session{id = Id, + socket = Socket, + socket_type = SocketType}, request = undefined, profile_name = ProfileName, timers = Timers, pipeline = Pipeline, keep_alive = KeepAlive} = State) -> + ?hcrt("terminate", + [{id, Id}, {profile, ProfileName}, {reason, Reason}]), + + %% Clobber session (catch httpc_manager:delete_session(Id, ProfileName)), maybe_retry_queue(Pipeline, State), @@ -772,59 +777,55 @@ maybe_send_answer(#request{from = answer_sent}, _Reason, State) -> maybe_send_answer(Request, Answer, State) -> answer_request(Request, Answer, State). -deliver_answers([]) -> - ?hcrd("deliver answer done", []), - ok; -deliver_answers([#request{id = Id, from = From} = Request | Requests]) +deliver_answer(#request{id = Id, from = From} = Request) when is_pid(From) -> Response = httpc_response:error(Request, socket_closed_remotely), ?hcrd("deliver answer", [{id, Id}, {from, From}, {response, Response}]), - httpc_response:send(From, Response), - deliver_answers(Requests); -deliver_answers([Request|Requests]) -> + httpc_response:send(From, Response); +deliver_answer(Request) -> ?hcrd("skip deliver answer", [{request, Request}]), - deliver_answers(Requests). + ok. %%-------------------------------------------------------------------- %% Func: code_change(_OldVsn, State, Extra) -> {ok, NewState} %% Purpose: Convert process state when code is changed %%-------------------------------------------------------------------- -code_change(_, #state{request = Request, pipeline = Queue} = State, - [{from, '5.0.1'}, {to, '5.0.2'}]) -> - Settings = new_http_options(Request#request.settings), - NewRequest = Request#request{settings = Settings}, - NewQueue = new_queue(Queue, fun new_http_options/1), - {ok, State#state{request = NewRequest, pipeline = NewQueue}}; - -code_change(_, #state{request = Request, pipeline = Queue} = State, - [{from, '5.0.2'}, {to, '5.0.1'}]) -> - Settings = old_http_options(Request#request.settings), - NewRequest = Request#request{settings = Settings}, - NewQueue = new_queue(Queue, fun old_http_options/1), - {ok, State#state{request = NewRequest, pipeline = NewQueue}}; +%% code_change(_, #state{request = Request, pipeline = Queue} = State, +%% [{from, '5.0.1'}, {to, '5.0.2'}]) -> +%% Settings = new_http_options(Request#request.settings), +%% NewRequest = Request#request{settings = Settings}, +%% NewQueue = new_queue(Queue, fun new_http_options/1), +%% {ok, State#state{request = NewRequest, pipeline = NewQueue}}; + +%% code_change(_, #state{request = Request, pipeline = Queue} = State, +%% [{from, '5.0.2'}, {to, '5.0.1'}]) -> +%% Settings = old_http_options(Request#request.settings), +%% NewRequest = Request#request{settings = Settings}, +%% NewQueue = new_queue(Queue, fun old_http_options/1), +%% {ok, State#state{request = NewRequest, pipeline = NewQueue}}; code_change(_, State, _) -> {ok, State}. -new_http_options({http_options, TimeOut, AutoRedirect, SslOpts, - Auth, Relaxed}) -> - {http_options, "HTTP/1.1", TimeOut, AutoRedirect, SslOpts, - Auth, Relaxed}. - -old_http_options({http_options, _, TimeOut, AutoRedirect, - SslOpts, Auth, Relaxed}) -> - {http_options, TimeOut, AutoRedirect, SslOpts, Auth, Relaxed}. - -new_queue(Queue, Fun) -> - List = queue:to_list(Queue), - NewList = - lists:map(fun(Request) -> - Settings = - Fun(Request#request.settings), - Request#request{settings = Settings} - end, List), - queue:from_list(NewList). +%% new_http_options({http_options, TimeOut, AutoRedirect, SslOpts, +%% Auth, Relaxed}) -> +%% {http_options, "HTTP/1.1", TimeOut, AutoRedirect, SslOpts, +%% Auth, Relaxed}. + +%% old_http_options({http_options, _, TimeOut, AutoRedirect, +%% SslOpts, Auth, Relaxed}) -> +%% {http_options, TimeOut, AutoRedirect, SslOpts, Auth, Relaxed}. + +%% new_queue(Queue, Fun) -> +%% List = queue:to_list(Queue), +%% NewList = +%% lists:map(fun(Request) -> +%% Settings = +%% Fun(Request#request.settings), +%% Request#request{settings = Settings} +%% end, List), +%% queue:from_list(NewList). %%%-------------------------------------------------------------------- @@ -854,12 +855,18 @@ connect(SocketType, ToAddress, inet6fb4 -> Opts3 = [inet6 | Opts2], case http_transport:connect(SocketType, ToAddress, Opts3, Timeout) of - {error, Reason} when ((Reason =:= nxdomain) orelse - (Reason =:= eafnosupport)) -> + {error, _Reason} = Error -> Opts4 = [inet | Opts2], - http_transport:connect(SocketType, ToAddress, Opts4, Timeout); - Other -> - Other + case http_transport:connect(SocketType, + ToAddress, Opts4, Timeout) of + {error, _} -> + %% Reply with the "original" error + Error; + OK -> + OK + end; + OK -> + OK end; _ -> Opts3 = [IpFamily | Opts2], @@ -1440,6 +1447,12 @@ answer_request(#request{id = RequestId, from = From} = Request, Msg, timers = Timers#timers{request_timers = lists:delete(Timer, RequestTimers)}}. + +cancel_timers(#timers{request_timers = ReqTmrs, queue_timer = QTmr}) -> + cancel_timer(QTmr, timeout_queue), + CancelTimer = fun({_, Timer}) -> cancel_timer(Timer, timeout) end, + lists:foreach(CancelTimer, ReqTmrs). + cancel_timer(undefined, _) -> ok; cancel_timer(Timer, TimeoutMsg) -> diff --git a/lib/inets/src/http_client/httpc_internal.hrl b/lib/inets/src/http_client/httpc_internal.hrl index 3cdd95a02b..1d8a5b6a92 100644 --- a/lib/inets/src/http_client/httpc_internal.hrl +++ b/lib/inets/src/http_client/httpc_internal.hrl @@ -60,7 +60,11 @@ relaxed = false, %% integer() - ms before a connect times out - connect_timeout = ?HTTP_REQUEST_CTIMEOUT + connect_timeout = ?HTTP_REQUEST_CTIMEOUT, + + %% bool() - Use %-encoding rfc 2396 + url_encode + } ). diff --git a/lib/inets/src/http_client/httpc_manager.erl b/lib/inets/src/http_client/httpc_manager.erl index 1e1bde220b..591cb78c29 100644 --- a/lib/inets/src/http_client/httpc_manager.erl +++ b/lib/inets/src/http_client/httpc_manager.erl @@ -734,10 +734,11 @@ handle_connect_and_send(_StarterPid, ReqId, HandlerPid, Result, ok; [] -> - error_report(Profile, - "handler (~p) successfully started " - "for unknown request ~p => canceling", - [HandlerPid, ReqId]), + ?hcri("handler successfully started " + "for unknown request => canceling", + [{profile, Profile}, + {handler, HandlerPid}, + {request, ReqId}]), httpc_handler:cancel(ReqId, HandlerPid) end. diff --git a/lib/inets/src/http_lib/Makefile b/lib/inets/src/http_lib/Makefile index 5dac3b0c00..aaf3cfb995 100644 --- a/lib/inets/src/http_lib/Makefile +++ b/lib/inets/src/http_lib/Makefile @@ -45,7 +45,8 @@ MODULES = \ http_transport\ http_util \ http_request \ - http_response + http_response \ + http_uri HRL_FILES = http_internal.hrl diff --git a/lib/inets/src/http_lib/http_transport.erl b/lib/inets/src/http_lib/http_transport.erl index b8121852b8..0024d19fc1 100644 --- a/lib/inets/src/http_lib/http_transport.erl +++ b/lib/inets/src/http_lib/http_transport.erl @@ -192,24 +192,31 @@ listen_ip_comm(Addr, Port) -> case IpFamily of inet6fb4 -> Opts2 = [inet6 | Opts], + ?hlrt("try ipv6 listen", [{port, NewPort}, {opts, Opts2}]), case (catch gen_tcp:listen(NewPort, Opts2)) of {error, Reason} when ((Reason =:= nxdomain) orelse (Reason =:= eafnosupport)) -> Opts3 = [inet | Opts], + ?hlrt("ipv6 listen failed - try ipv4 instead", + [{reason, Reason}, {port, NewPort}, {opts, Opts3}]), gen_tcp:listen(NewPort, Opts3); %% This is when a given hostname has resolved to a %% IPv4-address. The inet6-option together with a %% {ip, IPv4} option results in badarg - {'EXIT', _} -> + {'EXIT', Reason} -> Opts3 = [inet | Opts], + ?hlrt("ipv6 listen exit - try ipv4 instead", + [{reason, Reason}, {port, NewPort}, {opts, Opts3}]), gen_tcp:listen(NewPort, Opts3); Other -> + ?hlrt("ipv6 listen done", [{other, Other}]), Other end; _ -> Opts2 = [IpFamily | Opts], + ?hlrt("listen", [{port, NewPort}, {opts, Opts2}]), gen_tcp:listen(NewPort, Opts2) end. diff --git a/lib/inets/src/http_client/http_uri.erl b/lib/inets/src/http_lib/http_uri.erl index 615a0d8ec4..44b9face0b 100644 --- a/lib/inets/src/http_client/http_uri.erl +++ b/lib/inets/src/http_lib/http_uri.erl @@ -1,26 +1,26 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2006-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2006-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% %% -module(http_uri). --export([parse/1]). +-export([parse/1, encode/1, decode/1]). %%%========================================================================= %%% API @@ -34,10 +34,25 @@ parse(AbsURI) -> {UserInfo, Host, Port, Path, Query} -> {Scheme, UserInfo, Host, Port, Path, Query}; _ -> - {error, {malformed_url, AbsURI}} + {error, {malformed_url, AbsURI}} end end. +encode(URI) -> + Reserved = sets:from_list([$;, $:, $@, $&, $=, $+, $,, $/, $?, + $#, $[, $], $<, $>, $\", ${, $}, $|, + $\\, $', $^, $%, $ ]), + lists:append(lists:map(fun(Char) -> + uri_encode(Char, Reserved) + end, URI)). + +decode([$%,Hex1,Hex2|Rest]) -> + [hex2dec(Hex1)*16+hex2dec(Hex2)|decode(Rest)]; +decode([First|Rest]) -> + [First|decode(Rest)]; +decode([]) -> + []. + %%%======================================================================== %%% Internal functions %%%======================================================================== @@ -56,7 +71,7 @@ parse_scheme(AbsURI) -> parse_uri_rest(Scheme, "//" ++ URIPart) -> - {Authority, PathQuery} = + {Authority, PathQuery} = case split_uri(URIPart, "/", URIPart, 1, 0) of Split = {_, _} -> Split; @@ -68,7 +83,7 @@ parse_uri_rest(Scheme, "//" ++ URIPart) -> {URIPart,""} end end, - + {UserInfo, HostPort} = split_uri(Authority, "@", {"", Authority}, 1, 1), {Host, Port} = parse_host_port(Scheme, HostPort), {Path, Query} = parse_path_query(PathQuery), @@ -78,7 +93,6 @@ parse_uri_rest(Scheme, "//" ++ URIPart) -> parse_path_query(PathQuery) -> {Path, Query} = split_uri(PathQuery, "\\?", {PathQuery, ""}, 1, 0), {path(Path), Query}. - parse_host_port(Scheme,"[" ++ HostPort) -> %ipv6 DefaultPort = default_port(Scheme), @@ -90,12 +104,12 @@ parse_host_port(Scheme, HostPort) -> DefaultPort = default_port(Scheme), {Host, Port} = split_uri(HostPort, ":", {HostPort, DefaultPort}, 1, 1), {Host, int_port(Port)}. - + split_uri(UriPart, SplitChar, NoMatchResult, SkipLeft, SkipRight) -> case inets_regexp:first_match(UriPart, SplitChar) of {match, Match, _} -> {string:substr(UriPart, 1, Match - SkipLeft), - string:substr(UriPart, Match + SkipRight, length(UriPart))}; + string:substr(UriPart, Match + SkipRight, length(UriPart))}; nomatch -> NoMatchResult end. @@ -114,3 +128,15 @@ path("") -> "/"; path(Path) -> Path. + +uri_encode(Char, Reserved) -> + case sets:is_element(Char, Reserved) of + true -> + [ $% | http_util:integer_to_hexlist(Char)]; + false -> + [Char] + end. + +hex2dec(X) when (X>=$0) andalso (X=<$9) -> X-$0; +hex2dec(X) when (X>=$A) andalso (X=<$F) -> X-$A+10; +hex2dec(X) when (X>=$a) andalso (X=<$f) -> X-$a+10. diff --git a/lib/inets/src/http_server/Makefile b/lib/inets/src/http_server/Makefile index 879e605217..bdd8c5ee3c 100644 --- a/lib/inets/src/http_server/Makefile +++ b/lib/inets/src/http_server/Makefile @@ -82,7 +82,9 @@ MODULES = \ mod_security \ mod_security_server -HRL_FILES = httpd.hrl httpd_internal.hrl mod_auth.hrl +INCLUDE = ../../include + +HRL_FILES = $(INCLUDE)/httpd.hrl httpd_internal.hrl mod_auth.hrl ERL_FILES = $(MODULES:%=%.erl) @@ -98,9 +100,9 @@ include ../inets_app/inets.mk ERL_COMPILE_FLAGS += \ $(INETS_FLAGS) \ $(INETS_ERL_COMPILE_FLAGS) \ - -I../../include \ + -I$(INCLUDE) \ -I../inets_app \ - -I../http_lib + -I../http_lib \ # ---------------------------------------------------- diff --git a/lib/inets/src/http_server/httpd.erl b/lib/inets/src/http_server/httpd.erl index fb5fa1c758..93608dbf96 100644 --- a/lib/inets/src/http_server/httpd.erl +++ b/lib/inets/src/http_server/httpd.erl @@ -24,7 +24,6 @@ -include("httpd.hrl"). - %% Behavior callbacks -export([ start_standalone/1, @@ -271,8 +270,8 @@ foreach([KeyValue|Rest]) -> {ok, Plus2Space, _} = inets_regexp:gsub(KeyValue,"[\+]"," "), case inets_regexp:split(Plus2Space,"=") of {ok,[Key|Value]} -> - [{httpd_util:decode_hex(Key), - httpd_util:decode_hex(lists:flatten(Value))}|foreach(Rest)]; + [{http_uri:decode(Key), + http_uri:decode(lists:flatten(Value))}|foreach(Rest)]; {ok,_} -> foreach(Rest) end. diff --git a/lib/inets/src/http_server/httpd.hrl b/lib/inets/src/http_server/httpd.hrl deleted file mode 100644 index 0db8a029bb..0000000000 --- a/lib/inets/src/http_server/httpd.hrl +++ /dev/null @@ -1,82 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% -%% - --include_lib("kernel/include/file.hrl"). - --ifndef(SERVER_SOFTWARE). --define(SERVER_SOFTWARE,"inets/develop"). % Define in Makefile! --endif. --define(SERVER_PROTOCOL,"HTTP/1.1"). --define(DEFAULT_MODS, [mod_alias, mod_auth, mod_esi, mod_actions, mod_cgi, - mod_dir, mod_get, mod_head, mod_log, mod_disk_log]). --define(SOCKET_CHUNK_SIZE,8192). --define(SOCKET_MAX_POLL,25). --define(FILE_CHUNK_SIZE,64*1024). --define(GATEWAY_INTERFACE,"CGI/1.1"). --define(NICE(Reason),lists:flatten(atom_to_list(?MODULE)++": "++Reason)). --define(DEFAULT_CONTEXT, - [{errmsg,"[an error occurred while processing this directive]"}, - {timefmt,"%A, %d-%b-%y %T %Z"}, - {sizefmt,"abbrev"}]). - - --ifdef(inets_error). --define(ERROR(Format, Args), io:format("E(~p:~p:~p) : "++Format++"~n", - [self(),?MODULE,?LINE]++Args)). --else. --define(ERROR(F,A),[]). --endif. - --ifdef(inets_log). --define(LOG(Format, Args), io:format("L(~p:~p:~p) : "++Format++"~n", - [self(),?MODULE,?LINE]++Args)). --else. --define(LOG(F,A),[]). --endif. - --ifdef(inets_debug). --define(DEBUG(Format, Args), io:format("D(~p:~p:~p) : "++Format++"~n", - [self(),?MODULE,?LINE]++Args)). --else. --define(DEBUG(F,A),[]). --endif. - --ifdef(inets_cdebug). --define(CDEBUG(Format, Args), io:format("C(~p:~p:~p) : "++Format++"~n", - [self(),?MODULE,?LINE]++Args)). --else. --define(CDEBUG(F,A),[]). --endif. - - --record(init_data,{peername,resolve}). --record(mod,{init_data, - data=[], - socket_type=ip_comm, - socket, - config_db, - method, - absolute_uri=[], - request_uri, - http_version, - request_line, - parsed_header=[], - entity_body, - connection}). diff --git a/lib/inets/src/http_server/httpd_acceptor.erl b/lib/inets/src/http_server/httpd_acceptor.erl index c261eff6b2..bcebb6a9e3 100644 --- a/lib/inets/src/http_server/httpd_acceptor.erl +++ b/lib/inets/src/http_server/httpd_acceptor.erl @@ -21,6 +21,7 @@ -include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). %% Internal application API -export([start_link/5, start_link/6]). diff --git a/lib/inets/src/http_server/httpd_conf.erl b/lib/inets/src/http_server/httpd_conf.erl index 8438c4037e..f4d8a6c09f 100644 --- a/lib/inets/src/http_server/httpd_conf.erl +++ b/lib/inets/src/http_server/httpd_conf.erl @@ -31,8 +31,8 @@ validate_properties/1]). -define(VMODULE,"CONF"). --include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("httpd.hrl"). -include_lib("inets/src/http_lib/http_internal.hrl"). diff --git a/lib/inets/src/http_server/httpd_file.erl b/lib/inets/src/http_server/httpd_file.erl index 5fd529100e..7e21d9e158 100644 --- a/lib/inets/src/http_server/httpd_file.erl +++ b/lib/inets/src/http_server/httpd_file.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2009. All Rights Reserved. +%% Copyright Ericsson AB 2006-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,11 +22,13 @@ -export([handle_error/4]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). + handle_error(eacces, Op, ModData, Path) -> - handle_error(403, Op, ModData, Path,""); + handle_error(403, Op, ModData, Path,"Forbidden"); handle_error(enoent, Op, ModData, Path) -> - handle_error(404, Op, ModData, Path,""); + handle_error(404, Op, ModData, Path,"File not found"); handle_error(enotdir, Op, ModData, Path) -> handle_error(404, Op, ModData, Path, ": A component of the file name is not a directory"); @@ -34,8 +36,8 @@ handle_error(emfile, Op, _ModData, Path) -> handle_error(500, Op, none, Path, ": To many open files"); handle_error({enfile,_}, Op, _ModData, Path) -> handle_error(500, Op, none, Path, ": File table overflow"); -handle_error(_Reason, Op, _ModData, Path) -> - handle_error(500, Op, none, Path, ""). +handle_error(_Reason, Op, ModData, Path) -> + handle_error(404, Op, ModData, Path, "File not found"). handle_error(StatusCode, Op, none, Path, Reason) -> {StatusCode, none, ?NICE("Can't " ++ Op ++ Path ++ Reason)}; diff --git a/lib/inets/src/http_server/httpd_internal.hrl b/lib/inets/src/http_server/httpd_internal.hrl index 38b0ddefd3..108469ea0a 100644 --- a/lib/inets/src/http_server/httpd_internal.hrl +++ b/lib/inets/src/http_server/httpd_internal.hrl @@ -21,7 +21,50 @@ -ifndef(httpd_internal_hrl). -define(httpd_internal_hrl, true). --include_lib("inets/src/inets_app/inets_internal.hrl"). +-ifndef(SERVER_SOFTWARE). +-define(SERVER_SOFTWARE,"inets/develop"). % Define in Makefile! +-endif. +-define(SERVER_PROTOCOL,"HTTP/1.1"). +-define(DEFAULT_MODS, [mod_alias, mod_auth, mod_esi, mod_actions, mod_cgi, + mod_dir, mod_get, mod_head, mod_log, mod_disk_log]). +-define(SOCKET_CHUNK_SIZE,8192). +-define(SOCKET_MAX_POLL,25). +-define(FILE_CHUNK_SIZE,64*1024). +-define(GATEWAY_INTERFACE,"CGI/1.1"). +-define(NICE(Reason),lists:flatten(atom_to_list(?MODULE)++": "++Reason)). +-define(DEFAULT_CONTEXT, + [{errmsg,"[an error occurred while processing this directive]"}, + {timefmt,"%A, %d-%b-%y %T %Z"}, + {sizefmt,"abbrev"}]). + + +-ifdef(inets_error). +-define(ERROR(Format, Args), io:format("E(~p:~p:~p) : "++Format++"~n", + [self(),?MODULE,?LINE]++Args)). +-else. +-define(ERROR(F,A),[]). +-endif. + +-ifdef(inets_log). +-define(LOG(Format, Args), io:format("L(~p:~p:~p) : "++Format++"~n", + [self(),?MODULE,?LINE]++Args)). +-else. +-define(LOG(F,A),[]). +-endif. + +-ifdef(inets_debug). +-define(DEBUG(Format, Args), io:format("D(~p:~p:~p) : "++Format++"~n", + [self(),?MODULE,?LINE]++Args)). +-else. +-define(DEBUG(F,A),[]). +-endif. + +-ifdef(inets_cdebug). +-define(CDEBUG(Format, Args), io:format("C(~p:~p:~p) : "++Format++"~n", + [self(),?MODULE,?LINE]++Args)). +-else. +-define(CDEBUG(F,A),[]). +-endif. -define(SERVICE, httpd). -define(hdri(Label, Content), ?report_important(Label, ?SERVICE, Content)). diff --git a/lib/inets/src/http_server/httpd_request.erl b/lib/inets/src/http_server/httpd_request.erl index 883acbf585..7084d9824a 100644 --- a/lib/inets/src/http_server/httpd_request.erl +++ b/lib/inets/src/http_server/httpd_request.erl @@ -304,9 +304,9 @@ validate_uri(RequestURI) -> UriNoQueryNoHex = case string:str(RequestURI, "?") of 0 -> - (catch httpd_util:decode_hex(RequestURI)); + (catch http_uri:decode(RequestURI)); Ndx -> - (catch httpd_util:decode_hex(string:left(RequestURI, Ndx))) + (catch http_uri:decode(string:left(RequestURI, Ndx))) end, case UriNoQueryNoHex of {'EXIT',_Reason} -> diff --git a/lib/inets/src/http_server/httpd_script_env.erl b/lib/inets/src/http_server/httpd_script_env.erl index a742cbef76..d3115150b0 100644 --- a/lib/inets/src/http_server/httpd_script_env.erl +++ b/lib/inets/src/http_server/httpd_script_env.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% Copyright Ericsson AB 2005-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -23,6 +23,7 @@ -export([create_env/3]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). %%%========================================================================= %%% Internal application API diff --git a/lib/inets/src/http_server/httpd_sup.erl b/lib/inets/src/http_server/httpd_sup.erl index 1507c6852a..f94e5459c1 100644 --- a/lib/inets/src/http_server/httpd_sup.erl +++ b/lib/inets/src/http_server/httpd_sup.erl @@ -37,7 +37,7 @@ -define(TIMEOUT, 15000). -include("httpd_internal.hrl"). - +-include("inets_internal.hrl"). %%%========================================================================= %%% API diff --git a/lib/inets/src/http_server/httpd_util.erl b/lib/inets/src/http_server/httpd_util.erl index cfad79638f..789f12652b 100644 --- a/lib/inets/src/http_server/httpd_util.erl +++ b/lib/inets/src/http_server/httpd_util.erl @@ -21,7 +21,7 @@ -export([ip_address/2, lookup/2, lookup/3, multi_lookup/2, lookup_mime/2, lookup_mime/3, lookup_mime_default/2, lookup_mime_default/3, reason_phrase/1, message/3, rfc1123_date/0, - rfc1123_date/1, day/1, month/1, decode_hex/1, + rfc1123_date/1, day/1, month/1, flatlength/1, split_path/1, split_script_path/1, suffix/1, split/3, uniq/1, make_name/2,make_name/3,make_name/4,strip/1, @@ -32,7 +32,7 @@ dir_validate/2, file_validate/2, mime_type_validate/1, mime_types_validate/1, custom_date/0]). --export([encode_hex/1]). +-export([encode_hex/1, decode_hex/1]). -include_lib("kernel/include/file.hrl"). ip_address({_,_,_,_} = Address, _IpFamily) -> @@ -175,13 +175,13 @@ reason_phrase(_) -> "Internal Server Error". %% message message(301,URL,_) -> - "The document has moved <A HREF=\""++URL++"\">here</A>."; + "The document has moved <A HREF=\""++ maybe_encode(URL) ++"\">here</A>."; message(304, _URL,_) -> "The document has not been changed."; message(400,none,_) -> "Your browser sent a query that this server could not understand."; message(400,Msg,_) -> - "Your browser sent a query that this server could not understand. "++Msg; + "Your browser sent a query that this server could not understand. "++ maybe_encode(Msg); message(401,none,_) -> "This server could not verify that you are authorized to access the document you @@ -190,9 +190,9 @@ credentials (e.g., bad password), or your browser doesn't understand how to supply the credentials required."; message(403,RequestURI,_) -> - "You don't have permission to access "++RequestURI++" on this server."; + "You don't have permission to access "++ maybe_encode(RequestURI) ++" on this server."; message(404,RequestURI,_) -> - "The requested URL "++RequestURI++" was not found on this server."; + "The requested URL " ++ maybe_encode(RequestURI) ++ " was not found on this server."; message(408, Timeout, _) -> Timeout; message(412,none,_) -> @@ -200,7 +200,7 @@ message(412,none,_) -> message(413, Reason,_) -> "Entity: " ++ Reason; message(414,ReasonPhrase,_) -> - "Message "++ReasonPhrase++"."; + "Message "++ ReasonPhrase ++"."; message(416,ReasonPhrase,_) -> ReasonPhrase; @@ -216,15 +216,23 @@ message(501,{Method, RequestURI, HTTPVersion}, _ConfigDB) -> if is_atom(Method) -> atom_to_list(Method)++ - " to "++RequestURI++" ("++HTTPVersion++") not supported."; + " to "++ maybe_encode(RequestURI)++" ("++HTTPVersion++") not supported."; is_list(Method) -> Method++ - " to "++RequestURI++" ("++HTTPVersion++") not supported." + " to "++ maybe_encode(RequestURI)++" ("++HTTPVersion++") not supported." end; message(503, String, _ConfigDB) -> "This service in unavailable due to: "++String. +maybe_encode(URI) -> + case lists:member($%, URI) of + true -> + URI; + false -> + http_uri:encode(URI) + end. + %%convert_rfc_date(Date)->{{YYYY,MM,DD},{HH,MIN,SEC}} convert_request_date([D,A,Y,DateType| Rest])-> @@ -381,16 +389,11 @@ month(12) -> "Dec". %% decode_hex -decode_hex([$%,Hex1,Hex2|Rest]) -> - [hex2dec(Hex1)*16+hex2dec(Hex2)|decode_hex(Rest)]; -decode_hex([First|Rest]) -> - [First|decode_hex(Rest)]; -decode_hex([]) -> - []. +decode_hex(URI) -> + http_uri:decode(URI). -hex2dec(X) when (X>=$0) andalso (X=<$9) -> X-$0; -hex2dec(X) when (X>=$A) andalso (X=<$F) -> X-$A+10; -hex2dec(X) when (X>=$a) andalso (X=<$f) -> X-$a+10. +encode_hex(URI) -> + http_uri:encode(URI). %% flatlength flatlength(List) -> @@ -411,7 +414,7 @@ split_path(Path) -> case inets_regexp:match(Path,"[\?].*\$") of %% A QUERY_STRING exists! {match,Start,Length} -> - {httpd_util:decode_hex(string:substr(Path,1,Start-1)), + {http_uri:decode(string:substr(Path,1,Start-1)), string:substr(Path,Start,Length)}; %% A possible PATH_INFO exists! nomatch -> @@ -419,9 +422,9 @@ split_path(Path) -> end. split_path([],SoFar) -> - {httpd_util:decode_hex(lists:reverse(SoFar)),[]}; + {http_uri:decode(lists:reverse(SoFar)),[]}; split_path([$/|Rest],SoFar) -> - Path=httpd_util:decode_hex(lists:reverse(SoFar)), + Path=http_uri:decode(lists:reverse(SoFar)), case file:read_file_info(Path) of {ok,FileInfo} when FileInfo#file_info.type =:= regular -> {Path,[$/|Rest]}; @@ -454,7 +457,7 @@ pathinfo_querystring([C|Rest], SoFar) -> pathinfo_querystring(Rest, [C|SoFar]). split_script_path([$?|QueryString], SoFar) -> - Path = httpd_util:decode_hex(lists:reverse(SoFar)), + Path = http_uri:decode(lists:reverse(SoFar)), case file:read_file_info(Path) of {ok,FileInfo} when FileInfo#file_info.type =:= regular -> {Path, [$?|QueryString]}; @@ -464,7 +467,7 @@ split_script_path([$?|QueryString], SoFar) -> not_a_script end; split_script_path([], SoFar) -> - Path = httpd_util:decode_hex(lists:reverse(SoFar)), + Path = http_uri:decode(lists:reverse(SoFar)), case file:read_file_info(Path) of {ok,FileInfo} when FileInfo#file_info.type =:= regular -> {Path, []}; @@ -474,7 +477,7 @@ split_script_path([], SoFar) -> not_a_script end; split_script_path([$/|Rest], SoFar) -> - Path = httpd_util:decode_hex(lists:reverse(SoFar)), + Path = http_uri:decode(lists:reverse(SoFar)), case file:read_file_info(Path) of {ok, FileInfo} when FileInfo#file_info.type =:= regular -> {Path, [$/|Rest]}; @@ -608,9 +611,6 @@ hexlist_to_integer(List)-> %%---------------------------------------------------------------------- %%Converts an integer to an hexlist %%---------------------------------------------------------------------- -encode_hex(Num)-> - integer_to_hexlist(Num). - integer_to_hexlist(Num) when is_integer(Num) -> http_util:integer_to_hexlist(Num). @@ -735,7 +735,6 @@ valid_accept_timeout(A) -> valid_config(_) -> ok. - %%---------------------------------------------------------------------- %% Enable debugging, %%---------------------------------------------------------------------- diff --git a/lib/inets/src/http_server/mod_actions.erl b/lib/inets/src/http_server/mod_actions.erl index d50ed4b16c..c3946ff9b4 100644 --- a/lib/inets/src/http_server/mod_actions.erl +++ b/lib/inets/src/http_server/mod_actions.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -21,6 +21,7 @@ -export([do/1,load/2, store/2]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). %% do diff --git a/lib/inets/src/http_server/mod_alias.erl b/lib/inets/src/http_server/mod_alias.erl index 9c5a8cc1c6..0b9fe4cfe0 100644 --- a/lib/inets/src/http_server/mod_alias.erl +++ b/lib/inets/src/http_server/mod_alias.erl @@ -29,6 +29,7 @@ -include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). -define(VMODULE,"ALIAS"). diff --git a/lib/inets/src/http_server/mod_auth.erl b/lib/inets/src/http_server/mod_auth.erl index 07cafb4726..85a87ab884 100644 --- a/lib/inets/src/http_server/mod_auth.erl +++ b/lib/inets/src/http_server/mod_auth.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -38,6 +38,7 @@ -include("httpd.hrl"). -include("mod_auth.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). -define(VMODULE,"AUTH"). diff --git a/lib/inets/src/http_server/mod_auth_dets.erl b/lib/inets/src/http_server/mod_auth_dets.erl index bc6c2b70a0..a48725d5d9 100644 --- a/lib/inets/src/http_server/mod_auth_dets.erl +++ b/lib/inets/src/http_server/mod_auth_dets.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2009. All Rights Reserved. +%% Copyright Ericsson AB 1998-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -35,6 +35,7 @@ -export([store_directory_data/3]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). -include("mod_auth.hrl"). store_directory_data(_Directory, DirData, Server_root) -> diff --git a/lib/inets/src/http_server/mod_auth_plain.erl b/lib/inets/src/http_server/mod_auth_plain.erl index d88859d28a..c0a83711ba 100644 --- a/lib/inets/src/http_server/mod_auth_plain.erl +++ b/lib/inets/src/http_server/mod_auth_plain.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2009. All Rights Reserved. +%% Copyright Ericsson AB 1998-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,6 +22,8 @@ -include("httpd.hrl"). -include("mod_auth.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). + -define(VMODULE,"AUTH_PLAIN"). diff --git a/lib/inets/src/http_server/mod_auth_server.erl b/lib/inets/src/http_server/mod_auth_server.erl index 5f9e59be9d..947273bd9e 100644 --- a/lib/inets/src/http_server/mod_auth_server.erl +++ b/lib/inets/src/http_server/mod_auth_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,6 +22,7 @@ -include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). -behaviour(gen_server). diff --git a/lib/inets/src/http_server/mod_cgi.erl b/lib/inets/src/http_server/mod_cgi.erl index 33605b9698..c854166c29 100644 --- a/lib/inets/src/http_server/mod_cgi.erl +++ b/lib/inets/src/http_server/mod_cgi.erl @@ -27,6 +27,7 @@ -export([do/1, load/2, store/2]). -include("http_internal.hrl"). +-include("httpd_internal.hrl"). -include("httpd.hrl"). -define(VMODULE,"CGI"). diff --git a/lib/inets/src/http_server/mod_dir.erl b/lib/inets/src/http_server/mod_dir.erl index cdc7cc01e4..d791ee28e9 100644 --- a/lib/inets/src/http_server/mod_dir.erl +++ b/lib/inets/src/http_server/mod_dir.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -18,9 +18,11 @@ %% %% -module(mod_dir). --export([do/1]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). + +-export([do/1]). %% do @@ -57,7 +59,7 @@ do_dir(Info) -> case file:read_file_info(DefaultPath) of {ok,FileInfo} when FileInfo#file_info.type == directory -> DecodedRequestURI = - httpd_util:decode_hex(Info#mod.request_uri), + http_uri:decode(Info#mod.request_uri), ?DEBUG("do_dir -> ~n" " Path: ~p~n" " DefaultPath: ~p~n" diff --git a/lib/inets/src/http_server/mod_disk_log.erl b/lib/inets/src/http_server/mod_disk_log.erl index 95e0d00c70..5a3766de66 100644 --- a/lib/inets/src/http_server/mod_disk_log.erl +++ b/lib/inets/src/http_server/mod_disk_log.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -28,7 +28,7 @@ -define(VMODULE,"DISK_LOG"). -include("httpd.hrl"). - +-include("httpd_internal.hrl"). %%%========================================================================= %%% API diff --git a/lib/inets/src/http_server/mod_esi.erl b/lib/inets/src/http_server/mod_esi.erl index f7877aa9e2..929185a67a 100644 --- a/lib/inets/src/http_server/mod_esi.erl +++ b/lib/inets/src/http_server/mod_esi.erl @@ -30,6 +30,7 @@ -include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). -define(VMODULE,"ESI"). -define(DEFAULT_ERL_TIMEOUT,15000). diff --git a/lib/inets/src/http_server/mod_get.erl b/lib/inets/src/http_server/mod_get.erl index 9fd1fcec47..5cb30e3d97 100644 --- a/lib/inets/src/http_server/mod_get.erl +++ b/lib/inets/src/http_server/mod_get.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -20,7 +20,7 @@ -module(mod_get). -export([do/1]). -include("httpd.hrl"). - +-include("httpd_internal.hrl"). %% do do(Info) -> diff --git a/lib/inets/src/http_server/mod_head.erl b/lib/inets/src/http_server/mod_head.erl index 8b08d61651..c346fd4d23 100644 --- a/lib/inets/src/http_server/mod_head.erl +++ b/lib/inets/src/http_server/mod_head.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/inets/src/http_server/mod_htaccess.erl b/lib/inets/src/http_server/mod_htaccess.erl index d8835198f5..e1f66d01c8 100644 --- a/lib/inets/src/http_server/mod_htaccess.erl +++ b/lib/inets/src/http_server/mod_htaccess.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -23,6 +23,7 @@ -export([do/1, load/2, store/2]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Public methods that interface the eswapi %% diff --git a/lib/inets/src/http_server/mod_include.erl b/lib/inets/src/http_server/mod_include.erl index 534eba8a36..35f45bdd33 100644 --- a/lib/inets/src/http_server/mod_include.erl +++ b/lib/inets/src/http_server/mod_include.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -21,6 +21,7 @@ -export([do/1,parse/2,config/6,include/6,echo/6,fsize/6,flastmod/6,exec/6]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). -define(VMODULE,"INCLUDE"). @@ -186,9 +187,9 @@ document_uri(ConfigDB, RequestURI) -> FileName = string:substr(Path,Start,Length), case inets_regexp:match(VirtualPath, FileName++"\$") of {match, _, _} -> - httpd_util:decode_hex(VirtualPath)++AfterPath; + http_uri:decode(VirtualPath)++AfterPath; nomatch -> - string:strip(httpd_util:decode_hex(VirtualPath),right,$/)++ + string:strip(http_uri:decode(VirtualPath),right,$/)++ "/"++FileName++AfterPath end. diff --git a/lib/inets/src/http_server/mod_log.erl b/lib/inets/src/http_server/mod_log.erl index de24d5a569..c8a2ec0dc4 100644 --- a/lib/inets/src/http_server/mod_log.erl +++ b/lib/inets/src/http_server/mod_log.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -26,6 +26,7 @@ -export([do/1, load/2, store/2, remove/1]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). -define(VMODULE,"LOG"). %%%========================================================================= diff --git a/lib/inets/src/http_server/mod_range.erl b/lib/inets/src/http_server/mod_range.erl index 0698fb9099..a0408cba79 100644 --- a/lib/inets/src/http_server/mod_range.erl +++ b/lib/inets/src/http_server/mod_range.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -20,7 +20,7 @@ -module(mod_range). -export([do/1]). -include("httpd.hrl"). - +-include("httpd_internal.hrl"). %% do do(Info) -> diff --git a/lib/inets/src/http_server/mod_responsecontrol.erl b/lib/inets/src/http_server/mod_responsecontrol.erl index 79e2e1bdba..5d5b60cdbd 100644 --- a/lib/inets/src/http_server/mod_responsecontrol.erl +++ b/lib/inets/src/http_server/mod_responsecontrol.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,6 +22,7 @@ -export([do/1]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). do(Info) -> ?DEBUG("do -> response_control",[]), diff --git a/lib/inets/src/http_server/mod_security.erl b/lib/inets/src/http_server/mod_security.erl index 95793e1cfb..41988732ad 100644 --- a/lib/inets/src/http_server/mod_security.erl +++ b/lib/inets/src/http_server/mod_security.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2009. All Rights Reserved. +%% Copyright Ericsson AB 1998-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -32,6 +32,7 @@ -include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). -define(VMODULE,"SEC"). diff --git a/lib/inets/src/http_server/mod_security_server.erl b/lib/inets/src/http_server/mod_security_server.erl index 58060686b3..784b3eba70 100644 --- a/lib/inets/src/http_server/mod_security_server.erl +++ b/lib/inets/src/http_server/mod_security_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -45,6 +45,7 @@ -include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). -behaviour(gen_server). diff --git a/lib/inets/src/http_server/mod_trace.erl b/lib/inets/src/http_server/mod_trace.erl index df482228d8..7233925783 100644 --- a/lib/inets/src/http_server/mod_trace.erl +++ b/lib/inets/src/http_server/mod_trace.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/inets/src/inets_app/inets.appup.src b/lib/inets/src/inets_app/inets.appup.src index 84d8c9278d..0194c65db9 100644 --- a/lib/inets/src/inets_app/inets.appup.src +++ b/lib/inets/src/inets_app/inets.appup.src @@ -18,6 +18,11 @@ {"%VSN%", [ + {"5.5", + [ + {restart_application, inets} + ] + }, {"5.4", [ {restart_application, inets} @@ -29,6 +34,11 @@ [ {restart_application, inets} ] + }, + {"5.4", + [ + {restart_application, inets} + ] } ] }. diff --git a/lib/inets/test/Makefile b/lib/inets/test/Makefile index bb7f2186af..0492d33eab 100644 --- a/lib/inets/test/Makefile +++ b/lib/inets/test/Makefile @@ -288,16 +288,20 @@ release_spec: opt $(INSTALL_DATA) $(INETS_FILES) $(RELSYSDIR)/test @for d in $(DATADIRS); do \ echo "installing data dir $$d"; \ - echo $$d/TAR.exclude2 > $$d/TAR.exclude2; \ - cat $$d/TAR.exclude >> $$d/TAR.exclude2; \ - find $$d -name '*.contrib*' >> $$d/TAR.exclude2; \ - find $$d -name '*.keep*' >> $$d/TAR.exclude2; \ - find $$d -name '*.mkelem*' >> $$d/TAR.exclude2; \ - find $$d -name '*~' >> $$d/TAR.exclude2; \ - find $$d -name 'erl_crash.dump' >> $$d/TAR.exclude2; \ - find $$d -name 'core' >> $$d/TAR.exclude2; \ - find $$d -name '.cmake.state' >> $$d/TAR.exclude2; \ - tar cfX - $$d/TAR.exclude2 $$d | (cd $(RELSYSDIR)/test; tar xf -); \ + if test -f $$d/TAR.exclude; then \ + echo $$d/TAR.exclude2 > $$d/TAR.exclude2; \ + cat $$d/TAR.exclude >> $$d/TAR.exclude2; \ + find $$d -name '*.contrib*' >> $$d/TAR.exclude2; \ + find $$d -name '*.keep*' >> $$d/TAR.exclude2; \ + find $$d -name '*.mkelem*' >> $$d/TAR.exclude2; \ + find $$d -name '*~' >> $$d/TAR.exclude2; \ + find $$d -name 'erl_crash.dump' >> $$d/TAR.exclude2; \ + find $$d -name 'core' >> $$d/TAR.exclude2; \ + find $$d -name '.cmake.state' >> $$d/TAR.exclude2; \ + tar cfX - $$d/TAR.exclude2 $$d | (cd $(RELSYSDIR)/test; tar xf -); \ + else \ + tar cf - $$d | (cd $(RELSYSDIR)/test; tar xf -); \ + fi; \ done release_tests_spec: opt diff --git a/lib/inets/test/httpc_SUITE.erl b/lib/inets/test/httpc_SUITE.erl index 902e440c80..94d5a48ef6 100644 --- a/lib/inets/test/httpc_SUITE.erl +++ b/lib/inets/test/httpc_SUITE.erl @@ -254,9 +254,14 @@ init_per_testcase(Case, Timeout, Config) -> [{watchdog, Dog}, {local_server, Server} | TmpConfig2] end, + %% httpc:set_options([{proxy, {{?PROXY, ?PROXY_PORT}, + %% ["localhost", ?IPV6_LOCAL_HOST]}}]), + httpc:set_options([{proxy, {{?PROXY, ?PROXY_PORT}, - ["localhost", ?IPV6_LOCAL_HOST]}}]), - %% snmp:set_trace([gen_tcp, inet_tcp, prim_inet]), + ["localhost", ?IPV6_LOCAL_HOST]}}, + {ipfamily, inet6fb4}]), + + %% snmp:set_trace([gen_tcp]), NewConfig. @@ -471,7 +476,7 @@ http_relaxed(Config) when is_list(Config) -> DummyServerPid ! stop, ok = httpc:set_options([{ipv6, enabled}]), - %% ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + %% ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -489,7 +494,7 @@ http_dummy_pipe(Config) when is_list(Config) -> test_pipeline(URL), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. http_inets_pipe(doc) -> @@ -851,7 +856,7 @@ http_headers_dummy(Config) when is_list(Config) -> ], "text/plain", FooBar}, [], []), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -875,7 +880,7 @@ http_bad_response(Config) when is_list(Config) -> test_server:format("Wrong Statusline: ~p~n", [Reason]), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1157,7 +1162,7 @@ http_redirect(Config) when is_list(Config) -> tsp("http_redirect -> stop dummy server"), DummyServerPid ! stop, tsp("http_redirect -> reset ipfamily option (to inet6fb4)"), - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), tsp("http_redirect -> done"), ok; @@ -1181,7 +1186,7 @@ http_redirect_loop(Config) when is_list(Config) -> {ok, {{_,300,_}, [_ | _], _}} = httpc:request(get, {URL, []}, [], []), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. %%------------------------------------------------------------------------- @@ -1215,7 +1220,7 @@ http_internal_server_error(Config) when is_list(Config) -> ets:delete(unavailable), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1242,7 +1247,7 @@ http_userinfo(Config) when is_list(Config) -> httpc:request(get, {URLUnAuth, []}, [], []), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1271,9 +1276,9 @@ http_cookie(Config) when is_list(Config) -> ets:delete(cookie), - ok = httpc:set_options([{cookies, disabled}, {ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{cookies, disabled}]), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6************ + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. %%------------------------------------------------------------------------- @@ -1643,7 +1648,7 @@ http_stream_once(Config) when is_list(Config) -> p("http_stream_once -> stop dummy server", []), DummyServerPid ! stop, p("http_stream_once -> set ipfamily to inet6fb4", []), - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), p("http_stream_once -> done", []), ok. @@ -1847,7 +1852,7 @@ http_invalid_http(Config) when is_list(Config) -> test_server:format("Parse error: ~p ~n", [Reason]), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1901,7 +1906,7 @@ transfer_encoding_otp_6807(Config) when is_list(Config) -> "/capital_transfer_encoding.html", {ok, {{_,200,_}, [_|_], [_ | _]}} = httpc:request(URL), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1933,7 +1938,7 @@ empty_response_header_otp_6830(Config) when is_list(Config) -> URL = ?URL_START ++ integer_to_list(Port) ++ "/no_headers.html", {ok, {{_,200,_}, [], [_ | _]}} = httpc:request(URL), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1950,7 +1955,7 @@ no_content_204_otp_6982(Config) when is_list(Config) -> URL = ?URL_START ++ integer_to_list(Port) ++ "/no_content.html", {ok, {{_,204,_}, [], []}} = httpc:request(URL), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1968,7 +1973,7 @@ missing_CR_otp_7304(Config) when is_list(Config) -> URL = ?URL_START ++ integer_to_list(Port) ++ "/missing_CR.html", {ok, {{_,200,_}, _, [_ | _]}} = httpc:request(URL), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1990,7 +1995,7 @@ otp_7883_1(Config) when is_list(Config) -> {error, socket_closed_remotely} = httpc:request(URL), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. otp_7883_2(doc) -> @@ -2017,7 +2022,7 @@ otp_7883_2(Config) when is_list(Config) -> end, DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. diff --git a/lib/inets/test/httpd_basic_SUITE.erl b/lib/inets/test/httpd_basic_SUITE.erl index f86c1fcb49..9ba2e73942 100644 --- a/lib/inets/test/httpd_basic_SUITE.erl +++ b/lib/inets/test/httpd_basic_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2009. All Rights Reserved. +%% Copyright Ericsson AB 2007-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -25,13 +25,16 @@ %% Note: This directive should only be used in test suites. -compile(export_all). +-define(URL_START, "http://localhost:"). + all(doc) -> ["Basic test of httpd."]; all(suite) -> [ uri_too_long_414, - header_too_long_413 + header_too_long_413, + escaped_url_in_error_body ]. %%-------------------------------------------------------------------- @@ -131,6 +134,31 @@ header_too_long_413(Config) when is_list(Config) -> {version, "HTTP/1.1"}]), inets:stop(httpd, Pid). +escaped_url_in_error_body(doc) -> + ["Test Url-encoding see OTP-8940"]; +escaped_url_in_error_body(suite) -> + []; +escaped_url_in_error_body(Config) when is_list(Config) -> + HttpdConf = ?config(httpd_conf, Config), + {ok, Pid} = inets:start(httpd, [{port, 0} | HttpdConf]), + Info = httpd:info(Pid), + Port = proplists:get_value(port, Info), + Address = proplists:get_value(bind_address, Info), + Path = "/<b>this_is_bold<b>", + URL = ?URL_START ++ integer_to_list(Port) ++ Path, + EscapedPath = http_uri:encode(Path), + {ok, {404, Body}} = httpc:request(get, {URL, []}, + [{url_encode, true}], + [{version, "HTTP/1.0"}, {full_result, false}]), + EscapedPath = find_URL_path(string:tokens(Body, " ")), + {ok, {404, Body1}} = httpc:request(get, {URL, []}, [], + [{version, "HTTP/1.0"}, {full_result, false}]), + EscapedPath = find_URL_path(string:tokens(Body1, " ")), + inets:stop(httpd, Pid). - - +find_URL_path([]) -> + ""; +find_URL_path(["URL", URL | _]) -> + URL; +find_URL_path([_ | Rest]) -> + find_URL_path(Rest). diff --git a/lib/inets/test/inets_appup_test.erl b/lib/inets/test/inets_appup_test.erl index d580c6c4c5..2c9c687c91 100644 --- a/lib/inets/test/inets_appup_test.erl +++ b/lib/inets/test/inets_appup_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2009. All Rights Reserved. +%% Copyright Ericsson AB 2002-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -18,11 +18,12 @@ %% %% %%---------------------------------------------------------------------- -%% Purpose: Verify the application specifics of the Megaco application +%% Purpose: Verify the application specifics of the Inets application %%---------------------------------------------------------------------- -module(inets_appup_test). -compile(export_all). +-compile({no_auto_import,[error/1]}). -include("inets_test_lib.hrl"). diff --git a/lib/inets/vsn.mk b/lib/inets/vsn.mk index 5eff9e4e3f..f462290a99 100644 --- a/lib/inets/vsn.mk +++ b/lib/inets/vsn.mk @@ -1,5 +1,24 @@ +#-*-makefile-*- ; force emacs to enter makefile-mode + +# %CopyrightBegin% +# +# Copyright Ericsson AB 2001-2010. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% + APPLICATION = inets -INETS_VSN = 5.5 +INETS_VSN = 5.5.1 PRE_VSN = APP_VSN = "$(APPLICATION)-$(INETS_VSN)$(PRE_VSN)" diff --git a/lib/kernel/test/file_SUITE.erl b/lib/kernel/test/file_SUITE.erl index 17c47f871d..47592ddb14 100644 --- a/lib/kernel/test/file_SUITE.erl +++ b/lib/kernel/test/file_SUITE.erl @@ -3268,7 +3268,7 @@ large_file(Config) when is_list(Config) -> {{unix,sunos},{A,B,C}} when A == 5, B == 5, C >= 1; A == 5, B >= 6; A >= 6 -> do_large_file(Config); - {{unix,Unix},_} when Unix =:= linux; Unix =:= darwin -> + {{unix,Unix},_} when Unix =/= sunos -> N = unix_free(Config), io:format("Free: ~w KByte~n", [N]), if N < 5 * (1 bsl 20) -> @@ -3278,7 +3278,7 @@ large_file(Config) when is_list(Config) -> do_large_file(Config) end; _ -> - {skipped,"Only supported on Win32, Linux, or SunOS >= 5.5.1"} + {skipped,"Only supported on Win32, Unix or SunOS >= 5.5.1"} end. unix_free(Config) -> @@ -3290,7 +3290,7 @@ unix_free(Config) -> N. do_large_file(Config) -> - ?line Watchdog = ?t:timetrap(?t:minutes(4)), + ?line Watchdog = ?t:timetrap(?t:minutes(5)), %% ?line Name = filename:join(?config(priv_dir, Config), ?MODULE_STRING ++ "_large_file"), @@ -3329,6 +3329,17 @@ do_large_file(Config) -> ?line {ok,P} = ?FILE_MODULE:position(F, {eof,-L}), ?line {ok,Rs} = ?FILE_MODULE:read(F, L+1), ?line ok = ?FILE_MODULE:close(F), + %% Reopen the file with 'append'; used to fail on Windows causing + %% writes to go to the beginning of the file for files > 4GB. + ?line PL = P + L, + ?line PLL = PL + L, + ?line {ok,F1} = ?FILE_MODULE:open(Name, [raw,read,write,append]), + ?line ok = ?FILE_MODULE:write(F1, R), + ?line {ok,PLL} = ?FILE_MODULE:position(F1, {cur,0}), + ?line {ok,Rs} = ?FILE_MODULE:pread(F1, P, L), + ?line {ok,PL} = ?FILE_MODULE:position(F1, {eof,-L}), + ?line {ok,R} = ?FILE_MODULE:read(F1, L+1), + ?line ok = ?FILE_MODULE:close(F1), %% ?line Mref = erlang:monitor(process, Deleter), ?line Deleter ! {Tester,done}, diff --git a/lib/mnesia/src/mnesia.appup.src b/lib/mnesia/src/mnesia.appup.src index 47c9bf9979..22ef5178a7 100644 --- a/lib/mnesia/src/mnesia.appup.src +++ b/lib/mnesia/src/mnesia.appup.src @@ -1,7 +1,13 @@ %% -*- erlang -*- {"%VSN%", [ + {"4.4.15",[ + {update, mnesia_dumper, soft, soft_purge, soft_purge, []} + ]} ], [ + {"4.4.15",[ + {update, mnesia_dumper, soft, soft_purge, soft_purge, []} + ]} ] }. diff --git a/lib/mnesia/src/mnesia_dumper.erl b/lib/mnesia/src/mnesia_dumper.erl index f669d009c6..644133cf5d 100644 --- a/lib/mnesia/src/mnesia_dumper.erl +++ b/lib/mnesia/src/mnesia_dumper.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2009. All Rights Reserved. +%% Copyright Ericsson AB 1996-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -643,7 +643,7 @@ insert_op(Tid, _, {op, create_table, TabDef}, InPlace, InitBy) -> true -> ignore; false -> mnesia_log:open_log(temp, - mnesia_log:dcl_log_header(), + mnesia_log:dcd_log_header(), Dcd, false, false, diff --git a/lib/mnesia/vsn.mk b/lib/mnesia/vsn.mk index bce0f7b739..5b52bc6075 100644 --- a/lib/mnesia/vsn.mk +++ b/lib/mnesia/vsn.mk @@ -1 +1 @@ -MNESIA_VSN = 4.4.15 +MNESIA_VSN = 4.4.16 diff --git a/lib/odbc/src/odbc.appup.src b/lib/odbc/src/odbc.appup.src index e95e542ff5..f1a370d925 100644 --- a/lib/odbc/src/odbc.appup.src +++ b/lib/odbc/src/odbc.appup.src @@ -1 +1,8 @@ -{"%VSN%", [],[]} +%% -*- erlang -*- +{"%VSN%", + [ + {"2.10.8", [{restart_application, ssl}]} + ], + [ + {"2.10.8", [{restart_application, ssl}]} + ]}. diff --git a/lib/odbc/vsn.mk b/lib/odbc/vsn.mk index fac3f06d4b..aacf3924db 100644 --- a/lib/odbc/vsn.mk +++ b/lib/odbc/vsn.mk @@ -1 +1 @@ -ODBC_VSN = 2.10.8 +ODBC_VSN = 2.10.9 diff --git a/lib/public_key/src/pubkey_cert.erl b/lib/public_key/src/pubkey_cert.erl index c8953c6818..fadb993ed9 100644 --- a/lib/public_key/src/pubkey_cert.erl +++ b/lib/public_key/src/pubkey_cert.erl @@ -164,7 +164,7 @@ validate_signature(OtpCert, DerCert, Key, KeyParams, verify_fun(OtpCert, {bad_cert, invalid_signature}, UserState, VerifyFun) end. %%-------------------------------------------------------------------- --spec validate_names(#'OTPCertificate'{}, list(), list(), +-spec validate_names(#'OTPCertificate'{}, no_constraints | list(), list(), term(), term(), fun())-> term(). %% %% Description: Validate Subject Alternative Name. diff --git a/lib/public_key/src/public_key.appup.src b/lib/public_key/src/public_key.appup.src index 0f9f62d2f6..6b6b76d0a5 100644 --- a/lib/public_key/src/public_key.appup.src +++ b/lib/public_key/src/public_key.appup.src @@ -1,6 +1,12 @@ %% -*- erlang -*- {"%VSN%", [ + {"0.9", + [ + {update, public_key, soft, soft_purge, soft_purge, []}, + {update, pubkey_cert, soft, soft_purge, soft_purge, []} + ] + }, {"0.8", [ {update, 'OTP-PUB-KEY', soft, soft_purge, soft_purge, []}, @@ -12,6 +18,12 @@ } ], [ + {"0.9", + [ + {update, public_key, soft, soft_purge, soft_purge, []}, + {update, pubkey_cert, soft, soft_purge, soft_purge, []} + ] + }, {"0.8", [ {update, 'OTP-PUB-KEY', soft, soft_purge, soft_purge, []}, diff --git a/lib/public_key/src/public_key.erl b/lib/public_key/src/public_key.erl index 095a6ff0e0..30398df9cc 100644 --- a/lib/public_key/src/public_key.erl +++ b/lib/public_key/src/public_key.erl @@ -213,10 +213,13 @@ decrypt_private(CipherText, crypto:mpint(D)], Padding). %%-------------------------------------------------------------------- --spec decrypt_public(CipherText :: binary(), rsa_public_key()) -> +-spec decrypt_public(CipherText :: binary(), rsa_public_key() | rsa_private_key()) -> PlainText :: binary(). --spec decrypt_public(CipherText :: binary(), rsa_public_key(), +-spec decrypt_public(CipherText :: binary(), rsa_public_key() | rsa_private_key(), public_crypt_options()) -> PlainText :: binary(). +%% NOTE: The rsa_private_key() is not part of the documented API it is +%% here for testing purposes, in a real situation this is not a relevant +%% thing to do. %% %% Description: Public key decryption using the public key. %%-------------------------------------------------------------------- @@ -232,10 +235,14 @@ decrypt_public(CipherText,#'RSAPrivateKey'{modulus = N, publicExponent = E}, decrypt_public(CipherText, N,E, Options). %%-------------------------------------------------------------------- --spec encrypt_public(PlainText :: binary(), rsa_public_key()) -> +-spec encrypt_public(PlainText :: binary(), rsa_public_key() | rsa_private_key()) -> CipherText :: binary(). --spec encrypt_public(PlainText :: binary(), rsa_public_key(), +-spec encrypt_public(PlainText :: binary(), rsa_public_key() | rsa_private_key(), public_crypt_options()) -> CipherText :: binary(). + +%% NOTE: The rsa_private_key() is not part of the documented API it is +%% here for testing purposes, in a real situation this is not a relevant +%% thing to do. %% %% Description: Public key encryption using the public key. %%-------------------------------------------------------------------- @@ -280,8 +287,8 @@ encrypt_private(PlainText, #'RSAPrivateKey'{modulus = N, sign(PlainText, DigestType, #'RSAPrivateKey'{modulus = N, publicExponent = E, privateExponent = D}) when is_binary(PlainText), - DigestType == md5; - DigestType == sha -> + (DigestType == md5 orelse + DigestType == sha) -> crypto:rsa_sign(DigestType, sized_binary(PlainText), [crypto:mpint(E), crypto:mpint(N), @@ -571,11 +578,9 @@ validate(DerCert, #path_validation_state{working_issuer_name = Issuer, pubkey_cert:prepare_for_next_cert(OtpCert, ValidationState). -sized_binary(Binary) when is_binary(Binary) -> +sized_binary(Binary) -> Size = size(Binary), - <<?UINT32(Size), Binary/binary>>; -sized_binary(List) -> - sized_binary(list_to_binary(List)). + <<?UINT32(Size), Binary/binary>>. %%-------------------------------------------------------------------- %%% Deprecated functions diff --git a/lib/public_key/test/public_key_SUITE.erl b/lib/public_key/test/public_key_SUITE.erl index 81e01f3a02..88cfbcf2b6 100644 --- a/lib/public_key/test/public_key_SUITE.erl +++ b/lib/public_key/test/public_key_SUITE.erl @@ -249,10 +249,8 @@ sign_verify(Config) when is_list(Config) -> true = public_key:pkix_verify(Cert2, {Y, #'Dss-Parms'{p=P, q=Q, g=G}}), %% RSA sign - Msg0 = lists:duplicate(5, "Foo bar 100"), - Msg = list_to_binary(Msg0), + Msg = list_to_binary(lists:duplicate(5, "Foo bar 100")), - RSASign = public_key:sign(Msg0, sha, PrivateRSA), RSASign = public_key:sign(Msg, sha, PrivateRSA), true = public_key:verify(Msg, sha, RSASign, PublicRSA), false = public_key:verify(<<1:8, Msg/binary>>, sha, RSASign, PublicRSA), diff --git a/lib/public_key/vsn.mk b/lib/public_key/vsn.mk index 2810942171..334b9d792e 100644 --- a/lib/public_key/vsn.mk +++ b/lib/public_key/vsn.mk @@ -1 +1 @@ -PUBLIC_KEY_VSN = 0.9 +PUBLIC_KEY_VSN = 0.10 diff --git a/lib/ssl/doc/src/ssl.xml b/lib/ssl/doc/src/ssl.xml index 511f1e0bb2..ec272379bb 100644 --- a/lib/ssl/doc/src/ssl.xml +++ b/lib/ssl/doc/src/ssl.xml @@ -218,12 +218,12 @@ fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom()} | application is encountered. Additionally it will be called when a certificate is considered valid by the path validation to allow access to each certificate in the path to the user - application. Note that the it will differentiate between - the peer certificate and CA certificates by using valid_peer - or valid as the second argument to the verify fun. - See - <seealso marker="public_key:application">public_key(3)</seealso> - for definition of #'OTPCertificate'{} and #'Extension'{}.</p> + application. Note that the it will differentiate between the + peer certificate and CA certificates by using valid_peer or + valid as the second argument to the verify fun. See <seealso + marker="public_key:cert_records">the public_key User's + Guide</seealso> for definition of #'OTPCertificate'{} and + #'Extension'{}.</p> <p>If the verify callback fun returns {fail, Reason}, the verification process is immediately stopped and an alert is diff --git a/lib/ssl/src/ssl.appup.src b/lib/ssl/src/ssl.appup.src index f4e6b59b6d..51c5289bd2 100644 --- a/lib/ssl/src/ssl.appup.src +++ b/lib/ssl/src/ssl.appup.src @@ -1,9 +1,11 @@ %% -*- erlang -*- {"%VSN%", [ + {"4.1", [{restart_application, ssl}]}, {"4.0.1", [{restart_application, ssl}]} ], [ + {"4.1", [{restart_application, ssl}]}, {"4.0.1", [{restart_application, ssl}]} ]}. diff --git a/lib/ssl/src/ssl_certificate.erl b/lib/ssl/src/ssl_certificate.erl index 5571fb01f6..8c0c2bfa5d 100644 --- a/lib/ssl/src/ssl_certificate.erl +++ b/lib/ssl/src/ssl_certificate.erl @@ -28,7 +28,6 @@ -include("ssl_handshake.hrl"). -include("ssl_alert.hrl"). -include("ssl_internal.hrl"). --include("ssl_debug.hrl"). -include_lib("public_key/include/public_key.hrl"). -export([trusted_cert_and_path/2, diff --git a/lib/ssl/src/ssl_cipher.erl b/lib/ssl/src/ssl_cipher.erl index 1c8bbbaf06..72f02a4362 100644 --- a/lib/ssl/src/ssl_cipher.erl +++ b/lib/ssl/src/ssl_cipher.erl @@ -29,7 +29,6 @@ -include("ssl_record.hrl"). -include("ssl_cipher.hrl"). -include("ssl_alert.hrl"). --include("ssl_debug.hrl"). -include_lib("public_key/include/public_key.hrl"). -export([security_parameters/2, suite_definition/1, diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index ce90d22c09..6c9ac65b64 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -29,7 +29,6 @@ -behaviour(gen_fsm). --include("ssl_debug.hrl"). -include("ssl_handshake.hrl"). -include("ssl_alert.hrl"). -include("ssl_record.hrl"). diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index 7082a23fd0..c7a1c4965d 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -28,7 +28,6 @@ -include("ssl_cipher.hrl"). -include("ssl_alert.hrl"). -include("ssl_internal.hrl"). --include("ssl_debug.hrl"). -include_lib("public_key/include/public_key.hrl"). -export([master_secret/4, client_hello/5, server_hello/4, hello/4, diff --git a/lib/ssl/src/ssl_handshake.hrl b/lib/ssl/src/ssl_handshake.hrl index 74fba3786c..68a7802ef2 100644 --- a/lib/ssl/src/ssl_handshake.hrl +++ b/lib/ssl/src/ssl_handshake.hrl @@ -26,6 +26,13 @@ -ifndef(ssl_handshake). -define(ssl_handshake, true). +-include_lib("public_key/include/public_key.hrl"). + +-type algo_oid() :: ?'rsaEncryption' | ?'id-dsa'. +-type public_key() :: #'RSAPublicKey'{} | integer(). +-type public_key_params() :: #'Dss-Parms'{} | term(). +-type public_key_info() :: {algo_oid(), public_key(), public_key_params()}. + -record(session, { session_id, peer_certificate, diff --git a/lib/ssl/src/ssl_internal.hrl b/lib/ssl/src/ssl_internal.hrl index 4148032cb7..43a85c2d9d 100644 --- a/lib/ssl/src/ssl_internal.hrl +++ b/lib/ssl/src/ssl_internal.hrl @@ -25,6 +25,24 @@ -include_lib("public_key/include/public_key.hrl"). +-type reason() :: term(). +-type reply() :: term(). +-type msg() :: term(). +-type from() :: term(). +-type host() :: string() | tuple(). +-type port_num() :: integer(). +-type session_id() :: 0 | binary(). +-type tls_version() :: {integer(), integer()}. +-type tls_atom_version() :: sslv3 | tlsv1. +-type cache_ref() :: term(). +-type certdb_ref() :: term(). +-type key_algo() :: null | rsa | dhe_rsa | dhe_dss | dh_anon. +-type der_cert() :: binary(). +-type private_key() :: #'RSAPrivateKey'{} | #'DSAPrivateKey'{}. +-type issuer() :: tuple(). +-type serialnumber() :: integer(). +-type cert_key() :: {reference(), integer(), issuer()}. + %% basic binary constructors -define(BOOLEAN(X), X:8/unsigned-big-integer). -define(BYTE(X), X:8/unsigned-big-integer). @@ -93,28 +111,6 @@ active = true }). --type reason() :: term(). --type reply() :: term(). --type msg() :: term(). --type from() :: term(). --type host() :: string() | tuple(). --type port_num() :: integer(). --type session_id() :: 0 | binary(). --type tls_version() :: {integer(), integer()}. --type tls_atom_version() :: sslv3 | tlsv1. --type cache_ref() :: term(). --type certdb_ref() :: term(). --type key_algo() :: null | rsa | dhe_rsa | dhe_dss | dh_anon. --type oid() :: tuple(). --type public_key() :: #'RSAPublicKey'{} | integer(). --type public_key_params() :: #'Dss-Parms'{} | term(). --type public_key_info() :: {oid(), public_key(), public_key_params()}. --type der_cert() :: binary(). --type private_key() :: #'RSAPrivateKey'{} | #'DSAPrivateKey'{}. --type issuer() :: tuple(). --type serialnumber() :: integer(). --type cert_key() :: {reference(), integer(), issuer()}. - -endif. % -ifdef(ssl_internal). diff --git a/lib/ssl/src/ssl_record.erl b/lib/ssl/src/ssl_record.erl index e7a2d8ecf1..f1c0073965 100644 --- a/lib/ssl/src/ssl_record.erl +++ b/lib/ssl/src/ssl_record.erl @@ -30,7 +30,6 @@ -include("ssl_alert.hrl"). -include("ssl_handshake.hrl"). -include("ssl_cipher.hrl"). --include("ssl_debug.hrl"). %% Connection state handling -export([init_connection_states/1, diff --git a/lib/ssl/src/ssl_ssl3.erl b/lib/ssl/src/ssl_ssl3.erl index 18c3f4ed3d..c49f9f1e6d 100644 --- a/lib/ssl/src/ssl_ssl3.erl +++ b/lib/ssl/src/ssl_ssl3.erl @@ -25,7 +25,6 @@ -module(ssl_ssl3). -include("ssl_cipher.hrl"). --include("ssl_debug.hrl"). -include("ssl_internal.hrl"). -include("ssl_record.hrl"). % MD5 and SHA diff --git a/lib/ssl/src/ssl_tls1.erl b/lib/ssl/src/ssl_tls1.erl index 88394b23ba..3784483e9c 100644 --- a/lib/ssl/src/ssl_tls1.erl +++ b/lib/ssl/src/ssl_tls1.erl @@ -27,7 +27,6 @@ -include("ssl_cipher.hrl"). -include("ssl_internal.hrl"). -include("ssl_record.hrl"). --include("ssl_debug.hrl"). -export([master_secret/3, finished/3, certificate_verify/2, mac_hash/7, setup_keys/6, suites/0]). diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index ea84b3c9d1..8f9554f3ce 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -31,6 +31,7 @@ -define('24H_in_sec', 86400). -define(TIMEOUT, 60000). +-define(LONG_TIMEOUT, 600000). -define(EXPIRE, 10). -define(SLEEP, 500). @@ -45,7 +46,7 @@ %% variable, but should NOT alter/remove any existing entries. %%-------------------------------------------------------------------- init_per_suite(Config0) -> - Dog = ssl_test_lib:timetrap(?TIMEOUT *2), + Dog = ssl_test_lib:timetrap(?LONG_TIMEOUT *2), crypto:start(), application:start(public_key), ssl:start(), diff --git a/lib/ssl/test/ssl_session_cache_SUITE.erl b/lib/ssl/test/ssl_session_cache_SUITE.erl index 7a441e4599..0f39759d97 100644 --- a/lib/ssl/test/ssl_session_cache_SUITE.erl +++ b/lib/ssl/test/ssl_session_cache_SUITE.erl @@ -28,6 +28,7 @@ -define(SLEEP, 500). -define(TIMEOUT, 60000). +-define(LONG_TIMEOUT, 600000). -behaviour(ssl_session_cache_api). %% For the session cache tests @@ -45,7 +46,7 @@ %% variable, but should NOT alter/remove any existing entries. %%-------------------------------------------------------------------- init_per_suite(Config0) -> - Dog = ssl_test_lib:timetrap(?TIMEOUT *2), + Dog = ssl_test_lib:timetrap(?LONG_TIMEOUT *2), crypto:start(), application:start(public_key), ssl:start(), @@ -119,7 +120,7 @@ end_per_testcase(session_cache_process_list, Config) -> end_per_testcase(session_cache_process_mnesia, Config) -> application:unset_env(ssl, session_cb), application:unset_env(ssl, session_cb_init_args), - mnesia:stop(), + mnesia:kill(), ssl:stop(), ssl:start(), end_per_testcase(default_action, Config); diff --git a/lib/ssl/test/ssl_to_openssl_SUITE.erl b/lib/ssl/test/ssl_to_openssl_SUITE.erl index 55a0100b1e..afedeaf099 100644 --- a/lib/ssl/test/ssl_to_openssl_SUITE.erl +++ b/lib/ssl/test/ssl_to_openssl_SUITE.erl @@ -27,6 +27,7 @@ -include("test_server.hrl"). -define(TIMEOUT, 120000). +-define(LONG_TIMEOUT, 600000). -define(SLEEP, 1000). -define(OPENSSL_RENEGOTIATE, "r\n"). -define(OPENSSL_QUIT, "Q\n"). @@ -44,7 +45,7 @@ %% variable, but should NOT alter/remove any existing entries. %%-------------------------------------------------------------------- init_per_suite(Config0) -> - Dog = ssl_test_lib:timetrap(?TIMEOUT *2), + Dog = ssl_test_lib:timetrap(?LONG_TIMEOUT *2), case os:find_executable("openssl") of false -> {skip, "Openssl not found"}; diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk index 30a0a3b3f7..ee692adb3b 100644 --- a/lib/ssl/vsn.mk +++ b/lib/ssl/vsn.mk @@ -1,2 +1,2 @@ -SSL_VSN = 4.1 +SSL_VSN = 4.1.1 diff --git a/lib/tools/src/eprof.erl b/lib/tools/src/eprof.erl index 8e5c0ec46b..87fdc1fa34 100644 --- a/lib/tools/src/eprof.erl +++ b/lib/tools/src/eprof.erl @@ -136,7 +136,7 @@ handle_call({analyze, procs, Opts}, _, #state{ bpd = #bpd{ p = Ps, us = Tus} = B lists:foreach(fun ({Pid, Mfas}) -> {Pn, Pus} = sum_bp_total_n_us(Mfas), - format(Fd, "~n****** Process ~w -- ~s % of profiled time *** ~n", [Pid, s("~.2f", [100.0*(Pus/Tus)])]), + format(Fd, "~n****** Process ~w -- ~s % of profiled time *** ~n", [Pid, s("~.2f", [100.0*divide(Pus,Tus)])]), print_bp_mfa(Mfas, {Pn,Pus}, Fd, Opts), ok end, gb_trees:to_list(Ps)), @@ -443,8 +443,8 @@ string_bp_mfa([{Mfa, {Count, Time}}|Mfas], Tus, {MfaW, CountW, PercW, TimeW, TpC Smfa = s(Mfa), Scount = s(Count), Stime = s(Time), - Sperc = s("~.2f", [100*(Time/Tus)]), - Stpc = s("~.2f", [Time/Count]), + Sperc = s("~.2f", [100*divide(Time,Tus)]), + Stpc = s("~.2f", [divide(Time,Count)]), string_bp_mfa(Mfas, Tus, { erlang:max(MfaW, length(Smfa)), @@ -484,3 +484,6 @@ format(Fd, Format, Strings) -> io:format(Fd, Format, Strings), io:format(Format, Strings), ok. + +divide(_,0) -> 0.0; +divide(T,N) -> T/N. diff --git a/lib/xmerl/src/xmerl_sax_parser_base.erlsrc b/lib/xmerl/src/xmerl_sax_parser_base.erlsrc index 9d184152d1..3b9eaa309c 100644 --- a/lib/xmerl/src/xmerl_sax_parser_base.erlsrc +++ b/lib/xmerl/src/xmerl_sax_parser_base.erlsrc @@ -1,7 +1,7 @@ %%-*-erlang-*- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2009. All Rights Reserved. +%% Copyright Ericsson AB 2008-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -934,6 +934,13 @@ parse_att_value(?STRING_REST("&", Rest), State, Stop, Acc) -> parse_att_value(Rest1, State2, Stop, ParsedValue ++ Acc); {external_general, Name, _} -> ?fatal_error(State1, "External parsed entity reference in attribute value: " ++ Name); + {not_found, Name} -> + case State#xmerl_sax_parser_state.skip_external_dtd of + false -> + ?fatal_error(State1, "Entity not declared: " ++ Name); %%VC: Entity Declared + true -> + parse_att_value(Rest1, State1, Stop, ";" ++ lists:reverse(Name) ++ "&" ++ Acc) + end; {unparsed, Name, _} -> ?fatal_error(State1, "Unparsed entity reference in attribute value: " ++ Name) end; @@ -1098,6 +1105,13 @@ parse_content(?STRING_REST("&", Rest), State, Acc, _IgnorableWS) -> {external_general, _, {PubId, SysId}} -> State2 = parse_external_entity(State1, PubId, SysId), parse_content(Rest1, State2, Acc, false); + {not_found, Name} -> + case State#xmerl_sax_parser_state.skip_external_dtd of + false -> + ?fatal_error(State1, "Entity not declared: " ++ Name); %%VC: Entity Declared + true -> + parse_content(Rest1, State1, ";" ++ lists:reverse(Name) ++ "&" ++ Acc, false) + end; {unparsed, Name, _} -> ?fatal_error(State1, "Unparsed entity reference in content: " ++ Name) end; @@ -1357,7 +1371,7 @@ look_up_reference(Name, HaveToExist, State) -> yes -> ?fatal_error(State, "Entity not declared: " ++ Name); %%WFC: Entity Declared no -> - ?fatal_error(State, "Entity not declared: " ++ Name) %%VC: Entity Declared + {not_found, Name} %%VC: Entity Declared end; false -> {not_found, Name} @@ -1869,7 +1883,14 @@ parse_doctype_decl(?STRING_REST("%", Rest), State) -> parse_doctype_decl(?APPEND_STRING(IValue, Rest1), State1); {external_parameter, _, {PubId, SysId}} -> State2 = parse_external_entity(State1#xmerl_sax_parser_state{file_type = entity}, PubId, SysId), - parse_doctype_decl(Rest1, State2) + parse_doctype_decl(Rest1, State2); + {not_found, Name} -> + case State#xmerl_sax_parser_state.skip_external_dtd of + false -> + ?fatal_error(State1, "Entity not declared: " ++ Name); %%WFC: Entity Declared + true -> + parse_doctype_decl(Rest1, State1) + end end; parse_doctype_decl(?STRING_REST("<!", Rest1), State) -> parse_doctype_decl_1(Rest1, State); @@ -2443,7 +2464,7 @@ parse_ndata(Bytes, State) -> %% Acc = string() %% Result : {Value, Rest, State} %% Value = string() -%% Description: Parse an attribute value +%% Description: Parse an entity value %%---------------------------------------------------------------------- parse_entity_value(?STRING_EMPTY, State, undefined, Acc) -> {Acc, [], State}; %% stop clause when parsing references @@ -2473,7 +2494,7 @@ parse_entity_value(?STRING_REST("&", Rest), State, Stop, Acc) -> {external_general, Name, _} -> parse_entity_value(Rest1, State1, Stop, ";" ++ lists:reverse(Name) ++ "&" ++ Acc); {not_found, Name} -> - parse_entity_value(Rest1, State1, Stop, ";" ++ lists:reverse(Name) ++ "&" ++ Acc); + parse_entity_value(Rest1, State1, Stop, ";" ++ lists:reverse(Name) ++ "&" ++ Acc); {unparsed, Name, _} -> ?fatal_error(State1, "Unparsed entity reference in entity value: " ++ Name) end; @@ -2490,7 +2511,15 @@ parse_entity_value(?STRING_REST("%", Rest), #xmerl_sax_parser_state{file_type=Ty IValue = ?TO_INPUT_FORMAT(" " ++ RefValue ++ " "), parse_entity_value(?APPEND_STRING(IValue, Rest1), State1, Stop, Acc); {external_parameter, _, {_PubId, _SysId}} -> - ?fatal_error(State1, "Parameter references in entity value not supported yet.") + ?fatal_error(State1, "Parameter references in entity value not supported yet."); + {not_found, Name} -> + case State#xmerl_sax_parser_state.skip_external_dtd of + false -> + ?fatal_error(State1, "Entity not declared: " ++ Name); %%VC: Entity Declared + true -> + parse_entity_value(Rest1, State1, Stop, ";" ++ lists:reverse(Name) ++ "&" ++ Acc) + end + end end; parse_entity_value(?STRING_UNBOUND_REST(Stop, Rest), State, Stop, Acc) -> diff --git a/lib/xmerl/src/xmerl_scan.erl b/lib/xmerl/src/xmerl_scan.erl index e2e6f95c4a..e07d495fc7 100644 --- a/lib/xmerl/src/xmerl_scan.erl +++ b/lib/xmerl/src/xmerl_scan.erl @@ -34,7 +34,9 @@ %% See also <a href="xmerl_examples.html">tutorial</a> on customization %% functions. %% </p> +%% <p> %% Possible options are: +%% </p> %% <dl> %% <dt><code>{acc_fun, Fun}</code></dt> %% <dd>Call back function to accumulate contents of entity.</dd> diff --git a/lib/xmerl/src/xmerl_xpath.erl b/lib/xmerl/src/xmerl_xpath.erl index 182a186d2c..e654a8ef1d 100644 --- a/lib/xmerl/src/xmerl_xpath.erl +++ b/lib/xmerl/src/xmerl_xpath.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. +%% Copyright Ericsson AB 2003-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -57,7 +57,9 @@ %% @type option_list(). <p>Options allows to customize the behaviour of the %% XPath scanner. %% </p> +%% <p> %% Possible options are: +%% </p> %% <dl> %% <dt><code>{namespace, #xmlNamespace}</code></dt> %% <dd>Set namespace nodes, from XmlNamspace, in xmlContext</dd> diff --git a/lib/xmerl/src/xmerl_xsd.erl b/lib/xmerl/src/xmerl_xsd.erl index 1aedc9e270..c0923520c2 100644 --- a/lib/xmerl/src/xmerl_xsd.erl +++ b/lib/xmerl/src/xmerl_xsd.erl @@ -29,7 +29,9 @@ %% @type option_list(). <p>Options allow to customize the behaviour of the %% validation. %% </p> +%% <p> %% Possible options are : +%% </p> %% <dl> %% <dt><code>{tab2file,boolean()}</code></dt> %% <dd>Enables saving of abstract structure on file for debugging @@ -46,6 +48,7 @@ %% <dd>It is possible by this option to provide a state with process %% information from an earlier validation.</dd> %% </dl> +%% @end %%%------------------------------------------------------------------- -module(xmerl_xsd). |