diff options
338 files changed, 18344 insertions, 4372 deletions
diff --git a/INSTALL-WIN32.md b/INSTALL-WIN32.md index 6481ca1b06..59b9086c39 100644 --- a/INSTALL-WIN32.md +++ b/INSTALL-WIN32.md @@ -92,7 +92,9 @@ Frequently Asked Questions A: The SMP version of Erlang needs features in the Visual Studio 2005. Can't live without them. Besides the new compiler gives the Erlang - emulator a ~40% performance boost(!) + emulator a ~40% performance boost(!). Alternatively you can build Erlang + successfully using the free (proprietary) Visual Studio 2008 Express + edition C++ compiler. * Q: Can/will I build a Cygwin binary with the procedure you describe? @@ -208,20 +210,15 @@ Frequently Asked Questions * (Buy and) Install Microsoft Visual studio 2005 and SP1 (or higher) - * Get and install Sun's JDK 1.4.2 + * Alternatively install the free MS Visual Studio 2008 Express [msvc++] + and the Windows SDK [32bit-SDK] or [64bit-SDK] depending on the Windows + platform you are running. - * Get and install NSIS 2.01 or higher (up to 2.30 tried and working) + * Get and install Sun's JDK 1.4.2 - * Get and install OpenSSL 0.9.7c or higher + * Get and install NSIS 2.01 or higher (up to 2.46 tried and working) - * Get and unpack wxWidgets-2.8.9 or higher to `/opt/local/pgm` inside - cygwin. - * Open `/cygwin/opt/local/pgm/wxWidgets-2.8.9/build/msw/wx.dsw` - * Enable `wxUSE_GLCANVAS`, `wxUSE_POSTSCRIPT` and - `wxUSE_GRAPHICS_CONTEXT` in `include/wx/msw/setup.h` - * Build all unicode release (and unicode debug) packages - * Open `/cygwin/opt/local/pgm/wxWidgets-2.8.9/contrib/build/stc/stc.dsw` - * Build the unicode release (and unicode debug) packages + * Get and install OpenSSL 0.9.7c or higher (up to 1.0.0a tried & working) * Get the Erlang source distribution (from <http://www.erlang.org/download.html>) and unpack with Cygwin's `tar`. @@ -363,6 +360,15 @@ Well' here's the list: your `PATH` to allow the environment to find mc.exe. The next Visual Studio (2010) is expected to include this tool. + Alternatively install the free MS Visual Studio 2008 Express [msvc++] and + the Windows SDK [32bit-SDK] or [64bit-SDK] depending on the Windows + platform you are running, which includes the missing mc.exe message + compiler. + +[msvc++]: http://download.microsoft.com/download/E/8/E/E8EEB394-7F42-4963-A2D8-29559B738298/VS2008ExpressWithSP1ENUX1504728.iso +[32bit-SDK]: http://download.microsoft.com/download/2/E/9/2E911956-F90F-4BFB-8231-E292A7B6F287/GRMSDK_EN_DVD.iso +[64bit-SDK]: http://download.microsoft.com/download/2/E/9/2E911956-F90F-4BFB-8231-E292A7B6F287/GRMSDKX_EN_DVD.iso + * Sun's Java JDK 1.5.0 or higher. Our Java code (jinterface, ic) is written for JDK 1.5.0. Get it for Windows and install it, the JRE is not enough. If you don't care about Java, you can skip this step, the @@ -401,9 +407,10 @@ Well' here's the list: on the `Related` link and then on the `Binaries` link (upper right corner of the page last time I looked), you can then reach the "Shining Lights Productions" Web site for Windows binaries - distributions. Get the latest or 0.9.7c if you get trouble with the - latest. It's a nifty installer. The rest should be handled by - `configure`, you needn't put anything in the path or anything. + distributions. Get the latest 32-bit installer, or use 0.9.7c if you get + trouble with the latest, and install to C:\OpenSSL which is where the + Makefiles are expecting to find it. It's a nifty installer. The rest should + be handled by `configure`, you needn't put anything in the path or anything. If you want to build openssl for windows yourself (which might be possible, as you wouldn't be reading this if you weren't a @@ -422,18 +429,51 @@ Well' here's the list: release (2.9.\* is a developer release which currently does not work with wxErlang). - Install or unpack it to `DRIVE:/PATH/cygwin/opt/local/pgm` + Install or unpack it to `DRIVE:/PATH/cygwin/opt/local/pgm`. Open from explorer (i.e. by double clicking the file) - `C:\cygwin\opt\local\pgm\wxMSW-2.8.10\build\msw\wx.dsw` + `C:\cygwin\opt\local\pgm\wxMSW-2.8.11\build\msw\wx.dsw` In Microsoft Visual Studio, click File/Open/File, locate and - open: `C:\cygwin\opt\local\pgm\wxMSW-2.8.10\include\wx\msw\setup.h` + open: `C:\cygwin\opt\local\pgm\wxMSW-2.8.11\include\wx\msw\setup.h` enable `wxUSE_GLCANVAS`, `wxUSE_POSTSCRIPT` and `wxUSE_GRAPHICS_CONTEXT` Build it by clicking Build/Batch Build and select all unicode release (and unicode debug) packages. - Open `C:\cygwin\opt\local\pgm\wxMSW-2.8.10\contrib/build/stc/stc.dsw` + Open `C:\cygwin\opt\local\pgm\wxMSW-2.8.11\contrib/build/stc/stc.dsw` and batch build all unicode packages. + If you are using Visual C++ 9.0 or higher (Visual Studio 2008 onwards) you + will also need to convert and re-create the project dependencies in the new + .sln "Solution" format. + + * Open VSC++ & the project `wxMSW-2.8.11\build\msw\wx.dsw`, accepting the + automatic conversion to the newer VC++ format and save as + `\wxMSW-2.8.11\build\msw\wx.sln` + + * right-click on the project, and set up the project dependencies for + `wx.dsw` to achieve the below build order + + jpeg, png, tiff, zlib, regex, expat, base, net, odbc, core, + gl, html, media, qa, adv, dbgrid, xrc, aui, richtext, xml + + Build all unicode release (and unicode debug) packages either from the + GUI or alternatively launch a new prompt from somewhere like Start -> + Programs -> Microsoft Visual C++ -> Visual Studio Tools -> VS2008 Cmd Prompt + and cd to where you unpacked wxMSW + + pushd c:\wxMSW*\build\msw + vcbuild /useenv /platform:Win32 /M4 wx.sln "Unicode Release|Win32" + vcbuild /useenv /platform:Win32 /M4 wx.sln "Unicode Debug|Win32" + + Open VSC++ & convert `C:\wxMSW-2.8.11\contrib\build\stc\stc.dsw` to + `C:\wxMSW-2.8.11\contrib\build\stc\stc.sln` + + * build the unicode release (and unicode debug) packages from the GUI or + alternatively open a VS2008 Cmd Prompt and cd to where you unpacked wxMSW + + pushd c:\wxMSW*\contrib\build\stc + vcbuild /useenv /platform:Win32 /M4 stc.sln "Unicode Release|Win32" + vcbuild /useenv /platform:Win32 /M4 stc.sln "Unicode Debug|Win32" + * The Erlang source distribution (from <http://www.erlang.org/download.html>). The same as for Unix platforms. Preferably use tar from within Cygwin to unpack the source tar.gz (`tar zxf otp_src_%OTP-REL%.tar.gz`). diff --git a/Makefile.in b/Makefile.in index 4b6e2e1190..ca92bf604d 100644 --- a/Makefile.in +++ b/Makefile.in @@ -393,7 +393,7 @@ endif # --------------------------------------------------------------- # Target only used when building commercial ERTS patches # --------------------------------------------------------------- -release_docs docs: +release_docs docs: mod2app ifeq ($(OTP_SMALL_BUILD),true) cd $(ERL_TOP)/lib && \ ERL_TOP=$(ERL_TOP) $(MAKE) TESTROOT=$(RELEASE_ROOT) $@ @@ -408,6 +408,9 @@ endif cd $(ERL_TOP)/system/doc && \ ERL_TOP=$(ERL_TOP) $(MAKE) TESTROOT=$(RELEASE_ROOT) $@ +mod2app: + $(ERL_TOP)/lib/erl_docgen/priv/bin/xref_mod_app.escript -topdir $(ERL_TOP) -outfile $(ERL_TOP)/make/$(TARGET)/mod2app.xml + # ---------------------------------------------------------------------- ERLANG_EARS=$(BOOTSTRAP_ROOT)/bootstrap/erts ELINK=$(BOOTSTRAP_ROOT)/bootstrap/erts/bin/elink diff --git a/bootstrap/bin/start.script b/bootstrap/bin/start.script index 3f472b5e8c..c5106e3709 100644 --- a/bootstrap/bin/start.script +++ b/bootstrap/bin/start.script @@ -1,4 +1,4 @@ -%% script generated at {2010,10,18} {14,19,26} +%% script generated at {2010,12,3} {17,41,47} {script, {"OTP APN 181 01","R14B01"}, [{preLoaded, diff --git a/bootstrap/bin/start_clean.script b/bootstrap/bin/start_clean.script index 3f472b5e8c..c5106e3709 100644 --- a/bootstrap/bin/start_clean.script +++ b/bootstrap/bin/start_clean.script @@ -1,4 +1,4 @@ -%% script generated at {2010,10,18} {14,19,26} +%% script generated at {2010,12,3} {17,41,47} {script, {"OTP APN 181 01","R14B01"}, [{preLoaded, diff --git a/bootstrap/lib/compiler/ebin/beam_utils.beam b/bootstrap/lib/compiler/ebin/beam_utils.beam Binary files differindex 7b9c08439e..c335748e8a 100644 --- a/bootstrap/lib/compiler/ebin/beam_utils.beam +++ b/bootstrap/lib/compiler/ebin/beam_utils.beam diff --git a/bootstrap/lib/compiler/egen/core_parse.erl b/bootstrap/lib/compiler/egen/core_parse.erl index 0e94fb414d..6e3fead4a3 100644 --- a/bootstrap/lib/compiler/egen/core_parse.erl +++ b/bootstrap/lib/compiler/egen/core_parse.erl @@ -13,7 +13,7 @@ tok_val(T) -> element(3, T). tok_line(T) -> element(2, T). --file("/usr/local/otp/releases/sles10_32_R14B_patched/lib/parsetools-2.0.4/include/yeccpre.hrl", 0). +-file("/usr/local/otp/releases/sles10_64_R14B_patched/lib/parsetools-2.0.4/include/yeccpre.hrl", 0). %% %% %CopyrightBegin% %% @@ -196,7 +196,7 @@ yecctoken2string(Other) -> --file("/ldisk/bjorn/otp/bootstrap/lib/compiler/egen/core_parse.erl", 199). +-file("/ldisk/pan/git/otp/bootstrap/lib/compiler/egen/core_parse.erl", 199). yeccpars2(0=S, Cat, Ss, Stack, T, Ts, Tzr) -> yeccpars2_0(S, Cat, Ss, Stack, T, Ts, Tzr); diff --git a/bootstrap/lib/kernel/ebin/code.beam b/bootstrap/lib/kernel/ebin/code.beam Binary files differindex dc4cf6cf41..18947d98c3 100644 --- a/bootstrap/lib/kernel/ebin/code.beam +++ b/bootstrap/lib/kernel/ebin/code.beam diff --git a/bootstrap/lib/kernel/ebin/file.beam b/bootstrap/lib/kernel/ebin/file.beam Binary files differindex 3671dea4f1..343cee3fe7 100644 --- a/bootstrap/lib/kernel/ebin/file.beam +++ b/bootstrap/lib/kernel/ebin/file.beam diff --git a/bootstrap/lib/kernel/ebin/file_io_server.beam b/bootstrap/lib/kernel/ebin/file_io_server.beam Binary files differindex ce14722f4c..f7c170fd28 100644 --- a/bootstrap/lib/kernel/ebin/file_io_server.beam +++ b/bootstrap/lib/kernel/ebin/file_io_server.beam diff --git a/bootstrap/lib/kernel/ebin/inet.beam b/bootstrap/lib/kernel/ebin/inet.beam Binary files differindex 63088b2e1b..622110bcdb 100644 --- a/bootstrap/lib/kernel/ebin/inet.beam +++ b/bootstrap/lib/kernel/ebin/inet.beam diff --git a/bootstrap/lib/kernel/ebin/kernel.appup b/bootstrap/lib/kernel/ebin/kernel.appup index 013c65b3e2..77f9f42fea 100644 --- a/bootstrap/lib/kernel/ebin/kernel.appup +++ b/bootstrap/lib/kernel/ebin/kernel.appup @@ -1 +1 @@ -{"2.14",[],[]}. +{"2.14.2",[],[]}. diff --git a/bootstrap/lib/kernel/ebin/kernel.beam b/bootstrap/lib/kernel/ebin/kernel.beam Binary files differindex 471ab154e8..e1db5986dc 100644 --- a/bootstrap/lib/kernel/ebin/kernel.beam +++ b/bootstrap/lib/kernel/ebin/kernel.beam diff --git a/bootstrap/lib/stdlib/ebin/c.beam b/bootstrap/lib/stdlib/ebin/c.beam Binary files differindex 2202b9105d..af54466541 100644 --- a/bootstrap/lib/stdlib/ebin/c.beam +++ b/bootstrap/lib/stdlib/ebin/c.beam diff --git a/bootstrap/lib/stdlib/ebin/dets.beam b/bootstrap/lib/stdlib/ebin/dets.beam Binary files differindex 3e133f4ff2..4b6e3756e9 100644 --- a/bootstrap/lib/stdlib/ebin/dets.beam +++ b/bootstrap/lib/stdlib/ebin/dets.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_v8.beam b/bootstrap/lib/stdlib/ebin/dets_v8.beam Binary files differindex 2ad134d371..968b9bcb28 100644 --- a/bootstrap/lib/stdlib/ebin/dets_v8.beam +++ b/bootstrap/lib/stdlib/ebin/dets_v8.beam diff --git a/bootstrap/lib/stdlib/ebin/dets_v9.beam b/bootstrap/lib/stdlib/ebin/dets_v9.beam Binary files differindex 2eabd08ed4..355c5819cf 100644 --- a/bootstrap/lib/stdlib/ebin/dets_v9.beam +++ b/bootstrap/lib/stdlib/ebin/dets_v9.beam diff --git a/bootstrap/lib/stdlib/ebin/epp.beam b/bootstrap/lib/stdlib/ebin/epp.beam Binary files differindex 8115d9c474..0483b561d4 100644 --- a/bootstrap/lib/stdlib/ebin/epp.beam +++ b/bootstrap/lib/stdlib/ebin/epp.beam diff --git a/bootstrap/lib/stdlib/ebin/erl_lint.beam b/bootstrap/lib/stdlib/ebin/erl_lint.beam Binary files differindex 5ebd228586..9b54fff03b 100644 --- a/bootstrap/lib/stdlib/ebin/erl_lint.beam +++ b/bootstrap/lib/stdlib/ebin/erl_lint.beam diff --git a/bootstrap/lib/stdlib/ebin/ets.beam b/bootstrap/lib/stdlib/ebin/ets.beam Binary files differindex 6b25af9b1b..9ade9e2edd 100644 --- a/bootstrap/lib/stdlib/ebin/ets.beam +++ b/bootstrap/lib/stdlib/ebin/ets.beam diff --git a/bootstrap/lib/stdlib/ebin/filelib.beam b/bootstrap/lib/stdlib/ebin/filelib.beam Binary files differindex 8a59a62379..7c1ba41e59 100644 --- a/bootstrap/lib/stdlib/ebin/filelib.beam +++ b/bootstrap/lib/stdlib/ebin/filelib.beam diff --git a/bootstrap/lib/stdlib/ebin/filename.beam b/bootstrap/lib/stdlib/ebin/filename.beam Binary files differindex 4cdb6064d2..69af1ef3c2 100644 --- a/bootstrap/lib/stdlib/ebin/filename.beam +++ b/bootstrap/lib/stdlib/ebin/filename.beam diff --git a/bootstrap/lib/stdlib/ebin/ordsets.beam b/bootstrap/lib/stdlib/ebin/ordsets.beam Binary files differindex c3f2c3b7b1..48effb764e 100644 --- a/bootstrap/lib/stdlib/ebin/ordsets.beam +++ b/bootstrap/lib/stdlib/ebin/ordsets.beam diff --git a/bootstrap/lib/stdlib/ebin/stdlib.appup b/bootstrap/lib/stdlib/ebin/stdlib.appup index 097f2b47ed..ca89dcf43a 100644 --- a/bootstrap/lib/stdlib/ebin/stdlib.appup +++ b/bootstrap/lib/stdlib/ebin/stdlib.appup @@ -1 +1 @@ -{"1.17",[],[]}. +{"1.17.2",[],[]}. diff --git a/bootstrap/lib/stdlib/ebin/string.beam b/bootstrap/lib/stdlib/ebin/string.beam Binary files differindex 3c3eaf6ed2..0de0b44cb8 100644 --- a/bootstrap/lib/stdlib/ebin/string.beam +++ b/bootstrap/lib/stdlib/ebin/string.beam diff --git a/bootstrap/lib/stdlib/ebin/supervisor.beam b/bootstrap/lib/stdlib/ebin/supervisor.beam Binary files differindex a9e30b4d5d..13cb6032a9 100644 --- a/bootstrap/lib/stdlib/ebin/supervisor.beam +++ b/bootstrap/lib/stdlib/ebin/supervisor.beam diff --git a/bootstrap/lib/stdlib/ebin/unicode.beam b/bootstrap/lib/stdlib/ebin/unicode.beam Binary files differindex 4ca769b9a2..cc3a3c1859 100644 --- a/bootstrap/lib/stdlib/ebin/unicode.beam +++ b/bootstrap/lib/stdlib/ebin/unicode.beam diff --git a/bootstrap/lib/stdlib/egen/erl_parse.erl b/bootstrap/lib/stdlib/egen/erl_parse.erl index cd5102a680..f15deb37f1 100644 --- a/bootstrap/lib/stdlib/egen/erl_parse.erl +++ b/bootstrap/lib/stdlib/egen/erl_parse.erl @@ -556,7 +556,7 @@ get_attribute(L, Name) -> get_attributes(L) -> erl_scan:attributes_info(L). --file("/usr/local/otp/releases/sles10_32_R14B_patched/lib/parsetools-2.0.4/include/yeccpre.hrl", 0). +-file("/usr/local/otp/releases/sles10_64_R14B_patched/lib/parsetools-2.0.4/include/yeccpre.hrl", 0). %% %% %CopyrightBegin% %% @@ -739,7 +739,7 @@ yecctoken2string(Other) -> --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 742). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 742). yeccpars2(0=S, Cat, Ss, Stack, T, Ts, Tzr) -> yeccpars2_0(S, Cat, Ss, Stack, T, Ts, Tzr); @@ -8195,7 +8195,7 @@ yeccpars2_39_(__Stack0) -> [ __1 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8198). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8198). -compile({inline,yeccpars2_46_/1}). -file("erl_parse.yrl", 434). yeccpars2_46_(__Stack0) -> @@ -8204,7 +8204,7 @@ yeccpars2_46_(__Stack0) -> { [ ] , ? line ( __1 ) } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8207). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8207). -compile({inline,yeccpars2_70_/1}). -file("erl_parse.yrl", 325). yeccpars2_70_(__Stack0) -> @@ -8213,7 +8213,7 @@ yeccpars2_70_(__Stack0) -> { tuple , ? line ( __1 ) , [ ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8216). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8216). -compile({inline,yeccpars2_71_/1}). -file("erl_parse.yrl", 326). yeccpars2_71_(__Stack0) -> @@ -8222,7 +8222,7 @@ yeccpars2_71_(__Stack0) -> { tuple , ? line ( __1 ) , __2 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8225). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8225). -compile({inline,yeccpars2_73_/1}). -file("erl_parse.yrl", 408). yeccpars2_73_(__Stack0) -> @@ -8254,7 +8254,7 @@ yeccpars2_81_(__Stack0) -> [ __1 | __3 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8257). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8257). -compile({inline,yeccpars2_82_/1}). -file("erl_parse.yrl", 406). yeccpars2_82_(__Stack0) -> @@ -8287,7 +8287,7 @@ yeccpars2_88_(__Stack0) -> [ __1 | __3 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8290). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8290). -compile({inline,yeccpars2_89_/1}). -file("erl_parse.yrl", 381). yeccpars2_89_(__Stack0) -> @@ -8326,7 +8326,7 @@ yeccpars2_98_(__Stack0) -> [ ] end | __Stack0]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8329). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8329). -compile({inline,yeccpars2_100_/1}). -file("erl_parse.yrl", 427). yeccpars2_100_(__Stack0) -> @@ -8343,7 +8343,7 @@ yeccpars2_102_(__Stack0) -> [ ] end | __Stack0]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8346). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8346). -compile({inline,yeccpars2_104_/1}). -file("erl_parse.yrl", 424). yeccpars2_104_(__Stack0) -> @@ -8353,7 +8353,7 @@ yeccpars2_104_(__Stack0) -> { clause , L , [ { tuple , L , [ __1 , __3 , { var , L , '_' } ] } ] , __4 , __5 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8356). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8356). -compile({inline,yeccpars2_106_/1}). -file("erl_parse.yrl", 421). yeccpars2_106_(__Stack0) -> @@ -8395,7 +8395,7 @@ yeccpars2_114_(__Stack0) -> { [ ] , __2 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8398). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8398). -compile({inline,yeccpars2_115_/1}). -file("erl_parse.yrl", 452). yeccpars2_115_(__Stack0) -> @@ -8404,7 +8404,7 @@ yeccpars2_115_(__Stack0) -> { string , ? line ( __1 ) , element ( 3 , __1 ) ++ element ( 3 , __2 ) } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8407). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8407). -compile({inline,yeccpars2_120_/1}). -file("erl_parse.yrl", 386). yeccpars2_120_(__Stack0) -> @@ -8413,7 +8413,7 @@ yeccpars2_120_(__Stack0) -> { 'receive' , ? line ( __1 ) , [ ] , __3 , __4 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8416). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8416). -compile({inline,yeccpars2_122_/1}). -file("erl_parse.yrl", 384). yeccpars2_122_(__Stack0) -> @@ -8422,7 +8422,7 @@ yeccpars2_122_(__Stack0) -> { 'receive' , ? line ( __1 ) , __2 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8425). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8425). -compile({inline,yeccpars2_125_/1}). -file("erl_parse.yrl", 388). yeccpars2_125_(__Stack0) -> @@ -8439,7 +8439,7 @@ yeccpars2_131_(__Stack0) -> [ __1 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8442). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8442). -compile({inline,yeccpars2_135_/1}). -file("erl_parse.yrl", 323). yeccpars2_135_(__Stack0) -> @@ -8448,7 +8448,7 @@ yeccpars2_135_(__Stack0) -> { b_generate , ? line ( __2 ) , __1 , __3 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8451). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8451). -compile({inline,yeccpars2_137_/1}). -file("erl_parse.yrl", 322). yeccpars2_137_(__Stack0) -> @@ -8465,7 +8465,7 @@ yeccpars2_139_(__Stack0) -> [ __1 | __3 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8468). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8468). -compile({inline,yeccpars2_140_/1}). -file("erl_parse.yrl", 315). yeccpars2_140_(__Stack0) -> @@ -8474,7 +8474,7 @@ yeccpars2_140_(__Stack0) -> { lc , ? line ( __1 ) , __2 , __4 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8477). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8477). -compile({inline,yeccpars2_141_/1}). -file("erl_parse.yrl", 431). yeccpars2_141_(__Stack0) -> @@ -8491,7 +8491,7 @@ yeccpars2_143_(__Stack0) -> [ __1 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8494). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8494). -compile({inline,yeccpars2_145_/1}). -file("erl_parse.yrl", 371). yeccpars2_145_(__Stack0) -> @@ -8508,7 +8508,7 @@ yeccpars2_147_(__Stack0) -> [ __1 | __3 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8511). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8511). -compile({inline,yeccpars2_148_/1}). -file("erl_parse.yrl", 365). yeccpars2_148_(__Stack0) -> @@ -8532,7 +8532,7 @@ yeccpars2_151_(__Stack0) -> [ ] end | __Stack0]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8535). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8535). -compile({inline,yeccpars2_157_/1}). -file("erl_parse.yrl", 394). yeccpars2_157_(__Stack0) -> @@ -8541,7 +8541,7 @@ yeccpars2_157_(__Stack0) -> { 'fun' , ? line ( __1 ) , { function , element ( 3 , __2 ) , element ( 3 , __4 ) , element ( 3 , __6 ) } } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8544). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8544). -compile({inline,yeccpars2_158_/1}). -file("erl_parse.yrl", 392). yeccpars2_158_(__Stack0) -> @@ -8567,7 +8567,7 @@ yeccpars2_162_(__Stack0) -> [ __1 | __3 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8570). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8570). -compile({inline,yeccpars2_163_/1}). -file("erl_parse.yrl", 396). yeccpars2_163_(__Stack0) -> @@ -8576,7 +8576,7 @@ yeccpars2_163_(__Stack0) -> build_fun ( ? line ( __1 ) , __2 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8579). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8579). -compile({inline,yeccpars2_164_/1}). -file("erl_parse.yrl", 214). yeccpars2_164_(__Stack0) -> @@ -8585,7 +8585,7 @@ yeccpars2_164_(__Stack0) -> { 'catch' , ? line ( __1 ) , __2 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8588). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8588). -compile({inline,yeccpars2_168_/1}). -file("erl_parse.yrl", 375). yeccpars2_168_(__Stack0) -> @@ -8594,7 +8594,7 @@ yeccpars2_168_(__Stack0) -> { 'case' , ? line ( __1 ) , __2 , __4 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8597). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8597). -compile({inline,yeccpars2_170_/1}). -file("erl_parse.yrl", 270). yeccpars2_170_(__Stack0) -> @@ -8603,7 +8603,7 @@ yeccpars2_170_(__Stack0) -> { block , ? line ( __1 ) , __2 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8606). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8606). -compile({inline,yeccpars2_172_/1}). -file("erl_parse.yrl", 279). yeccpars2_172_(__Stack0) -> @@ -8612,7 +8612,7 @@ yeccpars2_172_(__Stack0) -> { nil , ? line ( __1 ) } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8615). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8615). -compile({inline,yeccpars2_173_/1}). -file("erl_parse.yrl", 280). yeccpars2_173_(__Stack0) -> @@ -8621,7 +8621,7 @@ yeccpars2_173_(__Stack0) -> { cons , ? line ( __1 ) , __2 , __3 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8624). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8624). -compile({inline,yeccpars2_175_/1}). -file("erl_parse.yrl", 282). yeccpars2_175_(__Stack0) -> @@ -8638,7 +8638,7 @@ yeccpars2_178_(__Stack0) -> __2 end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8641). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8641). -compile({inline,yeccpars2_180_/1}). -file("erl_parse.yrl", 284). yeccpars2_180_(__Stack0) -> @@ -8662,7 +8662,7 @@ yeccpars2_186_(__Stack0) -> [ __1 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8665). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8665). -compile({inline,yeccpars2_187_/1}). -file("erl_parse.yrl", 287). yeccpars2_187_(__Stack0) -> @@ -8679,7 +8679,7 @@ yeccpars2_189_(__Stack0) -> [ __1 | __3 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8682). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8682). -compile({inline,yeccpars2_190_/1}). -file("erl_parse.yrl", 288). yeccpars2_190_(__Stack0) -> @@ -8688,7 +8688,7 @@ yeccpars2_190_(__Stack0) -> { bin , ? line ( __1 ) , __2 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8691). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8691). -compile({inline,yeccpars2_193_/1}). -file("erl_parse.yrl", 317). yeccpars2_193_(__Stack0) -> @@ -8712,7 +8712,7 @@ yeccpars2_197_(__Stack0) -> __2 end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8715). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8715). -compile({inline,yeccpars2_198_/1}). -file("erl_parse.yrl", 294). yeccpars2_198_(__Stack0) -> @@ -8761,7 +8761,7 @@ yeccpars2_206_(__Stack0) -> [ __1 | __3 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8764). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8764). -compile({inline,yeccpars2_207_/1}). -file("erl_parse.yrl", 296). yeccpars2_207_(__Stack0) -> @@ -8770,7 +8770,7 @@ yeccpars2_207_(__Stack0) -> ? mkop1 ( __1 , __2 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8773). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8773). -compile({inline,yeccpars2_208_/1}). -file("erl_parse.yrl", 256). yeccpars2_208_(__Stack0) -> @@ -8787,7 +8787,7 @@ yeccpars2_210_(__Stack0) -> __2 end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8790). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8790). -compile({inline,yeccpars2_212_/1}). -file("erl_parse.yrl", 340). yeccpars2_212_(__Stack0) -> @@ -8812,7 +8812,7 @@ yeccpars2_219_(__Stack0) -> [ ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8815). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8815). -compile({inline,yeccpars2_221_/1}). -file("erl_parse.yrl", 356). yeccpars2_221_(__Stack0) -> @@ -8821,7 +8821,7 @@ yeccpars2_221_(__Stack0) -> { record_field , ? line ( __1 ) , __1 , __3 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8824). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8824). -compile({inline,yeccpars2_223_/1}). -file("erl_parse.yrl", 357). yeccpars2_223_(__Stack0) -> @@ -8846,7 +8846,7 @@ yeccpars2_226_(__Stack0) -> __2 end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8849). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8849). -compile({inline,yeccpars2_227_/1}). -file("erl_parse.yrl", 338). yeccpars2_227_(__Stack0) -> @@ -8863,7 +8863,7 @@ yeccpars2_229_(__Stack0) -> [ __1 | __3 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8866). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8866). -compile({inline,yeccpars2_232_/1}). -file("erl_parse.yrl", 217). yeccpars2_232_(__Stack0) -> @@ -8872,7 +8872,7 @@ yeccpars2_232_(__Stack0) -> { match , ? line ( __2 ) , __1 , __3 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8875). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8875). -compile({inline,yeccpars2_233_/1}). -file("erl_parse.yrl", 218). yeccpars2_233_(__Stack0) -> @@ -8881,7 +8881,7 @@ yeccpars2_233_(__Stack0) -> ? mkop2 ( __1 , __2 , __3 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8884). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8884). -compile({inline,yeccpars2_235_/1}). -file("erl_parse.yrl", 221). yeccpars2_235_(__Stack0) -> @@ -8890,7 +8890,7 @@ yeccpars2_235_(__Stack0) -> ? mkop2 ( __1 , __2 , __3 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8893). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8893). -compile({inline,yeccpars2_237_/1}). -file("erl_parse.yrl", 224). yeccpars2_237_(__Stack0) -> @@ -8899,7 +8899,7 @@ yeccpars2_237_(__Stack0) -> ? mkop2 ( __1 , __2 , __3 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8902). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8902). -compile({inline,yeccpars2_247_/1}). -file("erl_parse.yrl", 228). yeccpars2_247_(__Stack0) -> @@ -8908,7 +8908,7 @@ yeccpars2_247_(__Stack0) -> ? mkop2 ( __1 , __2 , __3 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8911). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8911). -compile({inline,yeccpars2_260_/1}). -file("erl_parse.yrl", 236). yeccpars2_260_(__Stack0) -> @@ -8917,7 +8917,7 @@ yeccpars2_260_(__Stack0) -> ? mkop2 ( __1 , __2 , __3 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8920). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8920). -compile({inline,yeccpars2_268_/1}). -file("erl_parse.yrl", 240). yeccpars2_268_(__Stack0) -> @@ -8926,7 +8926,7 @@ yeccpars2_268_(__Stack0) -> ? mkop2 ( __1 , __2 , __3 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8929). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8929). -compile({inline,yeccpars2_269_/1}). -file("erl_parse.yrl", 232). yeccpars2_269_(__Stack0) -> @@ -8935,7 +8935,7 @@ yeccpars2_269_(__Stack0) -> ? mkop2 ( __1 , __2 , __3 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8938). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8938). -compile({inline,yeccpars2_270_/1}). -file("erl_parse.yrl", 362). yeccpars2_270_(__Stack0) -> @@ -8944,7 +8944,7 @@ yeccpars2_270_(__Stack0) -> { call , ? line ( __1 ) , __1 , element ( 1 , __2 ) } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8947). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8947). -compile({inline,yeccpars2_273_/1}). -file("erl_parse.yrl", 252). yeccpars2_273_(__Stack0) -> @@ -8953,7 +8953,7 @@ yeccpars2_273_(__Stack0) -> { remote , ? line ( __2 ) , __1 , __3 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8956). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8956). -compile({inline,yeccpars2_274_/1}). -file("erl_parse.yrl", 258). yeccpars2_274_(__Stack0) -> @@ -8962,7 +8962,7 @@ yeccpars2_274_(__Stack0) -> { record_field , ? line ( __2 ) , __1 , __3 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8965). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8965). -compile({inline,yeccpars2_277_/1}). -file("erl_parse.yrl", 344). yeccpars2_277_(__Stack0) -> @@ -8971,7 +8971,7 @@ yeccpars2_277_(__Stack0) -> { record , ? line ( __2 ) , __1 , element ( 3 , __3 ) , __4 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8974). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8974). -compile({inline,yeccpars2_279_/1}). -file("erl_parse.yrl", 342). yeccpars2_279_(__Stack0) -> @@ -8980,7 +8980,7 @@ yeccpars2_279_(__Stack0) -> { record_field , ? line ( __2 ) , __1 , element ( 3 , __3 ) , __5 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8983). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8983). -compile({inline,yeccpars2_280_/1}). -file("erl_parse.yrl", 435). yeccpars2_280_(__Stack0) -> @@ -8989,7 +8989,7 @@ yeccpars2_280_(__Stack0) -> { __2 , ? line ( __1 ) } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8992). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 8992). -compile({inline,yeccpars2_281_/1}). -file("erl_parse.yrl", 244). yeccpars2_281_(__Stack0) -> @@ -8998,7 +8998,7 @@ yeccpars2_281_(__Stack0) -> ? mkop1 ( __1 , __2 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9001). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9001). -compile({inline,yeccpars2_284_/1}). -file("erl_parse.yrl", 348). yeccpars2_284_(__Stack0) -> @@ -9007,7 +9007,7 @@ yeccpars2_284_(__Stack0) -> { record , ? line ( __2 ) , __1 , element ( 3 , __3 ) , __4 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9010). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9010). -compile({inline,yeccpars2_286_/1}). -file("erl_parse.yrl", 346). yeccpars2_286_(__Stack0) -> @@ -9016,7 +9016,7 @@ yeccpars2_286_(__Stack0) -> { record_field , ? line ( __2 ) , __1 , element ( 3 , __3 ) , __5 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9019). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9019). -compile({inline,yeccpars2_288_/1}). -file("erl_parse.yrl", 493). yeccpars2_288_(__Stack0) -> @@ -9025,7 +9025,7 @@ yeccpars2_288_(__Stack0) -> { clause , ? line ( __1 ) , element ( 3 , __1 ) , __2 , __3 , __4 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9028). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9028). -compile({inline,yeccpars2_289_/1}). -file("erl_parse.yrl", 203). yeccpars2_289_(__Stack0) -> @@ -9090,7 +9090,7 @@ yeccpars2_318_(__Stack0) -> [ __1 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9093). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9093). -compile({inline,yeccpars2_332_/1}). -file("erl_parse.yrl", 152). yeccpars2_332_(__Stack0) -> @@ -9099,7 +9099,7 @@ yeccpars2_332_(__Stack0) -> { type , ? line ( __1 ) , tuple , [ ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9102). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9102). -compile({inline,yeccpars2_333_/1}). -file("erl_parse.yrl", 153). yeccpars2_333_(__Stack0) -> @@ -9108,7 +9108,7 @@ yeccpars2_333_(__Stack0) -> { type , ? line ( __1 ) , tuple , __2 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9111). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9111). -compile({inline,yeccpars2_335_/1}). -file("erl_parse.yrl", 116). yeccpars2_335_(__Stack0) -> @@ -9117,7 +9117,7 @@ yeccpars2_335_(__Stack0) -> { ann_type , ? line ( __1 ) , [ __1 , __3 ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9120). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9120). -compile({inline,yeccpars2_341_/1}). -file("erl_parse.yrl", 159). yeccpars2_341_(__Stack0) -> @@ -9126,7 +9126,7 @@ yeccpars2_341_(__Stack0) -> { type , ? line ( __1 ) , 'fun' , [ ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9129). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9129). -compile({inline,yeccpars2_345_/1}). -file("erl_parse.yrl", 163). yeccpars2_345_(__Stack0) -> @@ -9144,7 +9144,7 @@ yeccpars2_346_(__Stack0) -> __3 end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9147). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9147). -compile({inline,yeccpars2_352_/1}). -file("erl_parse.yrl", 144). yeccpars2_352_(__Stack0) -> @@ -9154,7 +9154,7 @@ yeccpars2_352_(__Stack0) -> [ __1 , __3 , [ ] ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9157). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9157). -compile({inline,yeccpars2_353_/1}). -file("erl_parse.yrl", 146). yeccpars2_353_(__Stack0) -> @@ -9172,7 +9172,7 @@ yeccpars2_355_(__Stack0) -> build_gen_type ( __1 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9175). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9175). -compile({inline,yeccpars2_356_/1}). -file("erl_parse.yrl", 142). yeccpars2_356_(__Stack0) -> @@ -9182,7 +9182,7 @@ yeccpars2_356_(__Stack0) -> normalise ( __1 ) , __3 } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9185). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9185). -compile({inline,yeccpars2_358_/1}). -file("erl_parse.yrl", 148). yeccpars2_358_(__Stack0) -> @@ -9191,7 +9191,7 @@ yeccpars2_358_(__Stack0) -> { type , ? line ( __1 ) , nil , [ ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9194). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9194). -compile({inline,yeccpars2_360_/1}). -file("erl_parse.yrl", 149). yeccpars2_360_(__Stack0) -> @@ -9200,7 +9200,7 @@ yeccpars2_360_(__Stack0) -> { type , ? line ( __1 ) , list , [ __2 ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9203). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9203). -compile({inline,yeccpars2_362_/1}). -file("erl_parse.yrl", 150). yeccpars2_362_(__Stack0) -> @@ -9210,7 +9210,7 @@ yeccpars2_362_(__Stack0) -> nonempty_list , [ __2 ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9213). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9213). -compile({inline,yeccpars2_365_/1}). -file("erl_parse.yrl", 179). yeccpars2_365_(__Stack0) -> @@ -9237,7 +9237,7 @@ yeccpars2_371_(__Stack0) -> build_bin_type ( [ __1 , __3 ] , __5 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9240). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9240). -compile({inline,yeccpars2_373_/1}). -file("erl_parse.yrl", 182). yeccpars2_373_(__Stack0) -> @@ -9247,7 +9247,7 @@ yeccpars2_373_(__Stack0) -> [ __2 , abstract ( 0 , ? line ( __1 ) ) ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9250). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9250). -compile({inline,yeccpars2_378_/1}). -file("erl_parse.yrl", 187). yeccpars2_378_(__Stack0) -> @@ -9256,7 +9256,7 @@ yeccpars2_378_(__Stack0) -> { type , ? line ( __1 ) , binary , [ __2 , __4 ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9259). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9259). -compile({inline,yeccpars2_379_/1}). -file("erl_parse.yrl", 184). yeccpars2_379_(__Stack0) -> @@ -9266,7 +9266,7 @@ yeccpars2_379_(__Stack0) -> [ abstract ( 0 , ? line ( __1 ) ) , __2 ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9269). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9269). -compile({inline,yeccpars2_381_/1}). -file("erl_parse.yrl", 167). yeccpars2_381_(__Stack0) -> @@ -9276,7 +9276,7 @@ yeccpars2_381_(__Stack0) -> [ { type , ? line ( __1 ) , product , [ ] } , __4 ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9279). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9279). -compile({inline,yeccpars2_383_/1}). -file("erl_parse.yrl", 138). yeccpars2_383_(__Stack0) -> @@ -9293,7 +9293,7 @@ yeccpars2_387_(__Stack0) -> [ __1 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9296). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9296). -compile({inline,yeccpars2_389_/1}). -file("erl_parse.yrl", 154). yeccpars2_389_(__Stack0) -> @@ -9302,7 +9302,7 @@ yeccpars2_389_(__Stack0) -> { type , ? line ( __1 ) , record , [ __2 ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9305). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9305). -compile({inline,yeccpars2_391_/1}). -file("erl_parse.yrl", 176). yeccpars2_391_(__Stack0) -> @@ -9320,7 +9320,7 @@ yeccpars2_393_(__Stack0) -> [ __1 | __3 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9323). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9323). -compile({inline,yeccpars2_394_/1}). -file("erl_parse.yrl", 155). yeccpars2_394_(__Stack0) -> @@ -9330,7 +9330,7 @@ yeccpars2_394_(__Stack0) -> record , [ __2 | __4 ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9333). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9333). -compile({inline,yeccpars2_395_/1}). -file("erl_parse.yrl", 135). yeccpars2_395_(__Stack0) -> @@ -9347,7 +9347,7 @@ yeccpars2_397_(__Stack0) -> [ __1 | __3 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9350). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9350). -compile({inline,yeccpars2_400_/1}). -file("erl_parse.yrl", 170). yeccpars2_400_(__Stack0) -> @@ -9365,7 +9365,7 @@ yeccpars2_402_(__Stack0) -> lift_unions ( __1 , __3 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9368). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9368). -compile({inline,yeccpars2_405_/1}). -file("erl_parse.yrl", 122). yeccpars2_405_(__Stack0) -> @@ -9376,7 +9376,7 @@ yeccpars2_405_(__Stack0) -> skip_paren ( __3 ) ] } end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9379). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9379). -compile({inline,yeccpars2_406_/1}). -file("erl_parse.yrl", 127). yeccpars2_406_(__Stack0) -> @@ -9386,7 +9386,7 @@ yeccpars2_406_(__Stack0) -> __2 , skip_paren ( __3 ) ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9389). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9389). -compile({inline,yeccpars2_408_/1}). -file("erl_parse.yrl", 131). yeccpars2_408_(__Stack0) -> @@ -9396,7 +9396,7 @@ yeccpars2_408_(__Stack0) -> __2 , skip_paren ( __3 ) ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9399). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9399). -compile({inline,yeccpars2_410_/1}). -file("erl_parse.yrl", 103). yeccpars2_410_(__Stack0) -> @@ -9422,7 +9422,7 @@ yeccpars2_415_(__Stack0) -> build_def ( __1 , __3 ) end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9425). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9425). -compile({inline,yeccpars2_418_/1}). -file("erl_parse.yrl", 109). yeccpars2_418_(__Stack0) -> @@ -9552,7 +9552,7 @@ yeccpars2_446_(__Stack0) -> [ __1 | __3 ] end | __Stack]. --file("/ldisk/bjorn/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9555). +-file("/ldisk/pan/git/otp/bootstrap/lib/stdlib/egen/erl_parse.erl", 9555). -compile({inline,yeccpars2_447_/1}). -file("erl_parse.yrl", 90). yeccpars2_447_(__Stack0) -> diff --git a/erts/aclocal.m4 b/erts/aclocal.m4 index 443d8622bf..a1211bbf0c 100644 --- a/erts/aclocal.m4 +++ b/erts/aclocal.m4 @@ -747,9 +747,124 @@ case "$THR_LIB_NAME" in if test $found_win32_winnt = no; then AC_MSG_ERROR([-D_WIN32_WINNT missing in CPPFLAGS]) fi - ethr_have_native_atomics=yes - ethr_have_native_spinlock=yes + AC_DEFINE(ETHR_WIN32_THREADS, 1, [Define if you have win32 threads]) + + have_ilckd=no + AC_MSG_CHECKING([for _InterlockedCompareExchange64()]) + AC_TRY_LINK([ + #define WIN32_LEAN_AND_MEAN + #include <windows.h> + ], + [ + volatile __int64 *var; + _InterlockedCompareExchange64(var, (__int64) 1, (__int64) 0); + return 0; + ], + have_ilckd=yes) + AC_MSG_RESULT([$have_ilckd]) + test $have_ilckd = yes && AC_DEFINE(ETHR_HAVE__INTERLOCKEDCOMPAREEXCHANGE64, 1, [Define if you have _InterlockedCompareExchange64()]) + + AC_CHECK_SIZEOF(void *) + case "$ac_cv_sizeof_void_p-$have_ilckd" in + 8-no) + ethr_have_native_atomics=no + ethr_have_native_spinlock=no;; + *) + ethr_have_native_atomics=yes + ethr_have_native_spinlock=yes;; + esac + + have_ilckd=no + AC_MSG_CHECKING([for _InterlockedDecrement64()]) + AC_TRY_LINK([ + #define WIN32_LEAN_AND_MEAN + #include <windows.h> + ], + [ + volatile __int64 *var; + _InterlockedDecrement64(var); + return 0; + ], + have_ilckd=yes) + AC_MSG_RESULT([$have_ilckd]) + test $have_ilckd = yes && AC_DEFINE(ETHR_HAVE__INTERLOCKEDDECREMENT64, 1, [Define if you have _InterlockedDecrement64()]) + + have_ilckd=no + AC_MSG_CHECKING([for _InterlockedIncrement64()]) + AC_TRY_LINK([ + #define WIN32_LEAN_AND_MEAN + #include <windows.h> + ], + [ + volatile __int64 *var; + _InterlockedIncrement64(var); + return 0; + ], + have_ilckd=yes) + AC_MSG_RESULT([$have_ilckd]) + test $have_ilckd = yes && AC_DEFINE(ETHR_HAVE__INTERLOCKEDINCREMENT64, 1, [Define if you have _InterlockedIncrement64()]) + + have_ilckd=no + AC_MSG_CHECKING([for _InterlockedExchangeAdd64()]) + AC_TRY_LINK([ + #define WIN32_LEAN_AND_MEAN + #include <windows.h> + ], + [ + volatile __int64 *var; + _InterlockedExchangeAdd64(var, (__int64) 1); + return 0; + ], + have_ilckd=yes) + AC_MSG_RESULT([$have_ilckd]) + test $have_ilckd = yes && AC_DEFINE(ETHR_HAVE__INTERLOCKEDEXCHANGEADD64, 1, [Define if you have _InterlockedExchangeAdd64()]) + + have_ilckd=no + AC_MSG_CHECKING([for _InterlockedExchange64()]) + AC_TRY_LINK([ + #define WIN32_LEAN_AND_MEAN + #include <windows.h> + ], + [ + volatile __int64 *var; + _InterlockedExchange64(var, (__int64) 1); + return 0; + ], + have_ilckd=yes) + AC_MSG_RESULT([$have_ilckd]) + test $have_ilckd = yes && AC_DEFINE(ETHR_HAVE__INTERLOCKEDEXCHANGE64, 1, [Define if you have _InterlockedExchange64()]) + + have_ilckd=no + AC_MSG_CHECKING([for _InterlockedAnd64()]) + AC_TRY_LINK([ + #define WIN32_LEAN_AND_MEAN + #include <windows.h> + ], + [ + volatile __int64 *var; + _InterlockedAnd64(var, (__int64) 1); + return 0; + ], + have_ilckd=yes) + AC_MSG_RESULT([$have_ilckd]) + test $have_ilckd = yes && AC_DEFINE(ETHR_HAVE__INTERLOCKEDAND64, 1, [Define if you have _InterlockedAnd64()]) + + have_ilckd=no + AC_MSG_CHECKING([for _InterlockedOr64()]) + AC_TRY_LINK([ + #define WIN32_LEAN_AND_MEAN + #include <windows.h> + ], + [ + volatile __int64 *var; + _InterlockedOr64(var, (__int64) 1); + return 0; + ], + have_ilckd=yes) + AC_MSG_RESULT([$have_ilckd]) + test $have_ilckd = yes && AC_DEFINE(ETHR_HAVE__INTERLOCKEDOR64, 1, [Define if you have _InterlockedOr64()]) + ;; pthread) @@ -1087,6 +1202,28 @@ fi AC_CHECK_SIZEOF(void *) AC_DEFINE_UNQUOTED(ETHR_SIZEOF_PTR, $ac_cv_sizeof_void_p, [Define to the size of pointers]) +AC_CHECK_SIZEOF(int) +AC_DEFINE_UNQUOTED(ETHR_SIZEOF_INT, $ac_cv_sizeof_int, [Define to the size of int]) +AC_CHECK_SIZEOF(long) +AC_DEFINE_UNQUOTED(ETHR_SIZEOF_LONG, $ac_cv_sizeof_long, [Define to the size of long]) +AC_CHECK_SIZEOF(long long) +AC_DEFINE_UNQUOTED(ETHR_SIZEOF_LONG_LONG, $ac_cv_sizeof_long_long, [Define to the size of long long]) +AC_CHECK_SIZEOF(__int64) +AC_DEFINE_UNQUOTED(ETHR_SIZEOF___INT64, $ac_cv_sizeof___int64, [Define to the size of __int64]) + + +case X$erl_xcomp_bigendian in + X) ;; + Xyes|Xno) ac_cv_c_bigendian=$erl_xcomp_bigendian;; + *) AC_MSG_ERROR([Bad erl_xcomp_bigendian value: $erl_xcomp_bigendian]);; +esac + +AC_C_BIGENDIAN + +if test "$ac_cv_c_bigendian" = "yes"; then + AC_DEFINE(ETHR_BIGENDIAN, 1, [Define if bigendian]) +fi + AC_ARG_ENABLE(native-ethr-impls, AS_HELP_STRING([--disable-native-ethr-impls], [disable native ethread implementations]), diff --git a/erts/autoconf/win32.config.cache.static b/erts/autoconf/win32.config.cache.static index cc33fc09b3..d25b1df9d9 100755 --- a/erts/autoconf/win32.config.cache.static +++ b/erts/autoconf/win32.config.cache.static @@ -61,7 +61,6 @@ ac_cv_func_fork=${ac_cv_func_fork=no} ac_cv_func_fork_works=${ac_cv_func_fork_works=no} ac_cv_func_fpsetmask=${ac_cv_func_fpsetmask=no} ac_cv_func_fstat=${ac_cv_func_fstat=yes} -ac_cv_func_getaddrinfo=${ac_cv_func_getaddrinfo=no} ac_cv_func_gethostbyaddr=${ac_cv_func_gethostbyaddr=no} ac_cv_func_gethostbyaddr_r=${ac_cv_func_gethostbyaddr_r=no} ac_cv_func_gethostbyname=${ac_cv_func_gethostbyname=no} @@ -71,7 +70,6 @@ ac_cv_func_gethostname=${ac_cv_func_gethostname=no} ac_cv_func_gethrtime=${ac_cv_func_gethrtime=no} ac_cv_func_getipnodebyaddr=${ac_cv_func_getipnodebyaddr=no} ac_cv_func_getipnodebyname=${ac_cv_func_getipnodebyname=no} -ac_cv_func_getnameinfo=${ac_cv_func_getnameinfo=no} ac_cv_func_getpagesize=${ac_cv_func_getpagesize=no} ac_cv_func_gettimeofday=${ac_cv_func_gettimeofday=no} ac_cv_func_gmtime_r=${ac_cv_func_gmtime_r=no} diff --git a/erts/configure.in b/erts/configure.in index 8d629c25ae..6e983a07b0 100644 --- a/erts/configure.in +++ b/erts/configure.in @@ -1479,7 +1479,7 @@ AC_CHECK_HEADERS(fcntl.h limits.h unistd.h syslog.h dlfcn.h ieeefp.h \ sys/ioctl.h sys/time.h sys/uio.h \ sys/socket.h sys/sockio.h sys/socketio.h \ net/errno.h malloc.h mach-o/dyld.h arpa/nameser.h \ - pty.h util.h utmp.h langinfo.h poll.h) + pty.h util.h utmp.h langinfo.h poll.h sdkddkver.h) AC_CHECK_HEADER(sys/resource.h, [AC_DEFINE(HAVE_SYS_RESOURCE_H, 1, @@ -1679,18 +1679,62 @@ LIBS="$LIBS $EMU_THR_X_LIBS" dnl Check if we have these, in which case we'll try to build dnl inet_gethost with ipv6 support. -AC_CHECK_FUNC(getaddrinfo, have_getaddrinfo=yes, have_getaddrinfo=no) +AC_CHECK_HEADERS(windows.h) +AC_CHECK_HEADERS(winsock2.h) +AC_CHECK_HEADERS(ws2tcpip.h,[],[],[ +#ifdef HAVE_WINSOCK2_H +#include <winsock2.h> +#endif +#ifdef HAVE_WINDOWS_H +#include <windows.h> +#endif +]) +dnl AC_CHECK_FUNC(getaddrinfo, have_getaddrinfo=yes, have_getaddrinfo=no) +AC_MSG_CHECKING(for getaddrinfo) +AC_TRY_LINK([ +#include <stdlib.h> +#include <string.h> +#ifdef HAVE_WINSOCK2_H +#include <winsock2.h> +#endif +#ifdef HAVE_WINDOWS_H +#include <windows.h> +#endif +#ifdef HAVE_WS2TCPIP_H +#include <ws2tcpip.h> +#endif +#ifndef __WIN32__ +#include <sys/socket.h> +#include <netdb.h> +#endif +], +[ +getaddrinfo("","",NULL,NULL); +],have_getaddrinfo=yes, have_getaddrinfo=no) if test $have_getaddrinfo = yes; then + AC_MSG_RESULT([yes]) AC_MSG_CHECKING([whether getaddrinfo accepts enough flags]) - AC_TRY_RUN([ + AC_TRY_COMPILE([ #include <stdlib.h> #include <string.h> +#ifdef HAVE_WINSOCK2_H +#include <winsock2.h> +#endif +#ifdef HAVE_WINDOWS_H +#include <windows.h> +#endif +#ifdef HAVE_WS2TCPIP_H +#include <ws2tcpip.h> +#endif +#ifndef __WIN32__ #include <sys/socket.h> #include <netdb.h> -int main(int argc, char **argv) { +#endif +], +[ struct addrinfo hints, *ai; memset(&hints, 0, sizeof(hints)); - hints.ai_flags = (AI_CANONNAME|AI_V4MAPPED|AI_ADDRCONFIG); + hints.ai_flags = AI_CANONNAME; hints.ai_socktype = SOCK_STREAM; hints.ai_family = AF_INET6; if (getaddrinfo("::", NULL, &hints, &ai) == 0) { @@ -1699,26 +1743,48 @@ int main(int argc, char **argv) { } else { exit(1); } -} - ],, have_getaddrinfo=no, - [ - case X$erl_xcomp_getaddrinfo in - X) have_getaddrinfo=cross;; - Xyes|Xno) have_getaddrinfo=$erl_xcomp_getaddrinfo;; - *) AC_MSG_ERROR([Bad erl_xcomp_getaddrinfo value: $erl_xcomp_getaddrinfo]);; - esac - ]) +],, have_getaddrinfo=no) AC_MSG_RESULT($have_getaddrinfo) case $have_getaddrinfo in yes) AC_DEFINE(HAVE_GETADDRINFO, [1], [Define to 1 if you have a good `getaddrinfo' function.]);; - cross) - AC_MSG_WARN([result no guessed because of cross compilation]);; *) ;; esac +else + AC_MSG_RESULT([no]) +fi +AC_MSG_CHECKING(for getnameinfo) +AC_TRY_LINK([ +#include <stdlib.h> +#include <string.h> +#ifdef HAVE_WINSOCK2_H +#include <winsock2.h> +#endif +#ifdef HAVE_WINDOWS_H +#include <windows.h> +#endif +#ifdef HAVE_WS2TCPIP_H +#include <ws2tcpip.h> +#endif +#ifndef __WIN32__ +#include <sys/socket.h> +#include <netdb.h> +#endif +], +[ +getnameinfo(NULL,0,NULL,0,NULL,0,0); +],have_getnameinfo=yes, have_getnameinfo=no) +if test $have_getnameinfo = yes; then + AC_MSG_RESULT([yes]) + AC_DEFINE(HAVE_GETNAMEINFO, [1], + [Define to 1 if you have a good `getnameinfo' function.]) +else + AC_MSG_RESULT([no]) fi -AC_CHECK_FUNCS([getnameinfo getipnodebyname getipnodebyaddr gethostbyname2]) + + +AC_CHECK_FUNCS([getipnodebyname getipnodebyaddr gethostbyname2]) AC_CHECK_FUNCS([ieee_handler fpsetmask finite isnan isinf res_gethostbyname dlopen \ pread pwrite writev memmove strerror strerror_r strncasecmp \ diff --git a/erts/doc/src/erl.xml b/erts/doc/src/erl.xml index 9224d73b6f..77bd952d41 100644 --- a/erts/doc/src/erl.xml +++ b/erts/doc/src/erl.xml @@ -550,6 +550,19 @@ <p>Force the <c>compressed</c> option on all ETS tables. Only intended for test and evaluation.</p> </item> + <tag><c><![CDATA[+fnl]]></c></tag> + <item> + <p>The VM works with file names as if they are encoded using the ISO-latin-1 encoding, disallowing Unicode characters with codepoints beyond 255. This is default on operating systems that have transparent file naming, i.e. all Unixes except MacOSX.</p> + </item> + <tag><c><![CDATA[+fnu]]></c></tag> + <item> + <p>The VM works with file names as if they are encoded using UTF-8 (or some other system specific Unicode encoding). This is the default on operating systems that enforce Unicode encoding, i.e. Windows and MacOSX.</p> + <p>By enabling Unicode file name translation on systems where this is not default, you open up to the possibility that some file names can not be interpreted by the VM and therefore will be returned to the program as raw binaries. The option is therefore considered experimental.</p> + </item> + <tag><c><![CDATA[+fna]]></c></tag> + <item> + <p>Selection between <c>+fnl</c> and <c>+fnu</c> is done based on the current locale settings in the OS, meaning that if you have set your terminal for UTF-8 encoding, the filesystem is expected to use the same encoding for filenames (use with care).</p> + </item> <tag><c><![CDATA[+hms Size]]></c></tag> <item> <p>Sets the default heap size of processes to the size diff --git a/erts/doc/src/erlang.xml b/erts/doc/src/erlang.xml index 638f7eef10..78d58a1e56 100644 --- a/erts/doc/src/erlang.xml +++ b/erts/doc/src/erlang.xml @@ -2781,14 +2781,17 @@ os_prompt%</pre> <name>open_port(PortName, PortSettings) -> port()</name> <fsummary>Open a port</fsummary> <type> - <v>PortName = {spawn, Command} | {spawn_driver, Command} | {spawn_executable, Command} | {fd, In, Out}</v> + <v>PortName = {spawn, Command} | {spawn_driver, Command} | {spawn_executable, FileName} | {fd, In, Out}</v> <v> Command = string()</v> + <v> FileName = [ FileNameChar ] | binary()</v> + <v> FileNameChar = int() (1..255 or any Unicode codepoint, see description)</v> <v> In = Out = int()</v> <v>PortSettings = [Opt]</v> - <v> Opt = {packet, N} | stream | {line, L} | {cd, Dir} | {env, Env} | {args, [ string() ]} | {arg0, string()} | exit_status | use_stdio | nouse_stdio | stderr_to_stdout | in | out | binary | eof</v> + <v> Opt = {packet, N} | stream | {line, L} | {cd, Dir} | {env, Env} | {args, [ ArgString ]} | {arg0, ArgString} | exit_status | use_stdio | nouse_stdio | stderr_to_stdout | in | out | binary | eof</v> <v> N = 1 | 2 | 4</v> <v> L = int()</v> <v> Dir = string()</v> + <v> ArgString = [ FileNameChar ] | binary()</v> <v> Env = [{Name, Val}]</v> <v> Name = string()</v> <v> Val = string() | false</v> @@ -2851,7 +2854,26 @@ os_prompt%</pre> executed, the appropriate command interpreter will implicitly be invoked, but there will still be no command argument expansion or implicit PATH search.</p> - + + <p>The name of the executable as well as the arguments + given in <c>args</c> and <c>arg0</c> is subject to + Unicode file name translation if the system is running + in Unicode file name mode. To avoid + translation or force i.e. UTF-8, supply the executable + and/or arguments as a binary in the correct + encoding. See the <seealso + marker="kernel:file">file</seealso> module, the + <seealso marker="kernel:file#native_name_encoding/0"> + file:native_name_encoding/0</seealso> function and the + <seealso marker="stdlib:unicode_usage">stdlib users guide + </seealso> for details.</p> + + <note>The characters in the name (if given as a list) + can only be > 255 if the Erlang VM is started in + Unicode file name translation mode, otherwise the name + of the executable is limited to the ISO-latin-1 + character set.</note> + <p>If the <c>Command</c> cannot be run, an error exception, with the posix error code as the reason, is raised. The error reason may differ between operating @@ -2954,6 +2976,21 @@ os_prompt%</pre> should not be given in this list. The proper executable name will automatically be used as argv[0] where applicable.</p> + <p>When the Erlang VM is running in Unicode file name + mode, the arguments can contain any Unicode characters and + will be translated into whatever is appropriate on the + underlying OS, which means UTF-8 for all platforms except + Windows, which has other (more transparent) ways of + dealing with Unicode arguments to programs. To avoid + Unicode translation of arguments, they can be supplied as + binaries in whatever encoding is deemed appropriate.</p> + + <note>The characters in the arguments (if given as a + list of characters) can only be > 255 if the Erlang + VM is started in Unicode file name mode, + otherwise the arguments are limited to the + ISO-latin-1 character set.</note> + <p>If one, for any reason, wants to explicitly set the program name in the argument vector, the <c>arg0</c> option can be used.</p> @@ -2969,6 +3006,9 @@ os_prompt%</pre> responds to this is highly system dependent and no specific effect is guaranteed.</p> + <p>The unicode file name translation rules of the + <c>args</c> option apply to this option as well.</p> + </item> <tag><c>exit_status</c></tag> diff --git a/erts/doc/src/notes.xml b/erts/doc/src/notes.xml index 1703ce0942..77181d3407 100644 --- a/erts/doc/src/notes.xml +++ b/erts/doc/src/notes.xml @@ -30,6 +30,273 @@ </header> <p>This document describes the changes made to the ERTS application.</p> +<section><title>Erts 5.8.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix format_man_pages so it handles all man sections + and remove warnings/errors in various man pages. </p> + <p> + Own Id: OTP-8600</p> + </item> + <item> + <p> + The <c>configure</c> command line argument <seealso + marker="doc/installation_guide:INSTALL#How-to-Build-and-Install-ErlangOTP_A-Closer-Look-at-the-individual-Steps_Configuring">--enable-ethread-pre-pentium4-compatibility</seealso> + had no effect. This option is now also automatically + enabled if required on the build machine.</p> + <p> + Own Id: OTP-8847</p> + </item> + <item> + <p> + Windows 2003 and Windows XP pre SP3 would sometimes not + start the Erlang R14B VM at all due to a bug in the cpu + topology detection. The bug affects Windows only, no + other platform is even remotely affected. The bug is now + corrected.</p> + <p> + Own Id: OTP-8876</p> + </item> + <item> + <p> + The HiPE run-time in the 64-bit emulator could do a + 64-bit write to a 32-bit struct field. It happened to be + harmless on Intel/AMD processors. Corrected. (Thanks to + Mikael Pettersson.)</p> + <p> + Own Id: OTP-8877</p> + </item> + <item> + <p> + A bug in <seealso + marker="erl_driver#erl_drv_tsd_get">erl_drv_tsd_get()</seealso> + and <seealso + marker="erl_nif#enif_tsd_get">enif_tsd_get()</seealso> + could cause an emulator crash. These functions are + currently not used in OTP. That is, the crash only occur + on systems with user implemented NIF libraries, or + drivers that use one of these functions.</p> + <p> + Own Id: OTP-8889</p> + </item> + <item> + <p> + Calling <c>erlang:system_info({cpu_topology, + CpuTopologyType})</c> with another <c>CpuTopologyType</c> + element than one of the documented atoms <c>defined</c>, + <c>detected</c>, or <c>used</c> caused an emulator crash. + (Thanks to Paul Guyot)</p> + <p> + Own Id: OTP-8914</p> + </item> + <item> + <p> + The ERTS internal rwlock implementation could get into an + inconsistent state. This bug was very seldom triggered, + but could be during heavy contention. The bug was + introduced in R14B (erts-5.8.1).</p> + <p> + The bug was most likely to be triggered when using the + <c>read_concurrency</c> option on an ETS table that was + frequently accessed from multiple processes doing lots of + writes and reads. That is, in a situation where you + typically don't want to use the <c>read_concurrency</c> + option in the first place.</p> + <p> + Own Id: OTP-8925 Aux Id: OTP-8544 </p> + </item> + <item> + <p> + Tracing to port could cause an emulator crash when + unloading the trace driver.</p> + <p> + Own Id: OTP-8932</p> + </item> + <item> + <p> + Removed use of CancelIoEx on Windows that had been shown + to cause problems with some drivers.</p> + <p> + Own Id: OTP-8937</p> + </item> + <item> + <p> + The fallback implementation used when no native atomic + implementation was found did not compile. (Thanks to + Patrick Baggett, and Tuncer Ayaz)</p> + <p> + Own Id: OTP-8944</p> + </item> + <item> + <p> + Some integer values used during load balancing could + under rare circumstances wrap causing a load unbalance + between schedulers.</p> + <p> + Own Id: OTP-8950</p> + </item> + <item> + <p> + The windows VM now correctly handles appending to large + files (> 4GB).</p> + <p> + Own Id: OTP-8958</p> + </item> + <item> + <p> + Name resolving of IPv6 addresses has been implemented for + Windows versions that support it. The use of ancient + resolver flags (AI_V4MAPPED | AI_ADDRCONFIG) to the + getaddrinfo() function has been removed since e.g FreeBSD + regard mapped IPv4 addresses to be a security problem and + the semantics of the address configured flag is + uncertain.</p> + <p> + Own Id: OTP-8969</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The help texts produced by the <c>configure</c> scripts + in the top directory and in the erts directory have been + aligned and cleaned up.</p> + <p> + Own Id: OTP-8859</p> + </item> + <item> + <p> + When the runtime system had fewer schedulers than logical + processors, the system could get an unnecessarily large + amount reader groups.</p> + <p> + Own Id: OTP-8861</p> + </item> + <item> + <p> + <c>run_rel</c> has been updated to support Solaris's + /dev/ptmx device and to load the necessary STREAMS + modules so that <c>to_erl</c> can provide terminal echo + of keyboard input. (Thanks to Ryan Tilder.)</p> + <p> + Own Id: OTP-8878</p> + </item> + <item> + <p> + The Erlang VM now supports Unicode filenames. The feature + is turned on by default on systems where Unicode + filenames are mandatory (Windows and MacOSX), but can be + enabled on other systems with the '+fnu' emulator option. + Enabling the Unicode filename feature on systems where it + is not default is however considered experimental and not + to be used for production. Together with the Unicode file + name support, the concept of "raw filenames" is + introduced, which means filenames provided without + implicit unicode encoding translation. Raw filenames are + provided as binaries, not lists. For further information, + see stdlib users guide and the chapter about using + Unicode in Erlang. Also see the file module manual page.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8887</p> + </item> + <item> + <p>Buffer overflows have been prevented in <c>erlc</c>, + <c>dialyzer</c>, <c>typer</c>, <c>run_test</c>, + <c>heart</c>, <c>escript</c>, and <c>erlexec</c>.</p> + (Thanks to Michael Santos.) + <p> + Own Id: OTP-8892</p> + </item> + <item> + <p> + The runtime system is now less eager to suspend processes + sending messages over the distribution. The default value + of the distribution buffer busy limit has also been + increased from 128 KB to 1 MB. This in order to improve + throughput.</p> + <p> + Own Id: OTP-8901</p> + </item> + <item> + <p> + The distribution buffer busy limit can now be configured + at system startup. For more information see the + documentation of the <c>erl</c> <seealso + marker="erl#+zdbbl">+zdbbl</seealso> command line flag. + (Thanks to Scott Lystig Fritchie)</p> + <p> + Own Id: OTP-8912</p> + </item> + <item> + <p> + The inet driver internal buffer stack implementation has + been rewritten in order to reduce lock contention.</p> + <p> + Own Id: OTP-8916</p> + </item> + <item> + <p> + New ETS option <c>compressed</c>, to enable a more + compact storage format at the expence of heavier table + operations. For test and evaluation, <c>erl +ec</c> can + be used to force compression on all ETS tables.</p> + <p> + Own Id: OTP-8922 Aux Id: seq11658 </p> + </item> + <item> + <p> + There is now a new function inet:getifaddrs/0 modeled + after C library function getifaddrs() on BSD and LInux + that reports existing interfaces and their addresses on + the host. This replaces the undocumented and unsupported + inet:getiflist/0 and inet:ifget/2.</p> + <p> + Own Id: OTP-8926</p> + </item> + <item> + <p> + Support for detection of CPU topology and binding of + schedulers on FreeBSD 8 have been added. (Thanks to Paul + Guyot)</p> + <p> + Own Id: OTP-8939</p> + </item> + <item> + <p> + Several bugs related to hibernate/3 and HiPE have been + corrected. (Thanks to Paul Guyot.)</p> + <p> + Own Id: OTP-8952</p> + </item> + <item> + <p> + Support for soft and hard links on Windows versions and + filesystems that support them is added.</p> + <p> + Own Id: OTP-8955</p> + </item> + <item> + <p> + The win32 virtual machine is now linked large address + aware. his allows the Erlang VM to use up to 3 gigs of + address space on Windows instead of the default of 2 + gigs.</p> + <p> + Own Id: OTP-8956</p> + </item> + </list> + </section> + +</section> + <section><title>Erts 5.8.1.2</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/erts/emulator/Makefile.in b/erts/emulator/Makefile.in index 4ed0ccabc6..6c33e2ca16 100644 --- a/erts/emulator/Makefile.in +++ b/erts/emulator/Makefile.in @@ -796,7 +796,8 @@ endif OS_OBJS += $(OBJDIR)/erl_mseg.o \ $(OBJDIR)/erl_$(ERLANG_OSTYPE)_sys_ddll.o \ - $(OBJDIR)/erl_mtrace_sys_wrap.o + $(OBJDIR)/erl_mtrace_sys_wrap.o \ + $(OBJDIR)/erl_sys_common_misc.o HIPE_x86_OS_OBJS=$(HIPE_x86_$(OPSYS)_OBJS) HIPE_x86_OBJS=$(OBJDIR)/hipe_x86.o $(OBJDIR)/hipe_x86_glue.o $(OBJDIR)/hipe_x86_bifs.o $(OBJDIR)/hipe_x86_signal.o $(OBJDIR)/hipe_x86_stack.o $(HIPE_x86_OS_OBJS) diff --git a/erts/emulator/beam/beam_bp.c b/erts/emulator/beam/beam_bp.c index 682f31b83f..31910888d1 100644 --- a/erts/emulator/beam/beam_bp.c +++ b/erts/emulator/beam/beam_bp.c @@ -950,8 +950,8 @@ static int set_function_break(Module *modp, BeamInstr *pc, int bif, MatchSetUnref(old_match_spec); } else { BpDataCount *bdc = (BpDataCount *) bd; - long count = 0; - long res = 0; + erts_aint_t count = 0; + erts_aint_t res = 0; ASSERT(! match_spec); ASSERT(is_nil(tracer_pid)); diff --git a/erts/emulator/beam/beam_bp.h b/erts/emulator/beam/beam_bp.h index ebc171078d..bd8a7249a7 100644 --- a/erts/emulator/beam/beam_bp.h +++ b/erts/emulator/beam/beam_bp.h @@ -157,7 +157,7 @@ do { \ BpData **bds = (BpData **) (pc)[-4]; \ BpDataCount *bdc = NULL; \ Uint ix = bp_sched2ix_proc( (p) ); \ - long count = 0; \ + erts_aint_t count = 0; \ \ ASSERT((pc)[-5] == (BeamInstr) BeamOp(op_i_func_info_IaaI)); \ ASSERT(bds); \ diff --git a/erts/emulator/beam/bif.c b/erts/emulator/beam/bif.c index 6e9755ad48..d4a43f6e5f 100644 --- a/erts/emulator/beam/bif.c +++ b/erts/emulator/beam/bif.c @@ -813,7 +813,7 @@ BIF_RETTYPE spawn_opt_1(BIF_ALIST_1) so.min_heap_size = H_MIN_SIZE; so.min_vheap_size = BIN_VH_MIN_SIZE; so.priority = PRIORITY_NORMAL; - so.max_gen_gcs = (Uint16) erts_smp_atomic_read(&erts_max_gen_gcs); + so.max_gen_gcs = (Uint16) erts_smp_atomic32_read(&erts_max_gen_gcs); so.scheduler = 0; /* @@ -3269,12 +3269,13 @@ BIF_RETTYPE ports_0(BIF_ALIST_0) erts_smp_mtx_lock(&ports_snapshot_mtx); /* One snapshot at a time */ - erts_smp_atomic_set(&erts_dead_ports_ptr, (long) (port_buf + erts_max_ports)); + erts_smp_atomic_set(&erts_dead_ports_ptr, + (erts_aint_t) (port_buf + erts_max_ports)); next_ss = erts_smp_atomic_inctest(&erts_ports_snapshot); if (erts_smp_atomic_read(&erts_ports_alive) > 0) { - long i; + erts_aint_t i; for (i = erts_max_ports-1; i >= 0; i--) { Port* prt = &erts_port[i]; erts_smp_port_state_lock(prt); @@ -3289,7 +3290,7 @@ BIF_RETTYPE ports_0(BIF_ALIST_0) } dead_ports = (Eterm*)erts_smp_atomic_xchg(&erts_dead_ports_ptr, - (long)NULL); + (erts_aint_t) NULL); erts_smp_mtx_unlock(&ports_snapshot_mtx); ASSERT(pp <= dead_ports); @@ -3300,7 +3301,7 @@ BIF_RETTYPE ports_0(BIF_ALIST_0) ASSERT((alive+dead) <= erts_max_ports); if (alive+dead > 0) { - long i; + erts_aint_t i; Eterm *hp = HAlloc(BIF_P, (alive+dead)*2); for (i = 0; i < alive; i++) { @@ -3796,7 +3797,8 @@ BIF_RETTYPE system_flag_2(BIF_ALIST_2) goto error; } nval = (n > (Sint) ((Uint16) -1)) ? ((Uint16) -1) : ((Uint16) n); - oval = (Uint) erts_smp_atomic_xchg(&erts_max_gen_gcs, (long) nval); + oval = (Uint) erts_smp_atomic32_xchg(&erts_max_gen_gcs, + (erts_aint32_t) nval); BIF_RET(make_small(oval)); } else if (BIF_ARG_1 == am_min_heap_size) { int oval = H_MIN_SIZE; @@ -4139,7 +4141,7 @@ void erts_init_bif(void) erts_smp_spinlock_init(&make_ref_lock, "make_ref"); erts_smp_mtx_init(&ports_snapshot_mtx, "ports_snapshot"); - erts_smp_atomic_init(&erts_dead_ports_ptr, (long)NULL); + erts_smp_atomic_init(&erts_dead_ports_ptr, (erts_aint_t) NULL); /* * bif_return_trap/1 is a hidden BIF that bifs that need to diff --git a/erts/emulator/beam/bif.tab b/erts/emulator/beam/bif.tab index 0674aae77f..60b4b1946b 100644 --- a/erts/emulator/beam/bif.tab +++ b/erts/emulator/beam/bif.tab @@ -795,6 +795,13 @@ bif erlang:nif_error/1 bif erlang:nif_error/2 # +# Helpers for unicode filenames +# +bif prim_file:internal_name2native/1 +bif prim_file:internal_native2name/1 +bif prim_file:internal_normalize_utf8/1 +bif file:native_name_encoding/0 +# # Obsolete # diff --git a/erts/emulator/beam/binary.c b/erts/emulator/beam/binary.c index 8ee8fbcb29..4be869f269 100644 --- a/erts/emulator/beam/binary.c +++ b/erts/emulator/beam/binary.c @@ -217,8 +217,8 @@ erts_get_aligned_binary_bytes_extra(Eterm bin, byte** base_ptr, ErtsAlcType_t al return bytes; } -static Eterm -bin_bytes_to_list(Eterm previous, Eterm* hp, byte* bytes, Uint size, Uint bitoffs) +Eterm +erts_bin_bytes_to_list(Eterm previous, Eterm* hp, byte* bytes, Uint size, Uint bitoffs) { if (bitoffs == 0) { while (size) { @@ -263,7 +263,7 @@ BIF_RETTYPE binary_to_list_1(BIF_ALIST_1) Eterm* hp = HAlloc(BIF_P, 2 * size); byte* bytes = binary_bytes(real_bin)+offset; - BIF_RET(bin_bytes_to_list(NIL, hp, bytes, size, bitoffs)); + BIF_RET(erts_bin_bytes_to_list(NIL, hp, bytes, size, bitoffs)); } error: @@ -295,7 +295,7 @@ BIF_RETTYPE binary_to_list_3(BIF_ALIST_3) } i = stop-start+1; hp = HAlloc(BIF_P, 2*i); - BIF_RET(bin_bytes_to_list(NIL, hp, bytes+start-1, i, bitoffs)); + BIF_RET(erts_bin_bytes_to_list(NIL, hp, bytes+start-1, i, bitoffs)); error: BIF_ERROR(BIF_P, BADARG); @@ -339,7 +339,7 @@ BIF_RETTYPE bitstring_to_list_1(BIF_ALIST_1) previous = CONS(hp, make_binary(last), previous); hp += 2; } - BIF_RET(bin_bytes_to_list(previous, hp, bytes, size, bitoffs)); + BIF_RET(erts_bin_bytes_to_list(previous, hp, bytes, size, bitoffs)); } diff --git a/erts/emulator/beam/break.c b/erts/emulator/beam/break.c index f339e19761..fd2ec91e65 100644 --- a/erts/emulator/beam/break.c +++ b/erts/emulator/beam/break.c @@ -98,7 +98,7 @@ process_killer(void) switch(j) { case 'k': if (rp->status == P_WAITING) { - Uint32 rp_locks = ERTS_PROC_LOCKS_XSIG_SEND; + ErtsProcLocks rp_locks = ERTS_PROC_LOCKS_XSIG_SEND; erts_smp_proc_inc_refc(rp); erts_smp_proc_lock(rp, rp_locks); (void) erts_send_exit_signal(NULL, @@ -624,9 +624,9 @@ bin_check(void) erts_printf("Process %T holding binary data \n", rp->id); printed = 1; } - erts_printf("0x%08lx orig_size: %ld, norefs = %ld\n", - (unsigned long)bp->val, - (long)bp->val->orig_size, + erts_printf("%p orig_size: %bpd, norefs = %bpd\n", + bp->val, + bp->val->orig_size, erts_smp_atomic_read(&bp->val->refc)); } } diff --git a/erts/emulator/beam/erl_bif_binary.c b/erts/emulator/beam/erl_bif_binary.c index b6a445c55c..684fa5d12f 100644 --- a/erts/emulator/beam/erl_bif_binary.c +++ b/erts/emulator/beam/erl_bif_binary.c @@ -1477,7 +1477,7 @@ BIF_RETTYPE binary_matches_3(BIF_ALIST_3) goto badarg; } if (hsend == 0) { - BIF_RET(am_nomatch); + BIF_RET(NIL); } if (is_tuple(BIF_ARG_2)) { tp = tuple_val(BIF_ARG_2); diff --git a/erts/emulator/beam/erl_bif_ddll.c b/erts/emulator/beam/erl_bif_ddll.c index 2c2e283f65..c9cdcb87a6 100644 --- a/erts/emulator/beam/erl_bif_ddll.c +++ b/erts/emulator/beam/erl_bif_ddll.c @@ -1193,7 +1193,7 @@ int erts_ddll_driver_ok(DE_Handle *dh) static void ddll_no_more_references(void *vdh) { DE_Handle *dh = (DE_Handle *) vdh; - int x; + erts_aint_t x; lock_drv_list(); @@ -1604,7 +1604,7 @@ static int do_load_driver_entry(DE_Handle *dh, char *path, char *name) erts_sys_ddll_close(dh->handle); return ERL_DE_LOAD_ERROR_BAD_NAME; } - erts_smp_atomic_init(&(dh->refc), (long) 0); + erts_smp_atomic_init(&(dh->refc), (erts_aint_t) 0); dh->port_count = 0; dh->full_path = erts_alloc(ERTS_ALC_T_DDLL_HANDLE, sys_strlen(path) + 1); sys_strcpy(dh->full_path, path); @@ -1672,7 +1672,7 @@ static int load_driver_entry(DE_Handle **dhp, char *path, char *name) dh->handle = NULL; dh->procs = NULL; dh->port_count = 0; - erts_refc_init(&(dh->refc), (long) 0); + erts_refc_init(&(dh->refc), (erts_aint_t) 0); dh->status = -1; dh->reload_full_path = NULL; dh->reload_driver_name = NULL; diff --git a/erts/emulator/beam/erl_bif_info.c b/erts/emulator/beam/erl_bif_info.c index 75d8db880c..4a717d7271 100644 --- a/erts/emulator/beam/erl_bif_info.c +++ b/erts/emulator/beam/erl_bif_info.c @@ -2020,7 +2020,7 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) res = TUPLE2(hp, am_sequential_tracer, val); BIF_RET(res); } else if (BIF_ARG_1 == am_garbage_collection){ - Uint val = (Uint) erts_smp_atomic_read(&erts_max_gen_gcs); + Uint val = (Uint) erts_smp_atomic32_read(&erts_max_gen_gcs); Eterm tup; hp = HAlloc(BIF_P, 3+2 + 3+2 + 3+2); @@ -2035,7 +2035,7 @@ BIF_RETTYPE system_info_1(BIF_ALIST_1) BIF_RET(res); } else if (BIF_ARG_1 == am_fullsweep_after){ - Uint val = (Uint) erts_smp_atomic_read(&erts_max_gen_gcs); + Uint val = (Uint) erts_smp_atomic32_read(&erts_max_gen_gcs); hp = HAlloc(BIF_P, 3); res = TUPLE2(hp, am_fullsweep_after, make_small(val)); BIF_RET(res); @@ -3430,8 +3430,8 @@ BIF_RETTYPE erts_debug_set_internal_state_2(BIF_ALIST_2) */ if (ERTS_IS_ATOM_STR("available_internal_state", BIF_ARG_1) && (BIF_ARG_2 == am_true || BIF_ARG_2 == am_false)) { - long on = (long) (BIF_ARG_2 == am_true); - long prev_on = erts_smp_atomic_xchg(&available_internal_state, on); + erts_aint_t on = (erts_aint_t) (BIF_ARG_2 == am_true); + erts_aint_t prev_on = erts_smp_atomic_xchg(&available_internal_state, on); if (on) { erts_dsprintf_buf_t *dsbufp = erts_create_logger_dsbuf(); erts_dsprintf(dsbufp, "Process %T ", BIF_P->id); @@ -3628,7 +3628,7 @@ BIF_RETTYPE erts_debug_set_internal_state_2(BIF_ALIST_2) } else if (ERTS_IS_ATOM_STR("hipe_test_reschedule_suspend", BIF_ARG_1)) { /* Used by hipe test suites */ - long flag = erts_smp_atomic_read(&hipe_test_reschedule_flag); + erts_aint_t flag = erts_smp_atomic_read(&hipe_test_reschedule_flag); if (!flag && BIF_ARG_2 != am_false) { erts_smp_atomic_set(&hipe_test_reschedule_flag, 1); erts_suspend(BIF_P, ERTS_PROC_LOCK_MAIN, NULL); @@ -3703,7 +3703,7 @@ BIF_RETTYPE erts_debug_set_internal_state_2(BIF_ALIST_2) #ifdef ERTS_ENABLE_LOCK_COUNT static Eterm lcnt_build_lock_stats_term(Eterm **hpp, Uint *szp, erts_lcnt_lock_stats_t *stats, Eterm res) { - unsigned long tries = 0, colls = 0; + Uint tries = 0, colls = 0; unsigned long timer_s = 0, timer_ns = 0, timer_n = 0; unsigned int line = 0; @@ -3716,8 +3716,8 @@ static Eterm lcnt_build_lock_stats_term(Eterm **hpp, Uint *szp, erts_lcnt_lock_s * [{{file, line}, {tries, colls, {seconds, nanoseconds, n_blocks}}}] */ - tries = (unsigned long) ethr_atomic_read(&stats->tries); - colls = (unsigned long) ethr_atomic_read(&stats->colls); + tries = (Uint) ethr_atomic_read(&stats->tries); + colls = (Uint) ethr_atomic_read(&stats->colls); line = stats->line; timer_s = stats->timer.s; diff --git a/erts/emulator/beam/erl_bif_port.c b/erts/emulator/beam/erl_bif_port.c index 378c5e73fd..fbc92b9730 100644 --- a/erts/emulator/beam/erl_bif_port.c +++ b/erts/emulator/beam/erl_bif_port.c @@ -610,6 +610,7 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) int binary_io; int soft_eof; Sint linebuf; + Eterm edir = NIL; byte dir[MAXPATHLEN]; /* These are the defaults */ @@ -686,19 +687,10 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) } else if (option == am_arg0) { char *a0; - int n; - if (is_nil(*tp)) { - n = 0; - } else if( (n = is_string(*tp)) == 0) { + + if ((a0 = erts_convert_filename_to_native(*tp, ERTS_ALC_T_TMP, 1)) == NULL) { goto badarg; } - a0 = (char *) erts_alloc(ERTS_ALC_T_TMP, - (n + 1) * sizeof(byte)); - if (intlist_to_buf(*tp, a0, n) != n) { - erl_exit(1, "%s:%d: Internal error\n", - __FILE__, __LINE__); - } - a0[n] = '\0'; if (opts.argv == NULL) { opts.argv = erts_alloc(ERTS_ALC_T_TMP, 2 * sizeof(char **)); @@ -711,22 +703,7 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) opts.argv[0] = a0; } } else if (option == am_cd) { - Eterm iolist; - DeclareTmpHeap(heap,4,p); - int r; - - UseTmpHeap(4,p); - heap[0] = *tp; - heap[1] = make_list(heap+2); - heap[2] = make_small(0); - heap[3] = NIL; - iolist = make_list(heap); - r = io_list_to_buf(iolist, (char*) dir, MAXPATHLEN); - UnUseTmpHeap(4,p); - if (r < 0) { - goto badarg; - } - opts.wd = (char *) dir; + edir = *tp; } else { goto badarg; } @@ -838,19 +815,7 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) goto badarg; } name = tp[1]; - if (is_atom(name)) { - name_buf = (char *) erts_alloc(ERTS_ALC_T_TMP, - atom_tab(atom_val(name))->len+1); - sys_memcpy((void *) name_buf, - (void *) atom_tab(atom_val(name))->name, - atom_tab(atom_val(name))->len); - name_buf[atom_tab(atom_val(name))->len] = '\0'; - } else if ((i = is_string(name))) { - name_buf = (char *) erts_alloc(ERTS_ALC_T_TMP, i + 1); - if (intlist_to_buf(name, name_buf, i) != i) - erl_exit(1, "%s:%d: Internal error\n", __FILE__, __LINE__); - name_buf[i] = '\0'; - } else { + if ((name_buf = erts_convert_filename_to_native(name,ERTS_ALC_T_TMP,0)) == NULL) { goto badarg; } opts.spawn_type = ERTS_SPAWN_EXECUTABLE; @@ -892,7 +857,33 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) /* Argument vector only if explicit spawn_executable */ goto badarg; } - + + if (edir != NIL) { + /* A working directory is expressed differently if spawn_executable, i.e. Unicode is handles + for spawn_executable... */ + if (opts.spawn_type != ERTS_SPAWN_EXECUTABLE) { + Eterm iolist; + DeclareTmpHeap(heap,4,p); + int r; + + UseTmpHeap(4,p); + heap[0] = edir; + heap[1] = make_list(heap+2); + heap[2] = make_small(0); + heap[3] = NIL; + iolist = make_list(heap); + r = io_list_to_buf(iolist, (char*) dir, MAXPATHLEN); + UnUseTmpHeap(4,p); + if (r < 0) { + goto badarg; + } + opts.wd = (char *) dir; + } else { + if ((opts.wd = erts_convert_filename_to_native(edir,ERTS_ALC_T_TMP,0)) == NULL) { + goto badarg; + } + } + } if (driver != &spawn_driver && opts.exit_status) { goto badarg; @@ -941,6 +932,9 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) if (opts.argv) { free_args(opts.argv); } + if (opts.wd && opts.wd != ((char *)dir)) { + erts_free(ERTS_ALC_T_TMP, (void *) opts.wd); + } return port_num; badarg: @@ -950,6 +944,7 @@ open_port(Process* p, Eterm name, Eterm settings, int *err_nump) #undef OPEN_PORT_ERROR } +/* Arguments can be given i unicode and as raw binaries, convert filename is used to convert */ static char **convert_args(Eterm l) { char **pp; @@ -966,22 +961,14 @@ static char **convert_args(Eterm l) pp[i++] = erts_default_arg0; while (is_list(l)) { str = CAR(list_val(l)); - - if (is_nil(str)) { - n = 0; - } else if( (n = is_string(str)) == 0) { - /* Not a string... */ + if ((b = erts_convert_filename_to_native(str,ERTS_ALC_T_TMP,1)) == NULL) { int j; for (j = 1; j < i; ++j) erts_free(ERTS_ALC_T_TMP, pp[j]); erts_free(ERTS_ALC_T_TMP, pp); return NULL; - } - b = (char *) erts_alloc(ERTS_ALC_T_TMP, (n + 1) * sizeof(byte)); - pp[i++] = (char *) b; - if (intlist_to_buf(str, b, n) != n) - erl_exit(1, "%s:%d: Internal error\n", __FILE__, __LINE__); - b[n] = '\0'; + } + pp[i++] = b; l = CDR(list_val(l)); } pp[i] = NULL; diff --git a/erts/emulator/beam/erl_binary.h b/erts/emulator/beam/erl_binary.h index a569fe2e85..bdf0fe23fc 100644 --- a/erts/emulator/beam/erl_binary.h +++ b/erts/emulator/beam/erl_binary.h @@ -152,6 +152,8 @@ do { \ void erts_init_binary(void); byte* erts_get_aligned_binary_bytes_extra(Eterm, byte**, ErtsAlcType_t, unsigned extra); +/* Used by unicode module */ +Eterm erts_bin_bytes_to_list(Eterm previous, Eterm* hp, byte* bytes, Uint size, Uint bitoffs); /* * Common implementation for erlang:list_to_binary/1 and binary:list_to_bin/1 diff --git a/erts/emulator/beam/erl_cpu_topology.c b/erts/emulator/beam/erl_cpu_topology.c index db95c4a5d4..8a6b4d8d6c 100644 --- a/erts/emulator/beam/erl_cpu_topology.c +++ b/erts/emulator/beam/erl_cpu_topology.c @@ -487,7 +487,7 @@ erts_sched_check_cpu_bind_post_suspend(ErtsSchedulerData *esdp) /* Make sure we check if we should bind to a cpu or not... */ if (esdp->run_queue->flags & ERTS_RUNQ_FLG_SHARED_RUNQ) - erts_smp_atomic_set(&esdp->chk_cpu_bind, 1); + erts_smp_atomic32_set(&esdp->chk_cpu_bind, 1); else esdp->run_queue->flags |= ERTS_RUNQ_FLG_CHK_CPU_BIND; } @@ -503,7 +503,7 @@ erts_sched_check_cpu_bind(ErtsSchedulerData *esdp) erts_cpu_groups_callback_call_t *cgcc; #ifdef ERTS_SMP if (erts_common_run_queue) - erts_smp_atomic_set(&esdp->chk_cpu_bind, 0); + erts_smp_atomic32_set(&esdp->chk_cpu_bind, 0); else { esdp->run_queue->flags &= ~ERTS_RUNQ_FLG_CHK_CPU_BIND; } diff --git a/erts/emulator/beam/erl_db.c b/erts/emulator/beam/erl_db.c index 8577354d27..65565cf3c7 100644 --- a/erts/emulator/beam/erl_db.c +++ b/erts/emulator/beam/erl_db.c @@ -3259,7 +3259,7 @@ erts_db_process_exiting(Process *c_p, ErtsProcLocks c_p_locks) pp = &(*pp)->next) { if ((*pp)->pid == pid) { DbFixation* fix = *pp; - long diff = -(long)fix->counter; + erts_aint_t diff = -((erts_aint_t) fix->counter); erts_refc_add(&tb->common.fixref,diff,0); *pp = fix->next; erts_db_free(ERTS_ALC_T_DB_FIXATION, @@ -3415,7 +3415,7 @@ static void unfix_table_locked(Process* p, DbTable* tb, unlocked: if (!IS_FIXED(tb) && IS_HASH_TABLE(tb->common.status) - && erts_smp_atomic_read(&tb->hash.fixdel) != (long)NULL) { + && erts_smp_atomic_read(&tb->hash.fixdel) != (erts_aint_t)NULL) { #ifdef ERTS_SMP if (*kind_p == LCK_READ && tb->common.is_thread_safe) { /* Must have write lock while purging pseudo-deleted (OTP-8166) */ diff --git a/erts/emulator/beam/erl_db.h b/erts/emulator/beam/erl_db.h index cb2da603f0..e0bdebcb01 100644 --- a/erts/emulator/beam/erl_db.h +++ b/erts/emulator/beam/erl_db.h @@ -83,7 +83,8 @@ Eterm erts_ets_colliding_names(Process*, Eterm name, Uint cnt); #define ERTS_DB_ALC_MEM_UPDATE_(TAB, FREE_SZ, ALLOC_SZ) \ do { \ - long sz__ = ((long) (ALLOC_SZ)) - ((long) (FREE_SZ)); \ + erts_aint_t sz__ = (((erts_aint_t) (ALLOC_SZ)) \ + - ((erts_aint_t) (FREE_SZ))); \ ASSERT((TAB)); \ erts_smp_atomic_add(&(TAB)->common.memory_size, sz__); \ } while (0) diff --git a/erts/emulator/beam/erl_db_hash.c b/erts/emulator/beam/erl_db_hash.c index 14ee63100a..1e50fee554 100644 --- a/erts/emulator/beam/erl_db_hash.c +++ b/erts/emulator/beam/erl_db_hash.c @@ -135,8 +135,8 @@ static ERTS_INLINE Uint hash_to_ix(DbTableHash* tb, HashValue hval) */ static ERTS_INLINE void add_fixed_deletion(DbTableHash* tb, int ix) { - long was_next; - long exp_next; + erts_aint_t was_next; + erts_aint_t exp_next; FixedDeletion* fixd = (FixedDeletion*) erts_db_alloc(ERTS_ALC_T_DB_FIX_DEL, (DbTable *) tb, sizeof(FixedDeletion)); @@ -146,7 +146,9 @@ static ERTS_INLINE void add_fixed_deletion(DbTableHash* tb, int ix) do { /* Lockless atomic insertion in linked list: */ exp_next = was_next; fixd->next = (FixedDeletion*) exp_next; - was_next = erts_smp_atomic_cmpxchg(&tb->fixdel, (long)fixd, exp_next); + was_next = erts_smp_atomic_cmpxchg(&tb->fixdel, + (erts_aint_t) fixd, + exp_next); }while (was_next != exp_next); } @@ -541,12 +543,12 @@ static void restore_fixdel(DbTableHash* tb, FixedDeletion* fixdel) { /*int tries = 0;*/ DEBUG_WAIT(); - if (erts_smp_atomic_cmpxchg(&tb->fixdel, (long)fixdel, - (long)NULL) != (long)NULL) { + if (erts_smp_atomic_cmpxchg(&tb->fixdel, (erts_aint_t)fixdel, + (erts_aint_t)NULL) != (erts_aint_t)NULL) { /* Oboy, must join lists */ FixedDeletion* last = fixdel; - long was_tail; - long exp_tail; + erts_aint_t was_tail; + erts_aint_t exp_tail; while (last->next != NULL) last = last->next; was_tail = erts_smp_atomic_read(&tb->fixdel); @@ -555,7 +557,7 @@ static void restore_fixdel(DbTableHash* tb, FixedDeletion* fixdel) last->next = (FixedDeletion*) exp_tail; /*++tries;*/ DEBUG_WAIT(); - was_tail = erts_smp_atomic_cmpxchg(&tb->fixdel, (long)fixdel, + was_tail = erts_smp_atomic_cmpxchg(&tb->fixdel, (erts_aint_t)fixdel, exp_tail); }while (was_tail != exp_tail); } @@ -573,7 +575,7 @@ void db_unfix_table_hash(DbTableHash *tb) || (erts_smp_lc_rwmtx_is_rlocked(&tb->common.rwlock) && !tb->common.is_thread_safe)); restart: - fixdel = (FixedDeletion*) erts_smp_atomic_xchg(&tb->fixdel, (long)NULL); + fixdel = (FixedDeletion*) erts_smp_atomic_xchg(&tb->fixdel, (erts_aint_t)NULL); while (fixdel != NULL) { FixedDeletion *fx = fixdel; int ix = fx->slot; @@ -642,8 +644,8 @@ int db_create_hash(Process *p, DbTable *tbl) erts_smp_atomic_init(&tb->szm, SEGSZ_MASK); erts_smp_atomic_init(&tb->nactive, SEGSZ); - erts_smp_atomic_init(&tb->fixdel, (long)NULL); - erts_smp_atomic_init(&tb->segtab, (long) alloc_ext_seg(tb,0,NULL)->segtab); + erts_smp_atomic_init(&tb->fixdel, (erts_aint_t)NULL); + erts_smp_atomic_init(&tb->segtab, (erts_aint_t) alloc_ext_seg(tb,0,NULL)->segtab); tb->nsegs = NSEG_1; tb->nslots = SEGSZ; @@ -1715,9 +1717,9 @@ static int db_select_delete_hash(Process *p, Eterm mpb; Eterm egot; #ifdef ERTS_SMP - int fixated_by_me = tb->common.is_thread_safe ? 0 : 1; /* ToDo: something nicer */ + erts_aint_t fixated_by_me = tb->common.is_thread_safe ? 0 : 1; /* ToDo: something nicer */ #else - int fixated_by_me = 0; + erts_aint_t fixated_by_me = 0; #endif erts_smp_rwmtx_t* lck; @@ -2124,11 +2126,11 @@ static int db_free_table_continue_hash(DbTable *tbl) sizeof(FixedDeletion)); ERTS_ETS_MISC_MEM_ADD(-sizeof(FixedDeletion)); if (++done >= 2*DELETE_RECORD_LIMIT) { - erts_smp_atomic_set(&tb->fixdel, (long)fixdel); + erts_smp_atomic_set(&tb->fixdel, (erts_aint_t)fixdel); return 0; /* Not done */ } } - erts_smp_atomic_set(&tb->fixdel, (long)NULL); + erts_smp_atomic_set(&tb->fixdel, (erts_aint_t)NULL); done /= 2; while(tb->nslots != 0) { @@ -2345,7 +2347,7 @@ static int alloc_seg(DbTableHash *tb) struct ext_segment* eseg; eseg = (struct ext_segment*) SEGTAB(tb)[seg_ix-1]; MY_ASSERT(eseg!=NULL && eseg->s.is_ext_segment); - erts_smp_atomic_set(&tb->segtab, (long) eseg->segtab); + erts_smp_atomic_set(&tb->segtab, (erts_aint_t) eseg->segtab); tb->nsegs = eseg->nsegs; } ASSERT(seg_ix < tb->nsegs); @@ -2417,7 +2419,7 @@ static int free_seg(DbTableHash *tb, int free_records) MY_ASSERT(newtop->s.is_ext_segment); if (newtop->prev_segtab != NULL) { /* Time to use a smaller segtab */ - erts_smp_atomic_set(&tb->segtab, (long)newtop->prev_segtab); + erts_smp_atomic_set(&tb->segtab, (erts_aint_t)newtop->prev_segtab); tb->nsegs = seg_ix; ASSERT(tb->nsegs == EXTSEG(SEGTAB(tb))->nsegs); } @@ -2434,7 +2436,7 @@ static int free_seg(DbTableHash *tb, int free_records) if (seg_ix > 0) { if (seg_ix < tb->nsegs) SEGTAB(tb)[seg_ix] = NULL; } else { - erts_smp_atomic_set(&tb->segtab, (long)NULL); + erts_smp_atomic_set(&tb->segtab, (erts_aint_t)NULL); } #endif tb->nslots -= SEGSZ; diff --git a/erts/emulator/beam/erl_db_util.c b/erts/emulator/beam/erl_db_util.c index e773361619..2852fb93fe 100644 --- a/erts/emulator/beam/erl_db_util.c +++ b/erts/emulator/beam/erl_db_util.c @@ -915,7 +915,7 @@ BIF_RETTYPE db_set_trace_control_word_1(Process *p, Eterm new) if (val != ((Uint32)val)) BIF_ERROR(p, BADARG); - old_tcw = (Uint32) erts_smp_atomic_xchg(&trace_control_word, (long) val); + old_tcw = (Uint32) erts_smp_atomic_xchg(&trace_control_word, (erts_aint_t) val); BIF_RET(erts_make_integer((Uint) old_tcw, p)); } diff --git a/erts/emulator/beam/erl_driver.h b/erts/emulator/beam/erl_driver.h index 9733c0e5b5..13a73e01bb 100644 --- a/erts/emulator/beam/erl_driver.h +++ b/erts/emulator/beam/erl_driver.h @@ -150,6 +150,27 @@ typedef struct { #define ERL_DRV_FLAG_SOFT_BUSY (1 << 1) /* + * Integer types + */ + +typedef unsigned long ErlDrvTermData; +typedef unsigned long ErlDrvUInt; +typedef signed long ErlDrvSInt; + +#if defined(__WIN32__) +typedef unsigned __int64 ErlDrvUInt64; +typedef __int64 ErlDrvSInt64; +#elif SIZEOF_LONG == 8 +typedef unsigned long ErlDrvUInt64; +typedef long ErlDrvSInt64; +#elif SIZEOF_LONG_LONG == 8 +typedef unsigned long long ErlDrvUInt64; +typedef long long ErlDrvSInt64; +#else +#error No 64-bit integer type +#endif + +/* * A binary as seen in a driver. Note that a binary should never be * altered by the driver when it has been sent to Erlang. */ @@ -179,26 +200,6 @@ struct erl_drv_event_data { #endif typedef struct erl_drv_event_data *ErlDrvEventData; /* Event data */ -/* - * Used in monitors... - */ -typedef unsigned long ErlDrvTermData; -typedef unsigned long ErlDrvUInt; -typedef signed long ErlDrvSInt; - -#if defined(__WIN32__) -typedef unsigned __int64 ErlDrvUInt64; -typedef __int64 ErlDrvSInt64; -#elif SIZEOF_LONG == 8 -typedef unsigned long ErlDrvUInt64; -typedef long ErlDrvSInt64; -#elif SIZEOF_LONG_LONG == 8 -typedef unsigned long long ErlDrvUInt64; -typedef long long ErlDrvSInt64; -#else -#error No 64-bit integer type -#endif - /* * A driver monitor */ @@ -394,9 +395,9 @@ EXTERN int driver_exit (ErlDrvPort port, int err); EXTERN ErlDrvPDL driver_pdl_create(ErlDrvPort); EXTERN void driver_pdl_lock(ErlDrvPDL); EXTERN void driver_pdl_unlock(ErlDrvPDL); -EXTERN long driver_pdl_get_refc(ErlDrvPDL); -EXTERN long driver_pdl_inc_refc(ErlDrvPDL); -EXTERN long driver_pdl_dec_refc(ErlDrvPDL); +EXTERN ErlDrvSInt driver_pdl_get_refc(ErlDrvPDL); +EXTERN ErlDrvSInt driver_pdl_inc_refc(ErlDrvPDL); +EXTERN ErlDrvSInt driver_pdl_dec_refc(ErlDrvPDL); /* * Process monitors @@ -432,9 +433,9 @@ EXTERN ErlDrvBinary* driver_realloc_binary(ErlDrvBinary *bin, int size); EXTERN void driver_free_binary(ErlDrvBinary *bin); /* Referenc count on driver binaries */ -EXTERN long driver_binary_get_refc(ErlDrvBinary *dbp); -EXTERN long driver_binary_inc_refc(ErlDrvBinary *dbp); -EXTERN long driver_binary_dec_refc(ErlDrvBinary *dbp); +EXTERN ErlDrvSInt driver_binary_get_refc(ErlDrvBinary *dbp); +EXTERN ErlDrvSInt driver_binary_inc_refc(ErlDrvBinary *dbp); +EXTERN ErlDrvSInt driver_binary_dec_refc(ErlDrvBinary *dbp); /* Allocation interface */ EXTERN void *driver_alloc(size_t size); diff --git a/erts/emulator/beam/erl_fun.c b/erts/emulator/beam/erl_fun.c index 84869f12d6..88947b5536 100644 --- a/erts/emulator/beam/erl_fun.c +++ b/erts/emulator/beam/erl_fun.c @@ -97,7 +97,7 @@ erts_put_fun_entry(Eterm mod, int uniq, int index) { ErlFunEntry template; ErlFunEntry* fe; - long refc; + erts_aint_t refc; ASSERT(is_atom(mod)); template.old_uniq = uniq; template.old_index = index; @@ -119,7 +119,7 @@ erts_put_fun_entry2(Eterm mod, int old_uniq, int old_index, { ErlFunEntry template; ErlFunEntry* fe; - long refc; + erts_aint_t refc; ASSERT(is_atom(mod)); template.old_uniq = old_uniq; @@ -157,7 +157,7 @@ erts_get_fun_entry(Eterm mod, int uniq, int index) erts_fun_read_lock(); ret = (ErlFunEntry *) hash_get(&erts_fun_table, (void*) &template); if (ret) { - long refc = erts_refc_inctest(&ret->refc, 1); + erts_aint_t refc = erts_refc_inctest(&ret->refc, 1); if (refc < 2) /* Pending delete */ erts_refc_inc(&ret->refc, 1); } @@ -257,7 +257,7 @@ erts_dump_fun_entries(int to, void *to_arg) #ifdef HIPE erts_print(to, to_arg, "Native_address: %p\n", fe->native_address); #endif - erts_print(to, to_arg, "Refc: %d\n", erts_refc_read(&fe->refc, 1)); + erts_print(to, to_arg, "Refc: %ld\n", erts_refc_read(&fe->refc, 1)); b = b->next; } } diff --git a/erts/emulator/beam/erl_gc.c b/erts/emulator/beam/erl_gc.c index 0f4d2a2ef9..2aa932e7d1 100644 --- a/erts/emulator/beam/erl_gc.c +++ b/erts/emulator/beam/erl_gc.c @@ -2471,7 +2471,7 @@ erts_check_off_heap2(Process *p, Eterm *htop) old = 0; for (u.hdr = MSO(p).first; u.hdr; u.hdr = u.hdr->next) { - long refc; + erts_aint_t refc; switch (thing_subtag(u.hdr->thing_word)) { case REFC_BINARY_SUBTAG: refc = erts_refc_read(&u.pb->val->refc, 1); diff --git a/erts/emulator/beam/erl_init.c b/erts/emulator/beam/erl_init.c index a9f4f041ac..f4e0717d30 100644 --- a/erts/emulator/beam/erl_init.c +++ b/erts/emulator/beam/erl_init.c @@ -100,7 +100,7 @@ int erts_backtrace_depth; /* How many functions to show in a backtrace int erts_async_max_threads; /* number of threads for async support */ int erts_async_thread_suggested_stack_size; -erts_smp_atomic_t erts_max_gen_gcs; +erts_smp_atomic32_t erts_max_gen_gcs; Eterm erts_error_logger_warnings; /* What to map warning logs to, am_error, am_info or am_warning, am_error is @@ -251,6 +251,7 @@ erl_init(int ncpu) erts_init_monitors(); erts_init_gc(); init_time(); + erts_init_sys_common_misc(); erts_init_process(ncpu); erts_init_scheduling(use_multi_run_queue, no_schedulers, @@ -322,7 +323,7 @@ init_shared_memory(int argc, char **argv) #endif global_gen_gcs = 0; - global_max_gen_gcs = erts_smp_atomic_read(&erts_max_gen_gcs); + global_max_gen_gcs = (Uint16) erts_smp_atomic32_read(&erts_max_gen_gcs); global_gc_flags = erts_default_process_flags; erts_global_offheap.mso = NULL; @@ -650,7 +651,7 @@ early_init(int *argc, char **argv) /* erts_writing_erl_crash_dump = 0; #endif - erts_smp_atomic_init(&erts_max_gen_gcs, (long)((Uint16) -1)); + erts_smp_atomic32_init(&erts_max_gen_gcs, (erts_aint32_t) ((Uint16) -1)); erts_pre_init_process(); #if defined(USE_THREADS) && !defined(ERTS_SMP) @@ -855,7 +856,7 @@ erl_start(int argc, char **argv) envbufsz = sizeof(envbuf); if (erts_sys_getenv("ERL_FULLSWEEP_AFTER", envbuf, &envbufsz) == 0) { Uint16 max_gen_gcs = atoi(envbuf); - erts_smp_atomic_set(&erts_max_gen_gcs, (long) max_gen_gcs); + erts_smp_atomic32_set(&erts_max_gen_gcs, (erts_aint32_t) max_gen_gcs); } envbufsz = sizeof(envbuf); @@ -907,7 +908,27 @@ erl_start(int argc, char **argv) VERBOSE(DEBUG_SYSTEM, ("using display items %d\n",display_items)); break; - + case 'f': + if (!strncmp(argv[i],"-fn",3)) { + arg = get_arg(argv[i]+3, argv[i+1], &i); + switch (*arg) { + case 'u': + erts_set_user_requested_filename_encoding(ERL_FILENAME_UTF8); + break; + case 'l': + erts_set_user_requested_filename_encoding(ERL_FILENAME_LATIN1); + break; + case 'a': + erts_set_user_requested_filename_encoding(ERL_FILENAME_UNKNOWN); + default: + erts_fprintf(stderr, "bad filename encoding %s, can be (l,u or a)\n", arg); + erts_usage(); + } + break; + } else { + erts_fprintf(stderr, "%s unknown flag %s\n", argv[0], argv[i]); + erts_usage(); + } case 'l': display_loads++; break; diff --git a/erts/emulator/beam/erl_lock_check.c b/erts/emulator/beam/erl_lock_check.c index 04c7dbd2ec..7a6aaa6bbe 100644 --- a/erts/emulator/beam/erl_lock_check.c +++ b/erts/emulator/beam/erl_lock_check.c @@ -978,10 +978,10 @@ erts_lc_trylock_force_busy_flg(erts_lc_lock_t *lck, Uint16 op_flags) /* We only force busy if a lock order violation would occur and when on an even millisecond. */ { - erts_thr_timeval_t time; - erts_thr_time_now(&time); + SysTimeval tv; + sys_gettimeofday(&tv); - if ((time.tv_nsec / 1000000) & 1) + if ((tv.tv_usec / 1000) & 1) return 0; } #endif diff --git a/erts/emulator/beam/erl_lock_count.c b/erts/emulator/beam/erl_lock_count.c index 239773f366..a36c53560e 100644 --- a/erts/emulator/beam/erl_lock_count.c +++ b/erts/emulator/beam/erl_lock_count.c @@ -159,7 +159,7 @@ static char* lock_opt(Uint16 flag) { } static void print_lock_x(erts_lcnt_lock_t *lock, Uint16 flag, char *action, char *extra) { - long int colls, tries, w_state, r_state; + erts_aint_t colls, tries, w_state, r_state; erts_lcnt_lock_stats_t *stats = NULL; char *type; @@ -385,7 +385,7 @@ void erts_lcnt_destroy_lock(erts_lcnt_lock_t *lock) { /* lock */ void erts_lcnt_lock_opt(erts_lcnt_lock_t *lock, Uint16 option) { - long r_state = 0, w_state = 0; + erts_aint_t r_state = 0, w_state = 0; erts_lcnt_thread_data_t *eltd; if (erts_lcnt_rt_options & ERTS_LCNT_OPT_SUSPEND) return; @@ -418,7 +418,7 @@ void erts_lcnt_lock_opt(erts_lcnt_lock_t *lock, Uint16 option) { } void erts_lcnt_lock(erts_lcnt_lock_t *lock) { - long w_state; + erts_aint_t w_state; erts_lcnt_thread_data_t *eltd; if (erts_lcnt_rt_options & ERTS_LCNT_OPT_SUSPEND) return; @@ -471,7 +471,7 @@ void erts_lcnt_lock_post_x(erts_lcnt_lock_t *lock, char *file, unsigned int line erts_lcnt_time_t time_wait; erts_lcnt_lock_stats_t *stats; #ifdef DEBUG - long flowstate; + erts_aint_t flowstate; #endif if (erts_lcnt_rt_options & ERTS_LCNT_OPT_SUSPEND) return; @@ -516,8 +516,8 @@ void erts_lcnt_unlock_opt(erts_lcnt_lock_t *lock, Uint16 option) { void erts_lcnt_unlock(erts_lcnt_lock_t *lock) { #ifdef DEBUG - long w_state; - long flowstate; + erts_aint_t w_state; + erts_aint_t flowstate; #endif if (erts_lcnt_rt_options & ERTS_LCNT_OPT_SUSPEND) return; #ifdef DEBUG @@ -552,7 +552,7 @@ void erts_lcnt_trylock_opt(erts_lcnt_lock_t *lock, int res, Uint16 option) { void erts_lcnt_trylock(erts_lcnt_lock_t *lock, int res) { /* Determine lock_state via res instead of state */ #ifdef DEBUG - long flowstate; + erts_aint_t flowstate; #endif if (erts_lcnt_rt_options & ERTS_LCNT_OPT_SUSPEND) return; if (res != EBUSY) { diff --git a/erts/emulator/beam/erl_node_tables.c b/erts/emulator/beam/erl_node_tables.c index 8cdda395df..6daa127d23 100644 --- a/erts/emulator/beam/erl_node_tables.c +++ b/erts/emulator/beam/erl_node_tables.c @@ -235,7 +235,7 @@ erts_sysname_to_connected_dist_entry(Eterm sysname) erts_smp_rwmtx_rlock(&erts_dist_table_rwmtx); res_dep = (DistEntry *) hash_get(&erts_dist_table, (void *) &de); if (res_dep) { - long refc = erts_refc_inctest(&res_dep->refc, 1); + erts_aint_t refc = erts_refc_inctest(&res_dep->refc, 1); if (refc < 2) /* Pending delete */ erts_refc_inc(&res_dep->refc, 1); } @@ -257,7 +257,7 @@ DistEntry *erts_find_or_insert_dist_entry(Eterm sysname) { DistEntry *res; DistEntry de; - long refc; + erts_aint_t refc; res = erts_find_dist_entry(sysname); if (res) return res; @@ -279,7 +279,7 @@ DistEntry *erts_find_dist_entry(Eterm sysname) erts_smp_rwmtx_rlock(&erts_dist_table_rwmtx); res = hash_get(&erts_dist_table, (void *) &de); if (res) { - long refc = erts_refc_inctest(&res->refc, 1); + erts_aint_t refc = erts_refc_inctest(&res->refc, 1); if (refc < 2) /* Pending delete */ erts_refc_inc(&res->refc, 1); } @@ -586,7 +586,7 @@ ErlNode *erts_find_or_insert_node(Eterm sysname, Uint creation) erts_smp_rwmtx_rlock(&erts_node_table_rwmtx); res = hash_get(&erts_node_table, (void *) &ne); if (res && res != erts_this_node) { - long refc = erts_refc_inctest(&res->refc, 0); + erts_aint_t refc = erts_refc_inctest(&res->refc, 0); if (refc < 2) /* New or pending delete */ erts_refc_inc(&res->refc, 1); } @@ -598,7 +598,7 @@ ErlNode *erts_find_or_insert_node(Eterm sysname, Uint creation) res = hash_put(&erts_node_table, (void *) &ne); ASSERT(res); if (res != erts_this_node) { - long refc = erts_refc_inctest(&res->refc, 0); + erts_aint_t refc = erts_refc_inctest(&res->refc, 0); if (refc < 2) /* New or pending delete */ erts_refc_inc(&res->refc, 1); } diff --git a/erts/emulator/beam/erl_port_task.c b/erts/emulator/beam/erl_port_task.c index c10724b951..1b07024ca1 100644 --- a/erts/emulator/beam/erl_port_task.c +++ b/erts/emulator/beam/erl_port_task.c @@ -129,7 +129,7 @@ reset_handle(ErtsPortTask *ptp) { if (ptp->handle) { ASSERT(ptp == handle2task(ptp->handle)); - erts_smp_atomic_set(ptp->handle, (long) NULL); + erts_smp_atomic_set(ptp->handle, (erts_aint_t) NULL); } } @@ -138,7 +138,7 @@ set_handle(ErtsPortTask *ptp, ErtsPortTaskHandle *pthp) { ptp->handle = pthp; if (pthp) { - erts_smp_atomic_set(pthp, (long) ptp); + erts_smp_atomic_set(pthp, (erts_aint_t) ptp); ASSERT(ptp == handle2task(ptp->handle)); } } @@ -568,7 +568,7 @@ erts_port_task_schedule(Eterm id, ErtsRunQueue *xrunq = erts_check_emigration_need(runq, ERTS_PORT_PRIO_LEVEL); if (xrunq) { /* Port emigrated ... */ - erts_smp_atomic_set(&pp->run_queue, (long) xrunq); + erts_smp_atomic_set(&pp->run_queue, (erts_aint_t) xrunq); erts_smp_runq_unlock(runq); runq = xrunq; } @@ -727,7 +727,8 @@ resume_after_block(void *vd) ErtsPortTaskExeBlockData *d = (ErtsPortTaskExeBlockData *) vd; erts_smp_runq_lock(d->runq); if (d->resp) - *d->resp = erts_smp_atomic_read(&erts_port_task_outstanding_io_tasks) != (long) 0; + *d->resp = (erts_smp_atomic_read(&erts_port_task_outstanding_io_tasks) + != (erts_aint_t) 0); } /* @@ -748,7 +749,7 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) ErtsPortTask *ptp; int res = 0; int reds = ERTS_PORT_REDS_EXECUTE; - long io_tasks_executed = 0; + erts_aint_t io_tasks_executed = 0; int fpe_was_unmasked; ErtsPortTaskExeBlockData blk_data = {runq, NULL}; @@ -942,7 +943,7 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) } else { /* Port emigrated ... */ - erts_smp_atomic_set(&pp->run_queue, (long) xrunq); + erts_smp_atomic_set(&pp->run_queue, (erts_aint_t) xrunq); enqueue_port(xrunq, pp); ASSERT(pp->sched.exe_taskq); pp->sched.exe_taskq = NULL; @@ -953,7 +954,8 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) port_was_enqueued = 1; } - res = erts_smp_atomic_read(&erts_port_task_outstanding_io_tasks) != (long) 0; + res = (erts_smp_atomic_read(&erts_port_task_outstanding_io_tasks) + != (erts_aint_t) 0); ERTS_PT_CHK_PRES_PORTQ(runq, pp); @@ -971,7 +973,7 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) erts_port_release(pp); #else { - long refc; + erts_aint_t refc; erts_smp_mtx_unlock(pp->lock); refc = erts_smp_atomic_dectest(&pp->refc); ASSERT(refc >= 0); @@ -979,7 +981,8 @@ erts_port_task_execute(ErtsRunQueue *runq, Port **curr_port_pp) erts_smp_runq_unlock(runq); erts_port_cleanup(pp); /* Might aquire runq lock */ erts_smp_runq_lock(runq); - res = erts_smp_atomic_read(&erts_port_task_outstanding_io_tasks) != (long) 0; + res = (erts_smp_atomic_read(&erts_port_task_outstanding_io_tasks) + != (erts_aint_t) 0); } } #endif @@ -1112,7 +1115,7 @@ erts_port_migrate(Port *prt, int *prt_locked, if (!ERTS_PORT_IS_IN_RUNQ(from_rq, prt)) return ERTS_MIGRATE_FAILED_NOT_IN_RUNQ; dequeue_port(from_rq, prt); - erts_smp_atomic_set(&prt->run_queue, (long) to_rq); + erts_smp_atomic_set(&prt->run_queue, (erts_aint_t) to_rq); enqueue_port(to_rq, prt); return ERTS_MIGRATE_SUCCESS; } @@ -1125,7 +1128,7 @@ erts_port_migrate(Port *prt, int *prt_locked, void erts_port_task_init(void) { - erts_smp_atomic_init(&erts_port_task_outstanding_io_tasks, (long) 0); + erts_smp_atomic_init(&erts_port_task_outstanding_io_tasks, (erts_aint_t) 0); init_port_task_alloc(); init_port_taskq_alloc(); } diff --git a/erts/emulator/beam/erl_port_task.h b/erts/emulator/beam/erl_port_task.h index f12d02da0c..714b4ea7dd 100644 --- a/erts/emulator/beam/erl_port_task.h +++ b/erts/emulator/beam/erl_port_task.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2006-2009. All Rights Reserved. + * Copyright Ericsson AB 2006-2010. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -79,7 +79,7 @@ ERTS_GLB_INLINE int erts_port_task_have_outstanding_io_tasks(void); ERTS_GLB_INLINE void erts_port_task_handle_init(ErtsPortTaskHandle *pthp) { - erts_smp_atomic_init(pthp, (long) NULL); + erts_smp_atomic_init(pthp, (erts_aint_t) NULL); } ERTS_GLB_INLINE int diff --git a/erts/emulator/beam/erl_process.c b/erts/emulator/beam/erl_process.c index f252c2cbe2..77ee1d6ac5 100644 --- a/erts/emulator/beam/erl_process.c +++ b/erts/emulator/beam/erl_process.c @@ -127,21 +127,22 @@ ErtsLcPSDLocks erts_psd_required_locks[ERTS_PSD_SIZE]; int erts_disable_proc_not_running_opt; -#define ERTS_SCHDLR_SSPND_CHNG_WAITER (((long) 1) << 0) -#define ERTS_SCHDLR_SSPND_CHNG_MSB (((long) 1) << 1) -#define ERTS_SCHDLR_SSPND_CHNG_ONLN (((long) 1) << 2) +#define ERTS_SCHDLR_SSPND_CHNG_WAITER (((erts_aint32_t) 1) << 0) +#define ERTS_SCHDLR_SSPND_CHNG_MSB (((erts_aint32_t) 1) << 1) +#define ERTS_SCHDLR_SSPND_CHNG_ONLN (((erts_aint32_t) 1) << 2) #ifndef DEBUG #define ERTS_SCHDLR_SSPND_CHNG_SET(VAL, OLD_VAL) \ - erts_smp_atomic_set(&schdlr_sspnd.changing, (VAL)) + erts_smp_atomic32_set(&schdlr_sspnd.changing, (VAL)) #else #define ERTS_SCHDLR_SSPND_CHNG_SET(VAL, OLD_VAL) \ do { \ - long old_val__ = erts_smp_atomic_xchg(&schdlr_sspnd.changing, \ - (VAL)); \ + erts_aint32_t old_val__; \ + old_val__ = erts_smp_atomic32_xchg(&schdlr_sspnd.changing, \ + (VAL)); \ ASSERT(old_val__ == (OLD_VAL)); \ } while (0) @@ -154,10 +155,10 @@ static struct { int online; int curr_online; int wait_curr_online; - erts_smp_atomic_t changing; - erts_smp_atomic_t active; + erts_smp_atomic32_t changing; + erts_smp_atomic32_t active; struct { - erts_smp_atomic_t ongoing; + erts_smp_atomic32_t ongoing; long wait_active; ErtsProcList *procs; } msb; /* Multi Scheduling Block */ @@ -165,11 +166,11 @@ static struct { static struct { erts_smp_mtx_t update_mtx; - erts_smp_atomic_t active_runqs; + erts_smp_atomic32_t active_runqs; int last_active_runqs; - erts_smp_atomic_t used_runqs; + erts_smp_atomic32_t used_runqs; int forced_check_balance; - erts_smp_atomic_t checking_balance; + erts_smp_atomic32_t checking_balance; int halftime; int full_reds_history_index; struct { @@ -199,11 +200,11 @@ static erts_tsd_key_t sched_data_key; static erts_smp_mtx_t proc_tab_mtx; -static erts_smp_atomic_t function_calls; +static erts_smp_atomic32_t function_calls; #ifdef ERTS_SMP -static erts_smp_atomic_t doing_sys_schedule; -static erts_smp_atomic_t no_empty_run_queues; +static erts_smp_atomic32_t doing_sys_schedule; +static erts_smp_atomic32_t no_empty_run_queues; #else /* !ERTS_SMP */ ErtsSchedulerData *erts_scheduler_data; #endif @@ -247,7 +248,10 @@ Uint erts_num_active_procs; Process** erts_active_procs; #endif -static erts_smp_atomic_t process_count; +#if ERTS_MAX_PROCESSES > 0x7fffffff +#error "Need to store process_count in another type" +#endif +static erts_smp_atomic32_t process_count; typedef struct ErtsTermProcElement_ ErtsTermProcElement; struct ErtsTermProcElement_ { @@ -407,7 +411,7 @@ erts_init_process(int ncpu) init_proclist_alloc(); - erts_smp_atomic_init(&process_count, 0); + erts_smp_atomic32_init(&process_count, 0); if (erts_use_r9_pids_ports) { proc_bits = ERTS_R9_PROC_BITS; @@ -568,7 +572,7 @@ erts_psd_set_init(Process *p, ErtsProcLocks plocks, int ix, void *data) #ifdef ERTS_SMP void -erts_sched_finish_poke(ErtsSchedulerSleepInfo *ssi, long flags) +erts_sched_finish_poke(ErtsSchedulerSleepInfo *ssi, erts_aint32_t flags) { switch (flags & ERTS_SSI_FLGS_SLEEP_TYPE) { case ERTS_SSI_FLG_POLL_SLEEPING: @@ -593,11 +597,11 @@ erts_smp_notify_check_children_needed(void) int i; for (i = 0; i < erts_no_schedulers; i++) { - long aux_work; + erts_aint32_t aux_work; ErtsSchedulerSleepInfo *ssi; ssi = ERTS_SCHED_SLEEP_INFO_IX(i); - aux_work = erts_smp_atomic_bor(&ssi->aux_work, - ERTS_SSI_AUX_WORK_CHECK_CHILDREN); + aux_work = erts_smp_atomic32_bor(&ssi->aux_work, + ERTS_SSI_AUX_WORK_CHECK_CHILDREN); if (!(aux_work & ERTS_SSI_AUX_WORK_CHECK_CHILDREN)) erts_sched_poke(ssi); } @@ -605,16 +609,16 @@ erts_smp_notify_check_children_needed(void) #endif #ifdef ERTS_SCHED_NEED_BLOCKABLE_AUX_WORK -static ERTS_INLINE long +static ERTS_INLINE erts_aint32_t blockable_aux_work(ErtsSchedulerData *esdp, ErtsSchedulerSleepInfo *ssi, - long aux_work) + erts_aint32_t aux_work) { if (aux_work & ERTS_SSI_BLOCKABLE_AUX_WORK_MASK) { #ifdef ERTS_SMP_SCHEDULERS_NEED_TO_CHECK_CHILDREN if (aux_work & ERTS_SSI_AUX_WORK_CHECK_CHILDREN) { - aux_work = erts_smp_atomic_band(&ssi->aux_work, - ~ERTS_SSI_AUX_WORK_CHECK_CHILDREN); + aux_work = erts_smp_atomic32_band(&ssi->aux_work, + ~ERTS_SSI_AUX_WORK_CHECK_CHILDREN); aux_work &= ~ERTS_SSI_AUX_WORK_CHECK_CHILDREN; erts_check_children(); } @@ -626,10 +630,10 @@ blockable_aux_work(ErtsSchedulerData *esdp, #endif #ifdef ERTS_SCHED_NEED_NONBLOCKABLE_AUX_WORK -static ERTS_INLINE long +static ERTS_INLINE erts_aint32_t nonblockable_aux_work(ErtsSchedulerData *esdp, ErtsSchedulerSleepInfo *ssi, - long aux_work) + erts_aint32_t aux_work) { if (aux_work & ERTS_SSI_NONBLOCKABLE_AUX_WORK_MASK) { @@ -694,10 +698,10 @@ prepare_for_sys_schedule(void) { #ifdef ERTS_SMP while (!erts_port_task_have_outstanding_io_tasks() - && !erts_smp_atomic_xchg(&doing_sys_schedule, 1)) { + && !erts_smp_atomic32_xchg(&doing_sys_schedule, 1)) { if (!erts_port_task_have_outstanding_io_tasks()) return 1; - erts_smp_atomic_set(&doing_sys_schedule, 0); + erts_smp_atomic32_set(&doing_sys_schedule, 0); } return 0; #else @@ -745,53 +749,55 @@ sched_active(Uint no, ErtsRunQueue *rq) static int ERTS_INLINE ongoing_multi_scheduling_block(void) { - return erts_smp_atomic_read(&schdlr_sspnd.msb.ongoing) != 0; + return erts_smp_atomic32_read(&schdlr_sspnd.msb.ongoing) != 0; } static ERTS_INLINE void empty_runq(ErtsRunQueue *rq) { - long oifls = erts_smp_atomic_band(&rq->info_flags, ~ERTS_RUNQ_IFLG_NONEMPTY); + erts_aint32_t oifls = erts_smp_atomic32_band(&rq->info_flags, + ~ERTS_RUNQ_IFLG_NONEMPTY); if (oifls & ERTS_RUNQ_IFLG_NONEMPTY) { #ifdef DEBUG - long empty = erts_smp_atomic_read(&no_empty_run_queues); + erts_aint32_t empty = erts_smp_atomic32_read(&no_empty_run_queues); /* * For a short period of time no_empty_run_queues may have * been increased twice for a specific run queue. */ ASSERT(0 <= empty && empty < 2*erts_no_run_queues); #endif - erts_smp_atomic_inc(&no_empty_run_queues); + erts_smp_atomic32_inc(&no_empty_run_queues); } } static ERTS_INLINE void non_empty_runq(ErtsRunQueue *rq) { - long oifls = erts_smp_atomic_bor(&rq->info_flags, ERTS_RUNQ_IFLG_NONEMPTY); + erts_aint32_t oifls = erts_smp_atomic32_bor(&rq->info_flags, + ERTS_RUNQ_IFLG_NONEMPTY); if (!(oifls & ERTS_RUNQ_IFLG_NONEMPTY)) { #ifdef DEBUG - long empty = erts_smp_atomic_read(&no_empty_run_queues); + erts_aint32_t empty = erts_smp_atomic32_read(&no_empty_run_queues); /* * For a short period of time no_empty_run_queues may have * been increased twice for a specific run queue. */ ASSERT(0 < empty && empty <= 2*erts_no_run_queues); #endif - erts_smp_atomic_dec(&no_empty_run_queues); + erts_smp_atomic32_dec(&no_empty_run_queues); } } -static long +static erts_aint32_t sched_prep_spin_wait(ErtsSchedulerSleepInfo *ssi) { - long oflgs; - long nflgs = (ERTS_SSI_FLG_SLEEPING - | ERTS_SSI_FLG_WAITING); - long xflgs = 0; + erts_aint32_t oflgs; + erts_aint32_t nflgs = (ERTS_SSI_FLG_SLEEPING + | ERTS_SSI_FLG_WAITING); + erts_aint32_t xflgs = 0; do { - oflgs = erts_smp_atomic_cmpxchg(&ssi->flags, nflgs, xflgs); + oflgs = erts_smp_atomic32_cmpxchg(&ssi->flags, nflgs, xflgs); if (oflgs == xflgs) return nflgs; xflgs = oflgs; @@ -799,16 +805,16 @@ sched_prep_spin_wait(ErtsSchedulerSleepInfo *ssi) return oflgs; } -static long +static erts_aint32_t sched_prep_cont_spin_wait(ErtsSchedulerSleepInfo *ssi) { - long oflgs; - long nflgs = (ERTS_SSI_FLG_SLEEPING - | ERTS_SSI_FLG_WAITING); - long xflgs = ERTS_SSI_FLG_WAITING; + erts_aint32_t oflgs; + erts_aint32_t nflgs = (ERTS_SSI_FLG_SLEEPING + | ERTS_SSI_FLG_WAITING); + erts_aint32_t xflgs = ERTS_SSI_FLG_WAITING; do { - oflgs = erts_smp_atomic_cmpxchg(&ssi->flags, nflgs, xflgs); + oflgs = erts_smp_atomic32_cmpxchg(&ssi->flags, nflgs, xflgs); if (oflgs == xflgs) return nflgs; xflgs = oflgs; @@ -817,15 +823,15 @@ sched_prep_cont_spin_wait(ErtsSchedulerSleepInfo *ssi) return oflgs; } -static long +static erts_aint32_t sched_spin_wait(ErtsSchedulerSleepInfo *ssi, int spincount) { - long until_yield = ERTS_SCHED_SPIN_UNTIL_YIELD; + int until_yield = ERTS_SCHED_SPIN_UNTIL_YIELD; int sc = spincount; - long flgs; + erts_aint32_t flgs; do { - flgs = erts_smp_atomic_read(&ssi->flags); + flgs = erts_smp_atomic32_read(&ssi->flags); if ((flgs & (ERTS_SSI_FLG_SLEEPING|ERTS_SSI_FLG_WAITING)) != (ERTS_SSI_FLG_SLEEPING|ERTS_SSI_FLG_WAITING)) { break; @@ -839,18 +845,18 @@ sched_spin_wait(ErtsSchedulerSleepInfo *ssi, int spincount) return flgs; } -static long -sched_set_sleeptype(ErtsSchedulerSleepInfo *ssi, long sleep_type) +static erts_aint32_t +sched_set_sleeptype(ErtsSchedulerSleepInfo *ssi, erts_aint32_t sleep_type) { - long oflgs; - long nflgs = ERTS_SSI_FLG_SLEEPING|ERTS_SSI_FLG_WAITING|sleep_type; - long xflgs = ERTS_SSI_FLG_SLEEPING|ERTS_SSI_FLG_WAITING; + erts_aint32_t oflgs; + erts_aint32_t nflgs = ERTS_SSI_FLG_SLEEPING|ERTS_SSI_FLG_WAITING|sleep_type; + erts_aint32_t xflgs = ERTS_SSI_FLG_SLEEPING|ERTS_SSI_FLG_WAITING; if (sleep_type == ERTS_SSI_FLG_TSE_SLEEPING) erts_tse_reset(ssi->event); while (1) { - oflgs = erts_smp_atomic_cmpxchg(&ssi->flags, nflgs, xflgs); + oflgs = erts_smp_atomic32_cmpxchg(&ssi->flags, nflgs, xflgs); if (oflgs == xflgs) return nflgs; if ((oflgs & (ERTS_SSI_FLG_SLEEPING|ERTS_SSI_FLG_WAITING)) @@ -867,14 +873,14 @@ sched_set_sleeptype(ErtsSchedulerSleepInfo *ssi, long sleep_type) != ERTS_SSI_FLG_WAITING) static void -scheduler_wait(long *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) +scheduler_wait(int *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) { ErtsSchedulerSleepInfo *ssi = esdp->ssi; int spincount; - long flgs; + erts_aint32_t flgs; #if defined(ERTS_SCHED_NEED_NONBLOCKABLE_AUX_WORK) \ || defined(ERTS_SCHED_NEED_BLOCKABLE_AUX_WORK) - long aux_work; + erts_aint32_t aux_work; #endif ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(rq)); @@ -910,7 +916,7 @@ scheduler_wait(long *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) tse_wait: #ifdef ERTS_SCHED_NEED_BLOCKABLE_AUX_WORK - aux_work = erts_smp_atomic_read(&ssi->aux_work); + aux_work = erts_smp_atomic32_read(&ssi->aux_work); tse_blockable_aux_work: aux_work = blockable_aux_work(esdp, ssi, aux_work); #endif @@ -920,7 +926,7 @@ scheduler_wait(long *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) #ifdef ERTS_SCHED_NEED_NONBLOCKABLE_AUX_WORK #ifndef ERTS_SCHED_NEED_BLOCKABLE_AUX_WORK - aux_work = erts_smp_atomic_read(&ssi->aux_work); + aux_work = erts_smp_atomic32_read(&ssi->aux_work); #endif nonblockable_aux_work(esdp, ssi, aux_work); #endif @@ -953,7 +959,7 @@ scheduler_wait(long *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) } #ifdef ERTS_SCHED_NEED_BLOCKABLE_AUX_WORK - aux_work = erts_smp_atomic_read(&ssi->aux_work); + aux_work = erts_smp_atomic32_read(&ssi->aux_work); if (aux_work & ERTS_SSI_BLOCKABLE_AUX_WORK_MASK) { erts_smp_activity_end(ERTS_ACTIVITY_WAIT, NULL, NULL, NULL); goto tse_blockable_aux_work; @@ -965,16 +971,16 @@ scheduler_wait(long *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) erts_smp_activity_end(ERTS_ACTIVITY_WAIT, NULL, NULL, NULL); if (flgs & ~ERTS_SSI_FLG_SUSPENDED) - erts_smp_atomic_band(&ssi->flags, ERTS_SSI_FLG_SUSPENDED); + erts_smp_atomic32_band(&ssi->flags, ERTS_SSI_FLG_SUSPENDED); erts_smp_runq_lock(rq); sched_active(esdp->no, rq); } else { - long dt; + erts_aint_t dt; - erts_smp_atomic_set(&function_calls, 0); + erts_smp_atomic32_set(&function_calls, 0); *fcalls = 0; sched_waiting_sys(esdp->no, rq); @@ -997,17 +1003,17 @@ scheduler_wait(long *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) sys_aux_work: #ifdef ERTS_SCHED_NEED_BLOCKABLE_AUX_WORK - aux_work = erts_smp_atomic_read(&ssi->aux_work); + aux_work = erts_smp_atomic32_read(&ssi->aux_work); aux_work = blockable_aux_work(esdp, ssi, aux_work); #endif #ifdef ERTS_SCHED_NEED_NONBLOCKABLE_AUX_WORK #ifndef ERTS_SCHED_NEED_BLOCKABLE_AUX_WORK - aux_work = erts_smp_atomic_read(&ssi->aux_work); + aux_work = erts_smp_atomic32_read(&ssi->aux_work); #endif nonblockable_aux_work(esdp, ssi, aux_work); #endif - flgs = erts_smp_atomic_read(&ssi->flags); + flgs = erts_smp_atomic32_read(&ssi->flags); if (!(flgs & ERTS_SSI_FLG_WAITING)) { ASSERT(!(flgs & ERTS_SSI_FLG_SLEEPING)); goto sys_woken; @@ -1025,7 +1031,7 @@ scheduler_wait(long *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) * call erl_sys_schedule() until it is handled. */ if (erts_port_task_have_outstanding_io_tasks()) { - erts_smp_atomic_set(&doing_sys_schedule, 0); + erts_smp_atomic32_set(&doing_sys_schedule, 0); /* * Got to check that we still got I/O tasks; otherwise * we have to continue checking for I/O... @@ -1044,7 +1050,7 @@ scheduler_wait(long *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) * sleep in erl_sys_schedule(). */ if (erts_port_task_have_outstanding_io_tasks()) { - erts_smp_atomic_set(&doing_sys_schedule, 0); + erts_smp_atomic32_set(&doing_sys_schedule, 0); /* * Got to check that we still got I/O tasks; otherwise @@ -1098,9 +1104,9 @@ scheduler_wait(long *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) sys_woken: erts_smp_runq_lock(rq); sys_locked_woken: - erts_smp_atomic_set(&doing_sys_schedule, 0); + erts_smp_atomic32_set(&doing_sys_schedule, 0); if (flgs & ~ERTS_SSI_FLG_SUSPENDED) - erts_smp_atomic_band(&ssi->flags, ERTS_SSI_FLG_SUSPENDED); + erts_smp_atomic32_band(&ssi->flags, ERTS_SSI_FLG_SUSPENDED); sched_active_sys(esdp->no, rq); } } @@ -1108,15 +1114,15 @@ scheduler_wait(long *fcalls, ErtsSchedulerData *esdp, ErtsRunQueue *rq) ERTS_SMP_LC_ASSERT(erts_smp_lc_runq_is_locked(rq)); } -static ERTS_INLINE long +static ERTS_INLINE erts_aint32_t ssi_flags_set_wake(ErtsSchedulerSleepInfo *ssi) { /* reset all flags but suspended */ - long oflgs; - long nflgs = 0; - long xflgs = ERTS_SSI_FLG_SLEEPING|ERTS_SSI_FLG_WAITING; + erts_aint32_t oflgs; + erts_aint32_t nflgs = 0; + erts_aint32_t xflgs = ERTS_SSI_FLG_SLEEPING|ERTS_SSI_FLG_WAITING; while (1) { - oflgs = erts_smp_atomic_cmpxchg(&ssi->flags, nflgs, xflgs); + oflgs = erts_smp_atomic32_cmpxchg(&ssi->flags, nflgs, xflgs); if (oflgs == xflgs) return oflgs; nflgs = oflgs & ERTS_SSI_FLG_SUSPENDED; @@ -1148,7 +1154,7 @@ wake_scheduler(ErtsRunQueue *rq, int incq, int one) if (!ssi) erts_smp_spin_unlock(&sl->lock); else if (one) { - long flgs; + erts_aint32_t flgs; if (ssi->prev) ssi->prev->next = ssi->next; else { @@ -1195,15 +1201,17 @@ wake_all_schedulers(void) static ERTS_INLINE int chk_wake_sched(ErtsRunQueue *crq, int ix, int activate) { - long iflgs; + erts_aint32_t iflgs; ErtsRunQueue *wrq; if (crq->ix == ix) return 0; wrq = ERTS_RUNQ_IX(ix); - iflgs = erts_smp_atomic_read(&wrq->info_flags); + iflgs = erts_smp_atomic32_read(&wrq->info_flags); if (!(iflgs & (ERTS_RUNQ_IFLG_SUSPENDED|ERTS_RUNQ_IFLG_NONEMPTY))) { if (activate) { - if (ix == erts_smp_atomic_cmpxchg(&balance_info.active_runqs, ix+1, ix)) { + if (ix == erts_smp_atomic32_cmpxchg(&balance_info.active_runqs, + ix+1, + ix)) { erts_smp_xrunq_lock(crq, wrq); wrq->flags &= ~ERTS_RUNQ_FLG_INACTIVE; erts_smp_xrunq_unlock(crq, wrq); @@ -1220,8 +1228,8 @@ wake_scheduler_on_empty_runq(ErtsRunQueue *crq) { int ix = crq->ix; int stop_ix = ix; - int active_ix = erts_smp_atomic_read(&balance_info.active_runqs); - int balance_ix = erts_smp_atomic_read(&balance_info.used_runqs); + int active_ix = erts_smp_atomic32_read(&balance_info.active_runqs); + int balance_ix = erts_smp_atomic32_read(&balance_info.used_runqs); if (active_ix > balance_ix) active_ix = balance_ix; @@ -1273,7 +1281,7 @@ erts_sched_notify_check_cpu_bind(void) int ix; if (erts_common_run_queue) { for (ix = 0; ix < erts_no_schedulers; ix++) - erts_smp_atomic_set(&ERTS_SCHEDULER_IX(ix)->chk_cpu_bind, 1); + erts_smp_atomic32_set(&ERTS_SCHEDULER_IX(ix)->chk_cpu_bind, 1); wake_all_schedulers(); } else { @@ -1441,14 +1449,15 @@ evacuate_run_queue(ErtsRunQueue *evac_rq, ErtsRunQueue *rq) erts_smp_runq_lock(evac_rq); - erts_smp_atomic_bor(&evac_rq->scheduler->ssi->flags, ERTS_SSI_FLG_SUSPENDED); + erts_smp_atomic32_bor(&evac_rq->scheduler->ssi->flags, + ERTS_SSI_FLG_SUSPENDED); evac_rq->flags &= ~ERTS_RUNQ_FLGS_IMMIGRATE_QMASK; evac_rq->flags |= (ERTS_RUNQ_FLGS_EMIGRATE_QMASK | ERTS_RUNQ_FLGS_EVACUATE_QMASK | ERTS_RUNQ_FLG_SUSPENDED); - erts_smp_atomic_bor(&evac_rq->info_flags, ERTS_RUNQ_IFLG_SUSPENDED); + erts_smp_atomic32_bor(&evac_rq->info_flags, ERTS_RUNQ_IFLG_SUSPENDED); /* * Need to set up evacuation paths first since we * may release the run queue lock on evac_rq @@ -1697,7 +1706,7 @@ static ERTS_INLINE int check_possible_steal_victim(ErtsRunQueue *rq, int *rq_lockedp, int vix) { ErtsRunQueue *vrq = ERTS_RUNQ_IX(vix); - long iflgs = erts_smp_atomic_read(&vrq->info_flags); + erts_aint32_t iflgs = erts_smp_atomic32_read(&vrq->info_flags); if (iflgs & ERTS_RUNQ_IFLG_NONEMPTY) return try_steal_task_from_victim(rq, rq_lockedp, vrq); else @@ -1727,8 +1736,8 @@ try_steal_task(ErtsRunQueue *rq) ERTS_SMP_LC_CHK_RUNQ_LOCK(rq, rq_locked); - active_rqs = erts_smp_atomic_read(&balance_info.active_runqs); - blnc_rqs = erts_smp_atomic_read(&balance_info.used_runqs); + active_rqs = erts_smp_atomic32_read(&balance_info.active_runqs); + blnc_rqs = erts_smp_atomic32_read(&balance_info.used_runqs); if (active_rqs > blnc_rqs) active_rqs = blnc_rqs; @@ -1739,7 +1748,7 @@ try_steal_task(ErtsRunQueue *rq) if (active_rqs < blnc_rqs) { int no = blnc_rqs - active_rqs; int stop_ix = vix = active_rqs + rq->ix % no; - while (erts_smp_atomic_read(&no_empty_run_queues) < blnc_rqs) { + while (erts_smp_atomic32_read(&no_empty_run_queues) < blnc_rqs) { res = check_possible_steal_victim(rq, &rq_locked, vix); if (res) goto done; @@ -1754,7 +1763,7 @@ try_steal_task(ErtsRunQueue *rq) vix = rq->ix; /* ... then try to steal a job from another active queue... */ - while (erts_smp_atomic_read(&no_empty_run_queues) < blnc_rqs) { + while (erts_smp_atomic32_read(&no_empty_run_queues) < blnc_rqs) { vix++; if (vix >= active_rqs) vix = 0; @@ -1842,20 +1851,23 @@ do { \ static void check_balance(ErtsRunQueue *c_rq) { +#if ERTS_MAX_PROCESSES >= (1 << 27) +# error check_balance() assumes ERTS_MAX_PROCESS < (1 << 27) +#endif ErtsRunQueueBalance avg = {0}; Sint64 scheds_reds, full_scheds_reds; int forced, active, current_active, oowc, half_full_scheds, full_scheds, mmax_len, blnc_no_rqs, qix, pix, freds_hist_ix; - if (erts_smp_atomic_xchg(&balance_info.checking_balance, 1)) { + if (erts_smp_atomic32_xchg(&balance_info.checking_balance, 1)) { c_rq->check_balance_reds = INT_MAX; return; } - blnc_no_rqs = (int) erts_smp_atomic_read(&balance_info.used_runqs); + blnc_no_rqs = (int) erts_smp_atomic32_read(&balance_info.used_runqs); if (blnc_no_rqs == 1) { c_rq->check_balance_reds = INT_MAX; - erts_smp_atomic_set(&balance_info.checking_balance, 0); + erts_smp_atomic32_set(&balance_info.checking_balance, 0); return; } @@ -1863,7 +1875,7 @@ check_balance(ErtsRunQueue *c_rq) if (balance_info.halftime) { balance_info.halftime = 0; - erts_smp_atomic_set(&balance_info.checking_balance, 0); + erts_smp_atomic32_set(&balance_info.checking_balance, 0); ERTS_FOREACH_RUNQ(rq, { if (rq->waiting) @@ -1891,12 +1903,12 @@ check_balance(ErtsRunQueue *c_rq) forced = balance_info.forced_check_balance; balance_info.forced_check_balance = 0; - blnc_no_rqs = (int) erts_smp_atomic_read(&balance_info.used_runqs); + blnc_no_rqs = (int) erts_smp_atomic32_read(&balance_info.used_runqs); if (blnc_no_rqs == 1) { erts_smp_mtx_unlock(&balance_info.update_mtx); erts_smp_runq_lock(c_rq); c_rq->check_balance_reds = INT_MAX; - erts_smp_atomic_set(&balance_info.checking_balance, 0); + erts_smp_atomic32_set(&balance_info.checking_balance, 0); return; } @@ -1905,7 +1917,7 @@ check_balance(ErtsRunQueue *c_rq) if (balance_info.full_reds_history_index >= ERTS_FULL_REDS_HISTORY_SIZE) balance_info.full_reds_history_index = 0; - current_active = erts_smp_atomic_read(&balance_info.active_runqs); + current_active = erts_smp_atomic32_read(&balance_info.active_runqs); /* Read balance information for all run queues */ for (qix = 0; qix < blnc_no_rqs; qix++) { @@ -1965,12 +1977,14 @@ check_balance(ErtsRunQueue *c_rq) run_queue_info[qix].prio[pix].avail = 0; } else { - int xreds = 0; - int procreds = treds; - procreds -= run_queue_info[qix].prio[ERTS_PORT_PRIO_LEVEL].reds; + Sint64 xreds = 0; + Sint64 procreds = treds; + procreds -= + ((Sint64) + run_queue_info[qix].prio[ERTS_PORT_PRIO_LEVEL].reds); for (pix = 0; pix < ERTS_NO_PROC_PRIO_LEVELS; pix++) { - int av; + Sint64 av; if (xreds == 0) av = 100; @@ -1981,9 +1995,10 @@ check_balance(ErtsRunQueue *c_rq) if (av == 0) av = 1; } - run_queue_info[qix].prio[pix].avail = av; + run_queue_info[qix].prio[pix].avail = (int) av; + ASSERT(run_queue_info[qix].prio[pix].avail >= 0); if (pix < PRIORITY_NORMAL) /* ie., max or high */ - xreds += run_queue_info[qix].prio[pix].reds; + xreds += (Sint64) run_queue_info[qix].prio[pix].reds; } run_queue_info[qix].prio[ERTS_PORT_PRIO_LEVEL].avail = 100; } @@ -2088,7 +2103,8 @@ check_balance(ErtsRunQueue *c_rq) if (max_len != 0) { int avail = avg.prio[pix].avail; if (avail != 0) { - max_len = ((100*max_len - 1) / avail) + 1; + max_len = (int) ((100*((Sint64) max_len) - 1) + / ((Sint64) avail)) + 1; avg.prio[pix].max_len = max_len; ASSERT(max_len >= 0); } @@ -2105,9 +2121,10 @@ check_balance(ErtsRunQueue *c_rq) || run_queue_info[qix].prio[pix].avail == 0) limit = 0; else - limit = (((avg.prio[pix].max_len - * run_queue_info[qix].prio[pix].avail) - 1) - / 100 + 1); + limit = (int) (((((Sint64) avg.prio[pix].max_len) + * ((Sint64) run_queue_info[qix].prio[pix].avail)) + - 1) + / 100 + 1); run_queue_info[qix].prio[pix].migration_limit = limit; } } @@ -2235,10 +2252,10 @@ erts_fprintf(stderr, "--------------------------------\n"); } balance_info.last_active_runqs = active; - erts_smp_atomic_set(&balance_info.active_runqs, active); + erts_smp_atomic32_set(&balance_info.active_runqs, active); balance_info.halftime = 1; - erts_smp_atomic_set(&balance_info.checking_balance, 0); + erts_smp_atomic32_set(&balance_info.checking_balance, 0); /* Write migration paths and reset balance statistics in all queues */ for (qix = 0; qix < blnc_no_rqs; qix++) { @@ -2387,7 +2404,7 @@ erts_init_scheduling(int mrq, int no_schedulers, int no_schedulers_online) ASSERT((((UWord) erts_aligned_run_queues) & ERTS_CACHE_LINE_MASK) == 0); #ifdef ERTS_SMP - erts_smp_atomic_init(&no_empty_run_queues, 0); + erts_smp_atomic32_init(&no_empty_run_queues, 0); #endif erts_no_run_queues = n; @@ -2397,7 +2414,7 @@ erts_init_scheduling(int mrq, int no_schedulers, int no_schedulers_online) ErtsRunQueue *rq = ERTS_RUNQ_IX(ix); rq->ix = ix; - erts_smp_atomic_init(&rq->info_flags, ERTS_RUNQ_IFLG_NONEMPTY); + erts_smp_atomic32_init(&rq->info_flags, ERTS_RUNQ_IFLG_NONEMPTY); /* make sure that the "extra" id correponds to the schedulers * id if the esdp->no <-> ix+1 mapping change. @@ -2494,9 +2511,9 @@ erts_init_scheduling(int mrq, int no_schedulers, int no_schedulers_online) ssi->next = NULL; ssi->prev = NULL; #endif - erts_smp_atomic_init(&ssi->flags, 0); + erts_smp_atomic32_init(&ssi->flags, 0); ssi->event = NULL; /* initialized in sched_thread_func */ - erts_smp_atomic_init(&ssi->aux_work, 0); + erts_smp_atomic32_init(&ssi->aux_work, 0); } #endif @@ -2547,7 +2564,7 @@ erts_init_scheduling(int mrq, int no_schedulers, int no_schedulers_online) } #ifdef ERTS_SMP - erts_smp_atomic_init(&esdp->chk_cpu_bind, 0); + erts_smp_atomic32_init(&esdp->chk_cpu_bind, 0); #endif } @@ -2555,21 +2572,21 @@ erts_init_scheduling(int mrq, int no_schedulers, int no_schedulers_online) erts_smp_mtx_init(&schdlr_sspnd.mtx, "schdlr_sspnd"); erts_smp_cnd_init(&schdlr_sspnd.cnd); - erts_smp_atomic_init(&schdlr_sspnd.changing, 0); + erts_smp_atomic32_init(&schdlr_sspnd.changing, 0); schdlr_sspnd.online = no_schedulers_online; schdlr_sspnd.curr_online = no_schedulers; - erts_smp_atomic_init(&schdlr_sspnd.msb.ongoing, 0); - erts_smp_atomic_init(&schdlr_sspnd.active, no_schedulers); + erts_smp_atomic32_init(&schdlr_sspnd.msb.ongoing, 0); + erts_smp_atomic32_init(&schdlr_sspnd.active, no_schedulers); schdlr_sspnd.msb.procs = NULL; - erts_smp_atomic_set(&balance_info.used_runqs, - erts_common_run_queue ? 1 : no_schedulers_online); - erts_smp_atomic_init(&balance_info.active_runqs, no_schedulers); + erts_smp_atomic32_set(&balance_info.used_runqs, + erts_common_run_queue ? 1 : no_schedulers_online); + erts_smp_atomic32_init(&balance_info.active_runqs, no_schedulers); balance_info.last_active_runqs = no_schedulers; erts_smp_mtx_init(&balance_info.update_mtx, "migration_info_update"); balance_info.forced_check_balance = 0; balance_info.halftime = 1; balance_info.full_reds_history_index = 0; - erts_smp_atomic_init(&balance_info.checking_balance, 0); + erts_smp_atomic32_init(&balance_info.checking_balance, 0); balance_info.prev_rise.active_runqs = 0; balance_info.prev_rise.max_len = 0; balance_info.prev_rise.reds = 0; @@ -2578,8 +2595,8 @@ erts_init_scheduling(int mrq, int no_schedulers, int no_schedulers_online) if (no_schedulers_online < no_schedulers) { if (erts_common_run_queue) { for (ix = no_schedulers_online; ix < no_schedulers; ix++) - erts_smp_atomic_bor(&ERTS_SCHED_SLEEP_INFO_IX(ix)->flags, - ERTS_SSI_FLG_SUSPENDED); + erts_smp_atomic32_bor(&ERTS_SCHED_SLEEP_INFO_IX(ix)->flags, + ERTS_SSI_FLG_SUSPENDED); } else { for (ix = no_schedulers_online; ix < erts_no_run_queues; ix++) @@ -2593,7 +2610,7 @@ erts_init_scheduling(int mrq, int no_schedulers, int no_schedulers_online) ERTS_SCHDLR_SSPND_CHNG_SET((ERTS_SCHDLR_SSPND_CHNG_ONLN | ERTS_SCHDLR_SSPND_CHNG_WAITER), 0); - erts_smp_atomic_init(&doing_sys_schedule, 0); + erts_smp_atomic32_init(&doing_sys_schedule, 0); #else /* !ERTS_SMP */ { @@ -2607,7 +2624,7 @@ erts_init_scheduling(int mrq, int no_schedulers, int no_schedulers_online) erts_no_schedulers = 1; #endif - erts_smp_atomic_init(&function_calls, 0); + erts_smp_atomic32_init(&function_calls, 0); /* init port tasks */ erts_port_task_init(); @@ -2740,13 +2757,13 @@ static void scheduler_ix_resume_wake(Uint ix) { ErtsSchedulerSleepInfo *ssi = ERTS_SCHED_SLEEP_INFO_IX(ix); - long xflgs = (ERTS_SSI_FLG_SLEEPING - | ERTS_SSI_FLG_TSE_SLEEPING - | ERTS_SSI_FLG_WAITING - | ERTS_SSI_FLG_SUSPENDED); - long oflgs; + erts_aint32_t xflgs = (ERTS_SSI_FLG_SLEEPING + | ERTS_SSI_FLG_TSE_SLEEPING + | ERTS_SSI_FLG_WAITING + | ERTS_SSI_FLG_SUSPENDED); + erts_aint32_t oflgs; do { - oflgs = erts_smp_atomic_cmpxchg(&ssi->flags, 0, xflgs); + oflgs = erts_smp_atomic32_cmpxchg(&ssi->flags, 0, xflgs); if (oflgs == xflgs) { erts_sched_finish_poke(ssi, oflgs); break; @@ -2755,17 +2772,17 @@ scheduler_ix_resume_wake(Uint ix) } while (oflgs & ERTS_SSI_FLG_SUSPENDED); } -static long -sched_prep_spin_suspended(ErtsSchedulerSleepInfo *ssi, long xpct) +static erts_aint32_t +sched_prep_spin_suspended(ErtsSchedulerSleepInfo *ssi, erts_aint32_t xpct) { - long oflgs; - long nflgs = (ERTS_SSI_FLG_SLEEPING - | ERTS_SSI_FLG_WAITING - | ERTS_SSI_FLG_SUSPENDED); - long xflgs = xpct; + erts_aint32_t oflgs; + erts_aint32_t nflgs = (ERTS_SSI_FLG_SLEEPING + | ERTS_SSI_FLG_WAITING + | ERTS_SSI_FLG_SUSPENDED); + erts_aint32_t xflgs = xpct; do { - oflgs = erts_smp_atomic_cmpxchg(&ssi->flags, nflgs, xflgs); + oflgs = erts_smp_atomic32_cmpxchg(&ssi->flags, nflgs, xflgs); if (oflgs == xflgs) return nflgs; xflgs = oflgs; @@ -2774,15 +2791,15 @@ sched_prep_spin_suspended(ErtsSchedulerSleepInfo *ssi, long xpct) return oflgs; } -static long +static erts_aint32_t sched_spin_suspended(ErtsSchedulerSleepInfo *ssi, int spincount) { int until_yield = ERTS_SCHED_SPIN_UNTIL_YIELD; int sc = spincount; - long flgs; + erts_aint32_t flgs; do { - flgs = erts_smp_atomic_read(&ssi->flags); + flgs = erts_smp_atomic32_read(&ssi->flags); if ((flgs & (ERTS_SSI_FLG_SLEEPING | ERTS_SSI_FLG_WAITING | ERTS_SSI_FLG_SUSPENDED)) @@ -2800,22 +2817,22 @@ sched_spin_suspended(ErtsSchedulerSleepInfo *ssi, int spincount) return flgs; } -static long +static erts_aint32_t sched_set_suspended_sleeptype(ErtsSchedulerSleepInfo *ssi) { - long oflgs; - long nflgs = (ERTS_SSI_FLG_SLEEPING - | ERTS_SSI_FLG_TSE_SLEEPING - | ERTS_SSI_FLG_WAITING - | ERTS_SSI_FLG_SUSPENDED); - long xflgs = (ERTS_SSI_FLG_SLEEPING - | ERTS_SSI_FLG_WAITING - | ERTS_SSI_FLG_SUSPENDED); + erts_aint32_t oflgs; + erts_aint32_t nflgs = (ERTS_SSI_FLG_SLEEPING + | ERTS_SSI_FLG_TSE_SLEEPING + | ERTS_SSI_FLG_WAITING + | ERTS_SSI_FLG_SUSPENDED); + erts_aint32_t xflgs = (ERTS_SSI_FLG_SLEEPING + | ERTS_SSI_FLG_WAITING + | ERTS_SSI_FLG_SUSPENDED); erts_tse_reset(ssi->event); while (1) { - oflgs = erts_smp_atomic_cmpxchg(&ssi->flags, nflgs, xflgs); + oflgs = erts_smp_atomic32_cmpxchg(&ssi->flags, nflgs, xflgs); if (oflgs == xflgs) return nflgs; if ((oflgs & (ERTS_SSI_FLG_SLEEPING @@ -2833,8 +2850,8 @@ sched_set_suspended_sleeptype(ErtsSchedulerSleepInfo *ssi) static void suspend_scheduler(ErtsSchedulerData *esdp) { - long flgs; - int changing; + erts_aint32_t flgs; + erts_aint32_t changing; long no = (long) esdp->no; ErtsSchedulerSleepInfo *ssi = esdp->ssi; long active_schedulers; @@ -2842,7 +2859,7 @@ suspend_scheduler(ErtsSchedulerData *esdp) int wake = 0; #if defined(ERTS_SCHED_NEED_NONBLOCKABLE_AUX_WORK) \ || defined(ERTS_SCHED_NEED_BLOCKABLE_AUX_WORK) - long aux_work; + erts_aint32_t aux_work; #endif /* @@ -2870,15 +2887,15 @@ suspend_scheduler(ErtsSchedulerData *esdp) flgs = sched_prep_spin_suspended(ssi, ERTS_SSI_FLG_SUSPENDED); if (flgs & ERTS_SSI_FLG_SUSPENDED) { - active_schedulers = erts_smp_atomic_dectest(&schdlr_sspnd.active); + active_schedulers = erts_smp_atomic32_dectest(&schdlr_sspnd.active); ASSERT(active_schedulers >= 1); - changing = erts_smp_atomic_read(&schdlr_sspnd.changing); + changing = erts_smp_atomic32_read(&schdlr_sspnd.changing); if (changing & ERTS_SCHDLR_SSPND_CHNG_MSB) { if (active_schedulers == schdlr_sspnd.msb.wait_active) wake = 1; if (active_schedulers == 1) { - changing = erts_smp_atomic_band(&schdlr_sspnd.changing, - ~ERTS_SCHDLR_SSPND_CHNG_MSB); + changing = erts_smp_atomic32_band(&schdlr_sspnd.changing, + ~ERTS_SCHDLR_SSPND_CHNG_MSB); changing &= ~ERTS_SCHDLR_SSPND_CHNG_MSB; } } @@ -2900,8 +2917,8 @@ suspend_scheduler(ErtsSchedulerData *esdp) && schdlr_sspnd.curr_online == schdlr_sspnd.wait_curr_online) wake = 1; if (schdlr_sspnd.online == schdlr_sspnd.curr_online) { - changing = erts_smp_atomic_band(&schdlr_sspnd.changing, - ~ERTS_SCHDLR_SSPND_CHNG_ONLN); + changing = erts_smp_atomic32_band(&schdlr_sspnd.changing, + ~ERTS_SCHDLR_SSPND_CHNG_ONLN); changing &= ~ERTS_SCHDLR_SSPND_CHNG_ONLN; } } @@ -2911,29 +2928,30 @@ suspend_scheduler(ErtsSchedulerData *esdp) wake = 0; } - flgs = erts_smp_atomic_read(&ssi->flags); + flgs = erts_smp_atomic32_read(&ssi->flags); if (!(flgs & ERTS_SSI_FLG_SUSPENDED)) break; erts_smp_mtx_unlock(&schdlr_sspnd.mtx); #ifdef ERTS_SCHED_NEED_BLOCKABLE_AUX_WORK - aux_work = erts_smp_atomic_read(&ssi->aux_work); + aux_work = erts_smp_atomic32_read(&ssi->aux_work); blockable_aux_work: blockable_aux_work(esdp, ssi, aux_work); #endif erts_smp_activity_begin(ERTS_ACTIVITY_WAIT, NULL, NULL, NULL); while (1) { - long flgs; + erts_aint32_t flgs; #ifdef ERTS_SCHED_NEED_NONBLOCKABLE_AUX_WORK #ifndef ERTS_SCHED_NEED_BLOCKABLE_AUX_WORK - aux_work = erts_smp_atomic_read(&ssi->aux_work); + aux_work = erts_smp_atomic32_read(&ssi->aux_work); #endif nonblockable_aux_work(esdp, ssi, aux_work); #endif - flgs = sched_spin_suspended(ssi, ERTS_SCHED_SUSPEND_SLEEP_SPINCOUNT); + flgs = sched_spin_suspended(ssi, + ERTS_SCHED_SUSPEND_SLEEP_SPINCOUNT); if (flgs == (ERTS_SSI_FLG_SLEEPING | ERTS_SSI_FLG_WAITING | ERTS_SSI_FLG_SUSPENDED)) { @@ -2953,13 +2971,13 @@ suspend_scheduler(ErtsSchedulerData *esdp) | ERTS_SSI_FLG_SUSPENDED)); if (!(flgs & ERTS_SSI_FLG_SUSPENDED)) break; - changing = erts_smp_atomic_read(&schdlr_sspnd.changing); + changing = erts_smp_atomic32_read(&schdlr_sspnd.changing); if (changing & ~ERTS_SCHDLR_SSPND_CHNG_WAITER) break; #ifdef ERTS_SCHED_NEED_BLOCKABLE_AUX_WORK - aux_work = erts_smp_atomic_read(&ssi->aux_work); + aux_work = erts_smp_atomic32_read(&ssi->aux_work); if (aux_work & ERTS_SSI_BLOCKABLE_AUX_WORK_MASK) { erts_smp_activity_end(ERTS_ACTIVITY_WAIT, NULL, NULL, NULL); goto blockable_aux_work; @@ -2971,19 +2989,19 @@ suspend_scheduler(ErtsSchedulerData *esdp) erts_smp_activity_end(ERTS_ACTIVITY_WAIT, NULL, NULL, NULL); erts_smp_mtx_lock(&schdlr_sspnd.mtx); - changing = erts_smp_atomic_read(&schdlr_sspnd.changing); + changing = erts_smp_atomic32_read(&schdlr_sspnd.changing); } - active_schedulers = erts_smp_atomic_inctest(&schdlr_sspnd.active); - changing = erts_smp_atomic_read(&schdlr_sspnd.changing); + active_schedulers = erts_smp_atomic32_inctest(&schdlr_sspnd.active); + changing = erts_smp_atomic32_read(&schdlr_sspnd.changing); if ((changing & ERTS_SCHDLR_SSPND_CHNG_MSB) && schdlr_sspnd.online == active_schedulers) { - erts_smp_atomic_band(&schdlr_sspnd.changing, - ~ERTS_SCHDLR_SSPND_CHNG_MSB); + erts_smp_atomic32_band(&schdlr_sspnd.changing, + ~ERTS_SCHDLR_SSPND_CHNG_MSB); } ASSERT(no <= schdlr_sspnd.online); - ASSERT(!erts_smp_atomic_read(&schdlr_sspnd.msb.ongoing)); + ASSERT(!erts_smp_atomic32_read(&schdlr_sspnd.msb.ongoing)); } @@ -3012,7 +3030,7 @@ do { \ (RQ)->flags |= (ERTS_RUNQ_FLG_OUT_OF_WORK \ | ERTS_RUNQ_FLG_HALFTIME_OUT_OF_WORK); \ (RQ)->check_balance_reds = ERTS_RUNQ_CALL_CHECK_BALANCE_REDS; \ - erts_smp_atomic_band(&(RQ)->info_flags, ~ERTS_RUNQ_IFLG_SUSPENDED); \ + erts_smp_atomic32_band(&(RQ)->info_flags, ~ERTS_RUNQ_IFLG_SUSPENDED);\ for (pix__ = 0; pix__ < ERTS_NO_PROC_PRIO_LEVELS; pix__++) { \ (RQ)->procs.prio_info[pix__].max_len = 0; \ (RQ)->procs.prio_info[pix__].reds = 0; \ @@ -3054,9 +3072,9 @@ erts_schedulers_state(Uint *total, int yield_allowed) { int res; - long changing; + erts_aint32_t changing; erts_smp_mtx_lock(&schdlr_sspnd.mtx); - changing = erts_smp_atomic_read(&schdlr_sspnd.changing); + changing = erts_smp_atomic32_read(&schdlr_sspnd.changing); if (yield_allowed && (changing & ~ERTS_SCHDLR_SSPND_CHNG_WAITER)) res = ERTS_SCHDLR_SSPND_YIELD_RESTART; else { @@ -3077,7 +3095,7 @@ erts_set_schedulers_online(Process *p, Sint *old_no) { int ix, res, no, have_unlocked_plocks; - long changing; + erts_aint32_t changing; if (new_no < 1 || erts_no_schedulers < new_no) return ERTS_SCHDLR_SSPND_EINVAL; @@ -3087,7 +3105,7 @@ erts_set_schedulers_online(Process *p, have_unlocked_plocks = 0; no = (int) new_no; - changing = erts_smp_atomic_read(&schdlr_sspnd.changing); + changing = erts_smp_atomic32_read(&schdlr_sspnd.changing); if (changing) { res = ERTS_SCHDLR_SSPND_YIELD_RESTART; } @@ -3134,7 +3152,7 @@ erts_set_schedulers_online(Process *p, ErtsRunQueue *to_rq = ERTS_RUNQ_IX(ix % no); evacuate_run_queue(from_rq, to_rq); } - erts_smp_atomic_set(&balance_info.used_runqs, no); + erts_smp_atomic32_set(&balance_info.used_runqs, no); erts_smp_mtx_unlock(&balance_info.update_mtx); erts_smp_mtx_lock(&schdlr_sspnd.mtx); } @@ -3162,8 +3180,8 @@ erts_set_schedulers_online(Process *p, for (ix = no; ix < online; ix++) { ErtsSchedulerSleepInfo *ssi; ssi = ERTS_SCHED_SLEEP_INFO_IX(ix); - erts_smp_atomic_bor(&ssi->flags, - ERTS_SSI_FLG_SUSPENDED); + erts_smp_atomic32_bor(&ssi->flags, + ERTS_SSI_FLG_SUSPENDED); } wake_all_schedulers(); } @@ -3188,7 +3206,7 @@ erts_set_schedulers_online(Process *p, for (ix = erts_no_run_queues-1; ix >= no; ix--) evacuate_run_queue(ERTS_RUNQ_IX(ix), ERTS_RUNQ_IX(ix % no)); - erts_smp_atomic_set(&balance_info.used_runqs, no); + erts_smp_atomic32_set(&balance_info.used_runqs, no); erts_smp_mtx_unlock(&balance_info.update_mtx); erts_smp_mtx_lock(&schdlr_sspnd.mtx); for (ix = no; ix < online; ix++) { @@ -3210,10 +3228,11 @@ erts_set_schedulers_online(Process *p, NULL); ASSERT(res != ERTS_SCHDLR_SSPND_DONE ? (ERTS_SCHDLR_SSPND_CHNG_WAITER - & erts_smp_atomic_read(&schdlr_sspnd.changing)) + & erts_smp_atomic32_read(&schdlr_sspnd.changing)) : (ERTS_SCHDLR_SSPND_CHNG_WAITER - == erts_smp_atomic_read(&schdlr_sspnd.changing))); - erts_smp_atomic_band(&schdlr_sspnd.changing, ~ERTS_SCHDLR_SSPND_CHNG_WAITER); + == erts_smp_atomic32_read(&schdlr_sspnd.changing))); + erts_smp_atomic32_band(&schdlr_sspnd.changing, + ~ERTS_SCHDLR_SSPND_CHNG_WAITER); } } @@ -3228,11 +3247,11 @@ ErtsSchedSuspendResult erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) { int ix, res, have_unlocked_plocks = 0; - long changing; + erts_aint32_t changing; ErtsProcList *plp; erts_smp_mtx_lock(&schdlr_sspnd.mtx); - changing = erts_smp_atomic_read(&schdlr_sspnd.changing); + changing = erts_smp_atomic32_read(&schdlr_sspnd.changing); if (changing) { res = ERTS_SCHDLR_SSPND_YIELD_RESTART; /* Yield */ } @@ -3242,7 +3261,7 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) plp->next = schdlr_sspnd.msb.procs; schdlr_sspnd.msb.procs = plp; p->flags |= F_HAVE_BLCKD_MSCHED; - ASSERT(erts_smp_atomic_read(&schdlr_sspnd.active) == 1); + ASSERT(erts_smp_atomic32_read(&schdlr_sspnd.active) == 1); ASSERT(p->scheduler_data->no == 1); res = ERTS_SCHDLR_SSPND_DONE_MSCHED_BLOCKED; } @@ -3253,11 +3272,11 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) have_unlocked_plocks = 1; erts_smp_proc_unlock(p, plocks); } - ASSERT(0 == erts_smp_atomic_read(&schdlr_sspnd.msb.ongoing)); - erts_smp_atomic_set(&schdlr_sspnd.msb.ongoing, 1); + ASSERT(0 == erts_smp_atomic32_read(&schdlr_sspnd.msb.ongoing)); + erts_smp_atomic32_set(&schdlr_sspnd.msb.ongoing, 1); if (online == 1) { res = ERTS_SCHDLR_SSPND_DONE_MSCHED_BLOCKED; - ASSERT(erts_smp_atomic_read(&schdlr_sspnd.active) == 1); + ASSERT(erts_smp_atomic32_read(&schdlr_sspnd.active) == 1); ASSERT(p->scheduler_data->no == 1); } else { @@ -3277,14 +3296,14 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) } if (erts_common_run_queue) { for (ix = 1; ix < online; ix++) - erts_smp_atomic_bor(&ERTS_SCHED_SLEEP_INFO_IX(ix)->flags, - ERTS_SSI_FLG_SUSPENDED); + erts_smp_atomic32_bor(&ERTS_SCHED_SLEEP_INFO_IX(ix)->flags, + ERTS_SSI_FLG_SUSPENDED); wake_all_schedulers(); } else { erts_smp_mtx_unlock(&schdlr_sspnd.mtx); erts_smp_mtx_lock(&balance_info.update_mtx); - erts_smp_atomic_set(&balance_info.used_runqs, 1); + erts_smp_atomic32_set(&balance_info.used_runqs, 1); for (ix = 0; ix < online; ix++) { ErtsRunQueue *rq = ERTS_RUNQ_IX(ix); erts_smp_runq_lock(rq); @@ -3306,7 +3325,7 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) susp_sched_prep_block, susp_sched_resume_block, NULL); - while (erts_smp_atomic_read(&schdlr_sspnd.active) + while (erts_smp_atomic32_read(&schdlr_sspnd.active) != schdlr_sspnd.msb.wait_active) erts_smp_cnd_wait(&schdlr_sspnd.cnd, &schdlr_sspnd.mtx); erts_smp_activity_end(ERTS_ACTIVITY_WAIT, @@ -3315,11 +3334,11 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) NULL); ASSERT(res != ERTS_SCHDLR_SSPND_DONE_MSCHED_BLOCKED ? (ERTS_SCHDLR_SSPND_CHNG_WAITER - & erts_smp_atomic_read(&schdlr_sspnd.changing)) + & erts_smp_atomic32_read(&schdlr_sspnd.changing)) : (ERTS_SCHDLR_SSPND_CHNG_WAITER - == erts_smp_atomic_read(&schdlr_sspnd.changing))); - erts_smp_atomic_band(&schdlr_sspnd.changing, - ~ERTS_SCHDLR_SSPND_CHNG_WAITER); + == erts_smp_atomic32_read(&schdlr_sspnd.changing))); + erts_smp_atomic32_band(&schdlr_sspnd.changing, + ~ERTS_SCHDLR_SSPND_CHNG_WAITER); } plp = proclist_create(p); plp->next = schdlr_sspnd.msb.procs; @@ -3386,16 +3405,16 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) }); #endif p->flags &= ~F_HAVE_BLCKD_MSCHED; - erts_smp_atomic_set(&schdlr_sspnd.msb.ongoing, 0); + erts_smp_atomic32_set(&schdlr_sspnd.msb.ongoing, 0); if (schdlr_sspnd.online == 1) { /* No schedulers to resume */ - ASSERT(erts_smp_atomic_read(&schdlr_sspnd.active) == 1); + ASSERT(erts_smp_atomic32_read(&schdlr_sspnd.active) == 1); ERTS_SCHDLR_SSPND_CHNG_SET(0, ERTS_SCHDLR_SSPND_CHNG_MSB); } else if (erts_common_run_queue) { for (ix = 1; ix < schdlr_sspnd.online; ix++) - erts_smp_atomic_band(&ERTS_SCHED_SLEEP_INFO_IX(ix)->flags, - ~ERTS_SSI_FLG_SUSPENDED); + erts_smp_atomic32_band(&ERTS_SCHED_SLEEP_INFO_IX(ix)->flags, + ~ERTS_SSI_FLG_SUSPENDED); wake_all_schedulers(); } else { @@ -3421,7 +3440,7 @@ erts_block_multi_scheduling(Process *p, ErtsProcLocks plocks, int on, int all) evacuate_run_queue(ERTS_RUNQ_IX(ix), ERTS_RUNQ_IX(ix % online)); - erts_smp_atomic_set(&balance_info.used_runqs, online); + erts_smp_atomic32_set(&balance_info.used_runqs, online); /* Make sure that we balance soon... */ balance_info.forced_check_balance = 1; erts_smp_runq_lock(ERTS_RUNQ_IX(0)); @@ -3445,7 +3464,7 @@ void erts_dbg_multi_scheduling_return_trap(Process *p, Eterm return_value) { if (return_value == am_blocked) { - long active = erts_smp_atomic_read(&schdlr_sspnd.active); + erts_aint32_t active = erts_smp_atomic32_read(&schdlr_sspnd.active); ASSERT(1 <= active && active <= 2); ASSERT(ERTS_PROC_GET_SCHDATA(p)->no == 1); } @@ -3528,12 +3547,12 @@ sched_thread_func(void *vesdp) erts_thread_init_float(); erts_smp_mtx_lock(&schdlr_sspnd.mtx); - ASSERT(erts_smp_atomic_read(&schdlr_sspnd.changing) + ASSERT(erts_smp_atomic32_read(&schdlr_sspnd.changing) & ERTS_SCHDLR_SSPND_CHNG_ONLN); if (--schdlr_sspnd.curr_online == schdlr_sspnd.wait_curr_online) { - erts_smp_atomic_band(&schdlr_sspnd.changing, - ~ERTS_SCHDLR_SSPND_CHNG_ONLN); + erts_smp_atomic32_band(&schdlr_sspnd.changing, + ~ERTS_SCHDLR_SSPND_CHNG_ONLN); if (((ErtsSchedulerData *) vesdp)->no != 1) erts_smp_cnd_signal(&schdlr_sspnd.cnd); } @@ -4906,10 +4925,10 @@ Process *schedule(Process *p, int calls) { ErtsRunQueue *rq; ErtsRunPrioQueue *rpq; - long dt; + erts_aint_t dt; ErtsSchedulerData *esdp; int context_reds; - long fcalls; + int fcalls; int input_reductions; int actual_reds; int reds; @@ -4932,7 +4951,7 @@ Process *schedule(Process *p, int calls) esdp = erts_get_scheduler_data(); rq = erts_get_runq_current(esdp); ASSERT(esdp); - fcalls = erts_smp_atomic_read(&function_calls); + fcalls = (int) erts_smp_atomic32_read(&function_calls); actual_reds = reds = 0; erts_smp_runq_lock(rq); } else { @@ -4950,7 +4969,7 @@ Process *schedule(Process *p, int calls) reds = ERTS_PROC_MIN_CONTEXT_SWITCH_REDS_COST; esdp->virtual_reds = 0; - fcalls = erts_smp_atomic_addtest(&function_calls, reds); + fcalls = (int) erts_smp_atomic32_addtest(&function_calls, reds); ASSERT(esdp && esdp == erts_get_scheduler_data()); rq = erts_get_runq_current(esdp); @@ -5083,14 +5102,14 @@ Process *schedule(Process *p, int calls) | ERTS_RUNQ_FLG_CHK_CPU_BIND | ERTS_RUNQ_FLG_SUSPENDED)) { if ((rq->flags & ERTS_RUNQ_FLG_SUSPENDED) - || (erts_smp_atomic_read(&esdp->ssi->flags) + || (erts_smp_atomic32_read(&esdp->ssi->flags) & ERTS_SSI_FLG_SUSPENDED)) { - ASSERT(erts_smp_atomic_read(&esdp->ssi->flags) + ASSERT(erts_smp_atomic32_read(&esdp->ssi->flags) & ERTS_SSI_FLG_SUSPENDED); suspend_scheduler(esdp); } if ((rq->flags & ERTS_RUNQ_FLG_CHK_CPU_BIND) - || erts_smp_atomic_read(&esdp->chk_cpu_bind)) { + || erts_smp_atomic32_read(&esdp->chk_cpu_bind)) { erts_sched_check_cpu_bind(esdp); } } @@ -5099,7 +5118,7 @@ Process *schedule(Process *p, int calls) || defined(ERTS_SCHED_NEED_NONBLOCKABLE_AUX_WORK) { ErtsSchedulerSleepInfo *ssi = esdp->ssi; - long aux_work = erts_smp_atomic_read(&ssi->aux_work); + erts_aint32_t aux_work = erts_smp_atomic32_read(&ssi->aux_work); if (aux_work) { erts_smp_runq_unlock(rq); #ifdef ERTS_SCHED_NEED_BLOCKABLE_AUX_WORK @@ -5141,9 +5160,9 @@ Process *schedule(Process *p, int calls) if (rq->flags & (ERTS_RUNQ_FLG_SHARED_RUNQ | ERTS_RUNQ_FLG_SUSPENDED)) { if ((rq->flags & ERTS_RUNQ_FLG_SUSPENDED) - || (erts_smp_atomic_read(&esdp->ssi->flags) + || (erts_smp_atomic32_read(&esdp->ssi->flags) & ERTS_SSI_FLG_SUSPENDED)) { - ASSERT(erts_smp_atomic_read(&esdp->ssi->flags) + ASSERT(erts_smp_atomic32_read(&esdp->ssi->flags) & ERTS_SSI_FLG_SUSPENDED); non_empty_runq(rq); goto continue_check_activities_to_run; @@ -5185,7 +5204,7 @@ Process *schedule(Process *p, int calls) * Schedule system-level activities. */ - erts_smp_atomic_set(&function_calls, 0); + erts_smp_atomic32_set(&function_calls, 0); fcalls = 0; ASSERT(!erts_port_task_have_outstanding_io_tasks()); @@ -5199,7 +5218,7 @@ Process *schedule(Process *p, int calls) if (dt) bump_timer(dt); #ifdef ERTS_SMP erts_smp_runq_lock(rq); - erts_smp_atomic_set(&doing_sys_schedule, 0); + erts_smp_atomic32_set(&doing_sys_schedule, 0); goto continue_check_activities_to_run; #else if (!runnable) @@ -5227,7 +5246,7 @@ Process *schedule(Process *p, int calls) if (erts_common_run_queue->waiting) wake_scheduler(erts_common_run_queue, 0, 1); } - else if (erts_smp_atomic_read(&no_empty_run_queues) != 0) { + else if (erts_smp_atomic32_read(&no_empty_run_queues) != 0) { wake_scheduler_on_empty_runq(rq); rq->wakeup_other = 0; } @@ -5684,7 +5703,7 @@ erts_test_next_pid(int set, Uint next) Uint erts_process_count(void) { - long res = erts_smp_atomic_read(&process_count); + erts_aint32_t res = erts_smp_atomic32_read(&process_count); ASSERT(res >= 0); return (Uint) res; } @@ -5733,7 +5752,7 @@ alloc_process(void) ASSERT(!process_tab[p_next]); process_tab[p_next] = p; - erts_smp_atomic_inc(&process_count); + erts_smp_atomic32_inc(&process_count); p->id = make_internal_pid(p_serial << p_serial_shift | p_next); if (p->id == ERTS_INVALID_PID) { /* Do not use the invalid pid; change serial */ @@ -5859,7 +5878,7 @@ erl_create_process(Process* parent, /* Parent of process (default group leader). p->min_heap_size = H_MIN_SIZE; p->min_vheap_size = BIN_VH_MIN_SIZE; p->prio = PRIORITY_NORMAL; - p->max_gen_gcs = (Uint16) erts_smp_atomic_read(&erts_max_gen_gcs); + p->max_gen_gcs = (Uint16) erts_smp_atomic32_read(&erts_max_gen_gcs); } p->skipped = 0; ASSERT(p->min_heap_size == erts_next_heap_size(p->min_heap_size, 0)); @@ -7316,8 +7335,8 @@ continue_exit_process(Process *p p->status_flags = 0; #endif process_tab[pix] = NULL; /* Time of death! */ - ASSERT(erts_smp_atomic_read(&process_count) > 0); - erts_smp_atomic_dec(&process_count); + ASSERT(erts_smp_atomic32_read(&process_count) > 0); + erts_smp_atomic32_dec(&process_count); #ifdef ERTS_SMP erts_pix_unlock(pix_lock); diff --git a/erts/emulator/beam/erl_process.h b/erts/emulator/beam/erl_process.h index c038e57b65..8e32121a68 100644 --- a/erts/emulator/beam/erl_process.h +++ b/erts/emulator/beam/erl_process.h @@ -174,8 +174,8 @@ extern int erts_sched_thread_suggested_stack_size; #define ERTS_UNSET_RUNQ_FLG_EVACUATE(FLGS, PRIO) \ ((FLGS) &= ~ERTS_RUNQ_FLG_EVACUATE((PRIO))) -#define ERTS_RUNQ_IFLG_SUSPENDED (((long) 1) << 0) -#define ERTS_RUNQ_IFLG_NONEMPTY (((long) 1) << 1) +#define ERTS_RUNQ_IFLG_SUSPENDED (((erts_aint32_t) 1) << 0) +#define ERTS_RUNQ_IFLG_NONEMPTY (((erts_aint32_t) 1) << 1) #ifdef DEBUG @@ -219,11 +219,11 @@ typedef enum { ERTS_MIGRATE_FAILED_RUNQ_SUSPENDED } ErtsMigrateResult; -#define ERTS_SSI_FLG_SLEEPING (((long) 1) << 0) -#define ERTS_SSI_FLG_POLL_SLEEPING (((long) 1) << 1) -#define ERTS_SSI_FLG_TSE_SLEEPING (((long) 1) << 2) -#define ERTS_SSI_FLG_WAITING (((long) 1) << 3) -#define ERTS_SSI_FLG_SUSPENDED (((long) 1) << 4) +#define ERTS_SSI_FLG_SLEEPING (((erts_aint32_t) 1) << 0) +#define ERTS_SSI_FLG_POLL_SLEEPING (((erts_aint32_t) 1) << 1) +#define ERTS_SSI_FLG_TSE_SLEEPING (((erts_aint32_t) 1) << 2) +#define ERTS_SSI_FLG_WAITING (((erts_aint32_t) 1) << 3) +#define ERTS_SSI_FLG_SUSPENDED (((erts_aint32_t) 1) << 4) #define ERTS_SSI_FLGS_SLEEP_TYPE \ (ERTS_SSI_FLG_TSE_SLEEPING|ERTS_SSI_FLG_POLL_SLEEPING) @@ -242,7 +242,7 @@ typedef enum { #define ERTS_SCHED_NEED_BLOCKABLE_AUX_WORK #endif -#define ERTS_SSI_AUX_WORK_CHECK_CHILDREN (((long) 1) << 0) +#define ERTS_SSI_AUX_WORK_CHECK_CHILDREN (((erts_aint32_t) 1) << 0) #define ERTS_SSI_BLOCKABLE_AUX_WORK_MASK \ (ERTS_SSI_AUX_WORK_CHECK_CHILDREN) @@ -259,9 +259,9 @@ typedef struct { struct ErtsSchedulerSleepInfo_ { ErtsSchedulerSleepInfo *next; ErtsSchedulerSleepInfo *prev; - erts_smp_atomic_t flags; + erts_smp_atomic32_t flags; erts_tse_t *event; - erts_smp_atomic_t aux_work; + erts_smp_atomic32_t aux_work; }; /* times to reschedule low prio process before running */ @@ -311,7 +311,7 @@ typedef struct { struct ErtsRunQueue_ { int ix; - erts_smp_atomic_t info_flags; + erts_smp_atomic32_t info_flags; erts_smp_mtx_t mtx; erts_smp_cnd_t cnd; @@ -421,7 +421,7 @@ struct ErtsSchedulerData_ { #ifdef ERTS_SMP /* NOTE: These fields are modified under held mutexes by other threads */ - erts_smp_atomic_t chk_cpu_bind; /* Only used when common run queue */ + erts_smp_atomic32_t chk_cpu_bind; /* Only used when common run queue */ #endif }; @@ -1555,7 +1555,7 @@ extern int erts_disable_proc_not_running_opt; void erts_smp_notify_inc_runq(ErtsRunQueue *runq); #ifdef ERTS_SMP -void erts_sched_finish_poke(ErtsSchedulerSleepInfo *, long); +void erts_sched_finish_poke(ErtsSchedulerSleepInfo *, erts_aint32_t); ERTS_GLB_INLINE void erts_sched_poke(ErtsSchedulerSleepInfo *ssi); #if ERTS_GLB_INLINE_INCL_FUNC_DEF @@ -1563,11 +1563,11 @@ ERTS_GLB_INLINE void erts_sched_poke(ErtsSchedulerSleepInfo *ssi); ERTS_GLB_INLINE void erts_sched_poke(ErtsSchedulerSleepInfo *ssi) { - long flags = erts_smp_atomic_read(&ssi->flags); + erts_aint32_t flags = erts_smp_atomic32_read(&ssi->flags); ASSERT(!(flags & ERTS_SSI_FLG_SLEEPING) || (flags & ERTS_SSI_FLG_WAITING)); if (flags & ERTS_SSI_FLG_SLEEPING) { - flags = erts_smp_atomic_band(&ssi->flags, ~ERTS_SSI_FLGS_SLEEP); + flags = erts_smp_atomic32_band(&ssi->flags, ~ERTS_SSI_FLGS_SLEEP); erts_sched_finish_poke(ssi, flags); } } diff --git a/erts/emulator/beam/erl_process_lock.c b/erts/emulator/beam/erl_process_lock.c index 1bebcdb911..72560aa124 100644 --- a/erts/emulator/beam/erl_process_lock.c +++ b/erts/emulator/beam/erl_process_lock.c @@ -124,7 +124,7 @@ erts_init_proc_lock(int cpus) for (i = 0; i < ERTS_NO_OF_PIX_LOCKS; i++) { #ifdef ERTS_ENABLE_LOCK_COUNT erts_smp_spinlock_init_x(&erts_pix_locks[i].u.spnlck, - "pix_lock", make_small(i)); + "pix_lock", make_small(i)); #else erts_smp_spinlock_init(&erts_pix_locks[i].u.spnlck, "pix_lock"); #endif @@ -413,7 +413,7 @@ transfer_locks(Process *p, do { erts_tse_t *tmp = wake; wake = wake->next; - erts_atomic_set(&tmp->uaflgs, 0); + erts_atomic32_set(&tmp->uaflgs, 0); erts_tse_set(tmp); } while (wake); @@ -509,14 +509,14 @@ wait_for_locks(Process *p, ASSERT((wtr->uflgs & ~ERTS_PROC_LOCKS_ALL) == 0); - erts_atomic_set(&wtr->uaflgs, 1); + erts_atomic32_set(&wtr->uaflgs, 1); erts_pix_unlock(pix_lock); while (1) { int res; erts_tse_reset(wtr); - if (erts_atomic_read(&wtr->uaflgs) == 0) + if (erts_atomic32_read(&wtr->uaflgs) == 0) break; /* @@ -955,7 +955,7 @@ erts_proc_lock_init(Process *p) { /* We always start with all locks locked */ #if ERTS_PROC_LOCK_ATOMIC_IMPL - erts_smp_atomic_init(&p->lock.flags, (long) ERTS_PROC_LOCKS_ALL); + erts_smp_atomic32_init(&p->lock.flags, (erts_aint32_t) ERTS_PROC_LOCKS_ALL); #else p->lock.flags = ERTS_PROC_LOCKS_ALL; #endif @@ -974,7 +974,7 @@ erts_proc_lock_init(Process *p) { int i; for (i = 0; i <= ERTS_PROC_LOCK_MAX_BIT; i++) - erts_smp_atomic_init(&p->lock.locked[i], (long) 1); + erts_smp_atomic32_init(&p->lock.locked[i], (erts_aint32_t) 1); } #endif } diff --git a/erts/emulator/beam/erl_process_lock.h b/erts/emulator/beam/erl_process_lock.h index 4fe30c7209..355179f084 100644 --- a/erts/emulator/beam/erl_process_lock.h +++ b/erts/emulator/beam/erl_process_lock.h @@ -54,20 +54,20 @@ #define ERTS_PROC_LOCK_MAX_BIT 3 -typedef Uint32 ErtsProcLocks; +typedef erts_aint32_t ErtsProcLocks; typedef struct erts_proc_lock_queues_t_ erts_proc_lock_queues_t; typedef struct erts_proc_lock_t_ { #if ERTS_PROC_LOCK_ATOMIC_IMPL - erts_smp_atomic_t flags; + erts_smp_atomic32_t flags; #else ErtsProcLocks flags; #endif erts_proc_lock_queues_t *queues; - long refc; + Sint32 refc; #ifdef ERTS_PROC_LOCK_DEBUG - erts_smp_atomic_t locked[ERTS_PROC_LOCK_MAX_BIT+1]; + erts_smp_atomic32_t locked[ERTS_PROC_LOCK_MAX_BIT+1]; #endif #ifdef ERTS_ENABLE_LOCK_COUNT erts_lcnt_lock_t lcnt_main; @@ -270,17 +270,19 @@ typedef struct { #if ERTS_PROC_LOCK_ATOMIC_IMPL #define ERTS_PROC_LOCK_FLGS_BAND_(L, MSK) \ - ((ErtsProcLocks) erts_smp_atomic_band(&(L)->flags, (long) (MSK))) + ((ErtsProcLocks) erts_smp_atomic32_band(&(L)->flags, (erts_aint32_t) (MSK))) #define ERTS_PROC_LOCK_FLGS_BOR_(L, MSK) \ - ((ErtsProcLocks) erts_smp_atomic_bor(&(L)->flags, (long) (MSK))) + ((ErtsProcLocks) erts_smp_atomic32_bor(&(L)->flags, (erts_aint32_t) (MSK))) #define ERTS_PROC_LOCK_FLGS_CMPXCHG_ACQB_(L, NEW, EXPECTED) \ - ((ErtsProcLocks) erts_smp_atomic_cmpxchg_acqb(&(L)->flags, \ - (long) (NEW), (long) (EXPECTED))) + ((ErtsProcLocks) erts_smp_atomic32_cmpxchg_acqb(&(L)->flags, \ + (erts_aint32_t) (NEW), \ + (erts_aint32_t) (EXPECTED))) #define ERTS_PROC_LOCK_FLGS_CMPXCHG_RELB_(L, NEW, EXPECTED) \ - ((ErtsProcLocks) erts_smp_atomic_cmpxchg_relb(&(L)->flags, \ - (long) (NEW), (long) (EXPECTED))) + ((ErtsProcLocks) erts_smp_atomic32_cmpxchg_relb(&(L)->flags, \ + (erts_aint32_t) (NEW), \ + (erts_aint32_t) (EXPECTED))) #define ERTS_PROC_LOCK_FLGS_READ_(L) \ - ((ErtsProcLocks) erts_smp_atomic_read(&(L)->flags)) + ((ErtsProcLocks) erts_smp_atomic32_read(&(L)->flags)) #else /* no opt atomic ops */ @@ -619,13 +621,13 @@ erts_proc_lock_op_debug(Process *p, ErtsProcLocks locks, int locked) for (i = 0; i <= ERTS_PROC_LOCK_MAX_BIT; i++) { ErtsProcLocks lock = ((ErtsProcLocks) 1) << i; if (locks & lock) { - long lock_count; + erts_aint32_t lock_count; if (locked) { - lock_count = erts_smp_atomic_inctest(&p->lock.locked[i]); + lock_count = erts_smp_atomic32_inctest(&p->lock.locked[i]); ERTS_LC_ASSERT(lock_count == 1); } else { - lock_count = erts_smp_atomic_dectest(&p->lock.locked[i]); + lock_count = erts_smp_atomic32_dectest(&p->lock.locked[i]); ERTS_LC_ASSERT(lock_count == 0); } } diff --git a/erts/emulator/beam/erl_smp.h b/erts/emulator/beam/erl_smp.h index b41fa70476..287327bfe1 100644 --- a/erts/emulator/beam/erl_smp.h +++ b/erts/emulator/beam/erl_smp.h @@ -54,10 +54,10 @@ typedef erts_cnd_t erts_smp_cnd_t; typedef erts_rwmtx_opt_t erts_smp_rwmtx_opt_t; typedef erts_rwmtx_t erts_smp_rwmtx_t; typedef erts_tsd_key_t erts_smp_tsd_key_t; -typedef ethr_atomic_t erts_smp_atomic_t; +typedef erts_atomic_t erts_smp_atomic_t; +typedef erts_atomic32_t erts_smp_atomic32_t; typedef erts_spinlock_t erts_smp_spinlock_t; typedef erts_rwlock_t erts_smp_rwlock_t; -typedef erts_thr_timeval_t erts_smp_thr_timeval_t; void erts_thr_fatal_error(int, char *); /* implemented in erl_init.c */ #else /* #ifdef ERTS_SMP */ @@ -83,7 +83,8 @@ typedef struct { } erts_smp_rwmtx_opt_t; typedef int erts_smp_rwmtx_t; typedef int erts_smp_tsd_key_t; -typedef long erts_smp_atomic_t; +typedef SWord erts_smp_atomic_t; +typedef Uint32 erts_smp_atomic32_t; #if __GNUC__ > 2 typedef struct { } erts_smp_spinlock_t; typedef struct { } erts_smp_rwlock_t; @@ -92,11 +93,6 @@ typedef struct { int gcc_is_buggy; } erts_smp_spinlock_t; typedef struct { int gcc_is_buggy; } erts_smp_rwlock_t; #endif -typedef struct { - long tv_sec; - long tv_nsec; -} erts_smp_thr_timeval_t; - #endif /* #ifdef ERTS_SMP */ ERTS_GLB_INLINE void erts_smp_thr_init(erts_smp_thr_init_data_t *id); @@ -164,33 +160,82 @@ ERTS_GLB_INLINE int erts_smp_rwmtx_tryrwlock(erts_smp_rwmtx_t *rwmtx); ERTS_GLB_INLINE void erts_smp_rwmtx_rwunlock(erts_smp_rwmtx_t *rwmtx); ERTS_GLB_INLINE int erts_smp_lc_rwmtx_is_rlocked(erts_smp_rwmtx_t *mtx); ERTS_GLB_INLINE int erts_smp_lc_rwmtx_is_rwlocked(erts_smp_rwmtx_t *mtx); -ERTS_GLB_INLINE void erts_smp_atomic_init(erts_smp_atomic_t *var, long i); -ERTS_GLB_INLINE void erts_smp_atomic_set(erts_smp_atomic_t *var, long i); -ERTS_GLB_INLINE long erts_smp_atomic_read(erts_smp_atomic_t *var); -ERTS_GLB_INLINE long erts_smp_atomic_inctest(erts_smp_atomic_t *incp); -ERTS_GLB_INLINE long erts_smp_atomic_dectest(erts_smp_atomic_t *decp); +ERTS_GLB_INLINE void erts_smp_atomic_init(erts_smp_atomic_t *var, + erts_aint_t i); +ERTS_GLB_INLINE void erts_smp_atomic_set(erts_smp_atomic_t *var, erts_aint_t i); +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_read(erts_smp_atomic_t *var); +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_inctest(erts_smp_atomic_t *incp); +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_dectest(erts_smp_atomic_t *decp); ERTS_GLB_INLINE void erts_smp_atomic_inc(erts_smp_atomic_t *incp); ERTS_GLB_INLINE void erts_smp_atomic_dec(erts_smp_atomic_t *decp); -ERTS_GLB_INLINE long erts_smp_atomic_addtest(erts_smp_atomic_t *addp, - long i); -ERTS_GLB_INLINE void erts_smp_atomic_add(erts_smp_atomic_t *addp, long i); -ERTS_GLB_INLINE long erts_smp_atomic_xchg(erts_smp_atomic_t *xchgp, - long new); -ERTS_GLB_INLINE long erts_smp_atomic_cmpxchg(erts_smp_atomic_t *xchgp, - long new, - long expected); -ERTS_GLB_INLINE long erts_smp_atomic_bor(erts_smp_atomic_t *var, long mask); -ERTS_GLB_INLINE long erts_smp_atomic_band(erts_smp_atomic_t *var, long mask); -ERTS_GLB_INLINE long erts_smp_atomic_read_acqb(erts_smp_atomic_t *var); -ERTS_GLB_INLINE void erts_smp_atomic_set_relb(erts_smp_atomic_t *var, long i); +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_addtest(erts_smp_atomic_t *addp, + erts_aint_t i); +ERTS_GLB_INLINE void erts_smp_atomic_add(erts_smp_atomic_t *addp, + erts_aint_t i); +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_xchg(erts_smp_atomic_t *xchgp, + erts_aint_t new); +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_cmpxchg(erts_smp_atomic_t *xchgp, + erts_aint_t new, + erts_aint_t expected); +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_bor(erts_smp_atomic_t *var, + erts_aint_t mask); +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_band(erts_smp_atomic_t *var, + erts_aint_t mask); +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_read_acqb(erts_smp_atomic_t *var); +ERTS_GLB_INLINE void erts_smp_atomic_set_relb(erts_smp_atomic_t *var, + erts_aint_t i); ERTS_GLB_INLINE void erts_smp_atomic_dec_relb(erts_smp_atomic_t *decp); -ERTS_GLB_INLINE long erts_smp_atomic_dectest_relb(erts_smp_atomic_t *decp); -ERTS_GLB_INLINE long erts_smp_atomic_cmpxchg_acqb(erts_smp_atomic_t *xchgp, - long new, - long exp); -ERTS_GLB_INLINE long erts_smp_atomic_cmpxchg_relb(erts_smp_atomic_t *xchgp, - long new, - long exp); +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_dectest_relb(erts_smp_atomic_t *decp); +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_cmpxchg_acqb(erts_smp_atomic_t *xchgp, + erts_aint_t new, + erts_aint_t exp); +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_cmpxchg_relb(erts_smp_atomic_t *xchgp, + erts_aint_t new, + erts_aint_t exp); +ERTS_GLB_INLINE void +erts_smp_atomic32_init(erts_smp_atomic32_t *var, erts_aint32_t i); +ERTS_GLB_INLINE void +erts_smp_atomic32_set(erts_smp_atomic32_t *var, erts_aint32_t i); +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_read(erts_smp_atomic32_t *var); +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_inctest(erts_smp_atomic32_t *incp); +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_dectest(erts_smp_atomic32_t *decp); +ERTS_GLB_INLINE void +erts_smp_atomic32_inc(erts_smp_atomic32_t *incp); +ERTS_GLB_INLINE void +erts_smp_atomic32_dec(erts_smp_atomic32_t *decp); +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_addtest(erts_smp_atomic32_t *addp, erts_aint32_t i); +ERTS_GLB_INLINE void +erts_smp_atomic32_add(erts_smp_atomic32_t *addp, erts_aint32_t i); +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_xchg(erts_smp_atomic32_t *xchgp, erts_aint32_t new); +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_cmpxchg(erts_smp_atomic32_t *xchgp, + erts_aint32_t new, + erts_aint32_t expected); +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_bor(erts_smp_atomic32_t *var, erts_aint32_t mask); +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_band(erts_smp_atomic32_t *var, erts_aint32_t mask); +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_read_acqb(erts_smp_atomic32_t *var); +ERTS_GLB_INLINE void +erts_smp_atomic32_set_relb(erts_smp_atomic32_t *var, erts_aint32_t i); +ERTS_GLB_INLINE void +erts_smp_atomic32_dec_relb(erts_smp_atomic32_t *decp); +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_dectest_relb(erts_smp_atomic32_t *decp); +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_cmpxchg_acqb(erts_smp_atomic32_t *xchgp, + erts_aint32_t new, + erts_aint32_t exp); +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_cmpxchg_relb(erts_smp_atomic32_t *xchgp, + erts_aint32_t new, + erts_aint32_t exp); ERTS_GLB_INLINE void erts_smp_spinlock_init_x(erts_smp_spinlock_t *lock, char *name, Eterm extra); @@ -221,7 +266,6 @@ ERTS_GLB_INLINE void erts_smp_write_lock(erts_smp_rwlock_t *lock); ERTS_GLB_INLINE void erts_smp_write_unlock(erts_smp_rwlock_t *lock); ERTS_GLB_INLINE int erts_smp_lc_rwlock_is_rlocked(erts_smp_rwlock_t *lock); ERTS_GLB_INLINE int erts_smp_lc_rwlock_is_rwlocked(erts_smp_rwlock_t *lock); -ERTS_GLB_INLINE void erts_smp_thr_time_now(erts_smp_thr_timeval_t *time); ERTS_GLB_INLINE void erts_smp_tsd_key_create(erts_smp_tsd_key_t *keyp); ERTS_GLB_INLINE void erts_smp_tsd_key_delete(erts_smp_tsd_key_t key); ERTS_GLB_INLINE void erts_smp_tsd_set(erts_smp_tsd_key_t key, void *value); @@ -611,7 +655,7 @@ erts_smp_lc_rwmtx_is_rwlocked(erts_smp_rwmtx_t *mtx) } ERTS_GLB_INLINE void -erts_smp_atomic_init(erts_smp_atomic_t *var, long i) +erts_smp_atomic_init(erts_smp_atomic_t *var, erts_aint_t i) { #ifdef ERTS_SMP erts_atomic_init(var, i); @@ -621,7 +665,7 @@ erts_smp_atomic_init(erts_smp_atomic_t *var, long i) } ERTS_GLB_INLINE void -erts_smp_atomic_set(erts_smp_atomic_t *var, long i) +erts_smp_atomic_set(erts_smp_atomic_t *var, erts_aint_t i) { #ifdef ERTS_SMP erts_atomic_set(var, i); @@ -630,7 +674,7 @@ erts_smp_atomic_set(erts_smp_atomic_t *var, long i) #endif } -ERTS_GLB_INLINE long +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_read(erts_smp_atomic_t *var) { #ifdef ERTS_SMP @@ -640,7 +684,7 @@ erts_smp_atomic_read(erts_smp_atomic_t *var) #endif } -ERTS_GLB_INLINE long +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_inctest(erts_smp_atomic_t *incp) { #ifdef ERTS_SMP @@ -650,7 +694,7 @@ erts_smp_atomic_inctest(erts_smp_atomic_t *incp) #endif } -ERTS_GLB_INLINE long +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_dectest(erts_smp_atomic_t *decp) { #ifdef ERTS_SMP @@ -680,8 +724,8 @@ erts_smp_atomic_dec(erts_smp_atomic_t *decp) #endif } -ERTS_GLB_INLINE long -erts_smp_atomic_addtest(erts_smp_atomic_t *addp, long i) +ERTS_GLB_INLINE erts_aint_t +erts_smp_atomic_addtest(erts_smp_atomic_t *addp, erts_aint_t i) { #ifdef ERTS_SMP return erts_atomic_addtest(addp, i); @@ -691,7 +735,7 @@ erts_smp_atomic_addtest(erts_smp_atomic_t *addp, long i) } ERTS_GLB_INLINE void -erts_smp_atomic_add(erts_smp_atomic_t *addp, long i) +erts_smp_atomic_add(erts_smp_atomic_t *addp, erts_aint_t i) { #ifdef ERTS_SMP erts_atomic_add(addp, i); @@ -700,59 +744,61 @@ erts_smp_atomic_add(erts_smp_atomic_t *addp, long i) #endif } -ERTS_GLB_INLINE long -erts_smp_atomic_xchg(erts_smp_atomic_t *xchgp, long new) +ERTS_GLB_INLINE erts_aint_t +erts_smp_atomic_xchg(erts_smp_atomic_t *xchgp, erts_aint_t new) { #ifdef ERTS_SMP return erts_atomic_xchg(xchgp, new); #else - long old; + erts_aint_t old; old = *xchgp; *xchgp = new; return old; #endif } -ERTS_GLB_INLINE long -erts_smp_atomic_cmpxchg(erts_smp_atomic_t *xchgp, long new, long expected) +ERTS_GLB_INLINE erts_aint_t +erts_smp_atomic_cmpxchg(erts_smp_atomic_t *xchgp, + erts_aint_t new, + erts_aint_t expected) { #ifdef ERTS_SMP return erts_atomic_cmpxchg(xchgp, new, expected); #else - long old = *xchgp; + erts_aint_t old = *xchgp; if (old == expected) *xchgp = new; return old; #endif } -ERTS_GLB_INLINE long -erts_smp_atomic_bor(erts_smp_atomic_t *var, long mask) +ERTS_GLB_INLINE erts_aint_t +erts_smp_atomic_bor(erts_smp_atomic_t *var, erts_aint_t mask) { #ifdef ERTS_SMP return erts_atomic_bor(var, mask); #else - long old; + erts_aint_t old; old = *var; *var |= mask; return old; #endif } -ERTS_GLB_INLINE long -erts_smp_atomic_band(erts_smp_atomic_t *var, long mask) +ERTS_GLB_INLINE erts_aint_t +erts_smp_atomic_band(erts_smp_atomic_t *var, erts_aint_t mask) { #ifdef ERTS_SMP return erts_atomic_band(var, mask); #else - long old; + erts_aint_t old; old = *var; *var &= mask; return old; #endif } -ERTS_GLB_INLINE long +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_read_acqb(erts_smp_atomic_t *var) { #ifdef ERTS_SMP @@ -763,7 +809,7 @@ erts_smp_atomic_read_acqb(erts_smp_atomic_t *var) } ERTS_GLB_INLINE void -erts_smp_atomic_set_relb(erts_smp_atomic_t *var, long i) +erts_smp_atomic_set_relb(erts_smp_atomic_t *var, erts_aint_t i) { #ifdef ERTS_SMP erts_atomic_set_relb(var, i); @@ -772,7 +818,8 @@ erts_smp_atomic_set_relb(erts_smp_atomic_t *var, long i) #endif } -ERTS_GLB_INLINE void erts_smp_atomic_dec_relb(erts_smp_atomic_t *decp) +ERTS_GLB_INLINE void +erts_smp_atomic_dec_relb(erts_smp_atomic_t *decp) { #ifdef ERTS_SMP erts_atomic_dec_relb(decp); @@ -781,7 +828,7 @@ ERTS_GLB_INLINE void erts_smp_atomic_dec_relb(erts_smp_atomic_t *decp) #endif } -ERTS_GLB_INLINE long +ERTS_GLB_INLINE erts_aint_t erts_smp_atomic_dectest_relb(erts_smp_atomic_t *decp) { #ifdef ERTS_SMP @@ -791,28 +838,244 @@ erts_smp_atomic_dectest_relb(erts_smp_atomic_t *decp) #endif } -ERTS_GLB_INLINE long erts_smp_atomic_cmpxchg_acqb(erts_smp_atomic_t *xchgp, - long new, - long exp) +ERTS_GLB_INLINE erts_aint_t +erts_smp_atomic_cmpxchg_acqb(erts_smp_atomic_t *xchgp, + erts_aint_t new, + erts_aint_t exp) { #ifdef ERTS_SMP return erts_atomic_cmpxchg_acqb(xchgp, new, exp); #else - long old = *xchgp; + erts_aint_t old = *xchgp; if (old == exp) *xchgp = new; return old; #endif } -ERTS_GLB_INLINE long erts_smp_atomic_cmpxchg_relb(erts_smp_atomic_t *xchgp, - long new, - long exp) +ERTS_GLB_INLINE erts_aint_t +erts_smp_atomic_cmpxchg_relb(erts_smp_atomic_t *xchgp, + erts_aint_t new, + erts_aint_t exp) { #ifdef ERTS_SMP return erts_atomic_cmpxchg_relb(xchgp, new, exp); #else - long old = *xchgp; + erts_aint_t old = *xchgp; + if (old == exp) + *xchgp = new; + return old; +#endif +} + +ERTS_GLB_INLINE void +erts_smp_atomic32_init(erts_smp_atomic32_t *var, erts_aint32_t i) +{ +#ifdef ERTS_SMP + erts_atomic32_init(var, i); +#else + *var = i; +#endif +} + +ERTS_GLB_INLINE void +erts_smp_atomic32_set(erts_smp_atomic32_t *var, erts_aint32_t i) +{ +#ifdef ERTS_SMP + erts_atomic32_set(var, i); +#else + *var = i; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_read(erts_smp_atomic32_t *var) +{ +#ifdef ERTS_SMP + return erts_atomic32_read(var); +#else + return *var; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_inctest(erts_smp_atomic32_t *incp) +{ +#ifdef ERTS_SMP + return erts_atomic32_inctest(incp); +#else + return ++(*incp); +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_dectest(erts_smp_atomic32_t *decp) +{ +#ifdef ERTS_SMP + return erts_atomic32_dectest(decp); +#else + return --(*decp); +#endif +} + +ERTS_GLB_INLINE void +erts_smp_atomic32_inc(erts_smp_atomic32_t *incp) +{ +#ifdef ERTS_SMP + erts_atomic32_inc(incp); +#else + ++(*incp); +#endif +} + +ERTS_GLB_INLINE void +erts_smp_atomic32_dec(erts_smp_atomic32_t *decp) +{ +#ifdef ERTS_SMP + erts_atomic32_dec(decp); +#else + --(*decp); +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_addtest(erts_smp_atomic32_t *addp, erts_aint32_t i) +{ +#ifdef ERTS_SMP + return erts_atomic32_addtest(addp, i); +#else + return *addp += i; +#endif +} + +ERTS_GLB_INLINE void +erts_smp_atomic32_add(erts_smp_atomic32_t *addp, erts_aint32_t i) +{ +#ifdef ERTS_SMP + erts_atomic32_add(addp, i); +#else + *addp += i; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_xchg(erts_smp_atomic32_t *xchgp, erts_aint32_t new) +{ +#ifdef ERTS_SMP + return erts_atomic32_xchg(xchgp, new); +#else + erts_aint32_t old; + old = *xchgp; + *xchgp = new; + return old; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_cmpxchg(erts_smp_atomic32_t *xchgp, + erts_aint32_t new, + erts_aint32_t expected) +{ +#ifdef ERTS_SMP + return erts_atomic32_cmpxchg(xchgp, new, expected); +#else + erts_aint32_t old = *xchgp; + if (old == expected) + *xchgp = new; + return old; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_bor(erts_smp_atomic32_t *var, erts_aint32_t mask) +{ +#ifdef ERTS_SMP + return erts_atomic32_bor(var, mask); +#else + erts_aint32_t old; + old = *var; + *var |= mask; + return old; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_band(erts_smp_atomic32_t *var, erts_aint32_t mask) +{ +#ifdef ERTS_SMP + return erts_atomic32_band(var, mask); +#else + erts_aint32_t old; + old = *var; + *var &= mask; + return old; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_read_acqb(erts_smp_atomic32_t *var) +{ +#ifdef ERTS_SMP + return erts_atomic32_read_acqb(var); +#else + return *var; +#endif +} + +ERTS_GLB_INLINE void +erts_smp_atomic32_set_relb(erts_smp_atomic32_t *var, erts_aint32_t i) +{ +#ifdef ERTS_SMP + erts_atomic32_set_relb(var, i); +#else + *var = i; +#endif +} + +ERTS_GLB_INLINE void +erts_smp_atomic32_dec_relb(erts_smp_atomic32_t *decp) +{ +#ifdef ERTS_SMP + erts_atomic32_dec_relb(decp); +#else + --(*decp); +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_dectest_relb(erts_smp_atomic32_t *decp) +{ +#ifdef ERTS_SMP + return erts_atomic32_dectest_relb(decp); +#else + return --(*decp); +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_cmpxchg_acqb(erts_smp_atomic32_t *xchgp, + erts_aint32_t new, + erts_aint32_t exp) +{ +#ifdef ERTS_SMP + return erts_atomic32_cmpxchg_acqb(xchgp, new, exp); +#else + erts_aint32_t old = *xchgp; + if (old == exp) + *xchgp = new; + return old; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_smp_atomic32_cmpxchg_relb(erts_smp_atomic32_t *xchgp, + erts_aint32_t new, + erts_aint32_t exp) +{ +#ifdef ERTS_SMP + return erts_atomic32_cmpxchg_relb(xchgp, new, exp); +#else + erts_aint32_t old = *xchgp; if (old == exp) *xchgp = new; return old; @@ -988,14 +1251,6 @@ erts_smp_lc_rwlock_is_rwlocked(erts_smp_rwlock_t *lock) } ERTS_GLB_INLINE void -erts_smp_thr_time_now(erts_smp_thr_timeval_t *time) -{ -#ifdef ERTS_SMP - erts_thr_time_now(time); -#endif -} - -ERTS_GLB_INLINE void erts_smp_tsd_key_create(erts_smp_tsd_key_t *keyp) { #ifdef ERTS_SMP diff --git a/erts/emulator/beam/erl_threads.h b/erts/emulator/beam/erl_threads.h index a74cf79b8c..84a20b51f2 100644 --- a/erts/emulator/beam/erl_threads.h +++ b/erts/emulator/beam/erl_threads.h @@ -89,7 +89,10 @@ typedef ethr_rwmutex_opt erts_rwmtx_opt_t; typedef ethr_tsd_key erts_tsd_key_t; typedef ethr_ts_event erts_tse_t; +typedef ethr_sint_t erts_aint_t; typedef ethr_atomic_t erts_atomic_t; +typedef ethr_sint32_t erts_aint32_t; +typedef ethr_atomic32_t erts_atomic32_t; /* spinlock */ typedef struct { @@ -113,7 +116,6 @@ typedef struct { #endif } erts_rwlock_t; -typedef ethr_timeval erts_thr_timeval_t; __decl_noreturn void __noreturn erts_thr_fatal_error(int, char *); /* implemented in erl_init.c */ @@ -152,7 +154,10 @@ typedef struct { typedef int erts_rwmtx_t; typedef int erts_tsd_key_t; typedef int erts_tse_t; -typedef long erts_atomic_t; +typedef SWord erts_aint_t; +typedef SWord erts_atomic_t; +typedef SWord erts_aint32_t; +typedef SWord erts_atomic32_t; #if __GNUC__ > 2 typedef struct { } erts_spinlock_t; typedef struct { } erts_rwlock_t; @@ -160,10 +165,6 @@ typedef struct { } erts_rwlock_t; typedef struct { int gcc_is_buggy; } erts_spinlock_t; typedef struct { int gcc_is_buggy; } erts_rwlock_t; #endif -typedef struct { - long tv_sec; - long tv_nsec; -} erts_thr_timeval_t; #define ERTS_MTX_INITER 0 #define ERTS_CND_INITER 0 @@ -173,6 +174,8 @@ typedef struct { #endif /* #ifdef USE_THREADS */ +#define ERTS_AINT_T_MAX (~(((erts_aint_t) 1) << (sizeof(erts_aint_t)*8-1))) + ERTS_GLB_INLINE void erts_thr_init(erts_thr_init_data_t *id); ERTS_GLB_INLINE void erts_thr_late_init(erts_thr_late_init_data_t *id); ERTS_GLB_INLINE void erts_thr_create(erts_tid_t *tid, void * (*func)(void *), @@ -231,33 +234,65 @@ ERTS_GLB_INLINE int erts_rwmtx_tryrwlock(erts_rwmtx_t *rwmtx); ERTS_GLB_INLINE void erts_rwmtx_rwunlock(erts_rwmtx_t *rwmtx); ERTS_GLB_INLINE int erts_lc_rwmtx_is_rlocked(erts_rwmtx_t *mtx); ERTS_GLB_INLINE int erts_lc_rwmtx_is_rwlocked(erts_rwmtx_t *mtx); -ERTS_GLB_INLINE void erts_atomic_init(erts_atomic_t *var, long i); -ERTS_GLB_INLINE void erts_atomic_set(erts_atomic_t *var, long i); -ERTS_GLB_INLINE long erts_atomic_read(erts_atomic_t *var); -ERTS_GLB_INLINE long erts_atomic_inctest(erts_atomic_t *incp); -ERTS_GLB_INLINE long erts_atomic_dectest(erts_atomic_t *decp); +ERTS_GLB_INLINE void erts_atomic_init(erts_atomic_t *var, erts_aint_t i); +ERTS_GLB_INLINE void erts_atomic_set(erts_atomic_t *var, erts_aint_t i); +ERTS_GLB_INLINE erts_aint_t erts_atomic_read(erts_atomic_t *var); +ERTS_GLB_INLINE erts_aint_t erts_atomic_inctest(erts_atomic_t *incp); +ERTS_GLB_INLINE erts_aint_t erts_atomic_dectest(erts_atomic_t *decp); ERTS_GLB_INLINE void erts_atomic_inc(erts_atomic_t *incp); ERTS_GLB_INLINE void erts_atomic_dec(erts_atomic_t *decp); -ERTS_GLB_INLINE long erts_atomic_addtest(erts_atomic_t *addp, - long i); -ERTS_GLB_INLINE void erts_atomic_add(erts_atomic_t *addp, long i); -ERTS_GLB_INLINE long erts_atomic_xchg(erts_atomic_t *xchgp, - long new); -ERTS_GLB_INLINE long erts_atomic_cmpxchg(erts_atomic_t *xchgp, - long new, - long expected); -ERTS_GLB_INLINE long erts_atomic_bor(erts_atomic_t *var, long mask); -ERTS_GLB_INLINE long erts_atomic_band(erts_atomic_t *var, long mask); -ERTS_GLB_INLINE long erts_atomic_read_acqb(erts_atomic_t *var); -ERTS_GLB_INLINE void erts_atomic_set_relb(erts_atomic_t *var, long i); +ERTS_GLB_INLINE erts_aint_t erts_atomic_addtest(erts_atomic_t *addp, + erts_aint_t i); +ERTS_GLB_INLINE void erts_atomic_add(erts_atomic_t *addp, erts_aint_t i); +ERTS_GLB_INLINE erts_aint_t erts_atomic_xchg(erts_atomic_t *xchgp, + erts_aint_t new); +ERTS_GLB_INLINE erts_aint_t erts_atomic_cmpxchg(erts_atomic_t *xchgp, + erts_aint_t new, + erts_aint_t expected); +ERTS_GLB_INLINE erts_aint_t erts_atomic_bor(erts_atomic_t *var, + erts_aint_t mask); +ERTS_GLB_INLINE erts_aint_t erts_atomic_band(erts_atomic_t *var, + erts_aint_t mask); +ERTS_GLB_INLINE erts_aint_t erts_atomic_read_acqb(erts_atomic_t *var); +ERTS_GLB_INLINE void erts_atomic_set_relb(erts_atomic_t *var, erts_aint_t i); ERTS_GLB_INLINE void erts_atomic_dec_relb(erts_atomic_t *decp); -ERTS_GLB_INLINE long erts_atomic_dectest_relb(erts_atomic_t *decp); -ERTS_GLB_INLINE long erts_atomic_cmpxchg_acqb(erts_atomic_t *xchgp, - long new, - long exp); -ERTS_GLB_INLINE long erts_atomic_cmpxchg_relb(erts_atomic_t *xchgp, - long new, - long exp); +ERTS_GLB_INLINE erts_aint_t erts_atomic_dectest_relb(erts_atomic_t *decp); +ERTS_GLB_INLINE erts_aint_t erts_atomic_cmpxchg_acqb(erts_atomic_t *xchgp, + erts_aint_t new, + erts_aint_t exp); +ERTS_GLB_INLINE erts_aint_t erts_atomic_cmpxchg_relb(erts_atomic_t *xchgp, + erts_aint_t new, + erts_aint_t exp); +ERTS_GLB_INLINE void erts_atomic32_init(erts_atomic32_t *var, erts_aint32_t i); +ERTS_GLB_INLINE void erts_atomic32_set(erts_atomic32_t *var, erts_aint32_t i); +ERTS_GLB_INLINE erts_aint32_t erts_atomic32_read(erts_atomic32_t *var); +ERTS_GLB_INLINE erts_aint32_t erts_atomic32_inctest(erts_atomic32_t *incp); +ERTS_GLB_INLINE erts_aint32_t erts_atomic32_dectest(erts_atomic32_t *decp); +ERTS_GLB_INLINE void erts_atomic32_inc(erts_atomic32_t *incp); +ERTS_GLB_INLINE void erts_atomic32_dec(erts_atomic32_t *decp); +ERTS_GLB_INLINE erts_aint32_t erts_atomic32_addtest(erts_atomic32_t *addp, + erts_aint32_t i); +ERTS_GLB_INLINE void erts_atomic32_add(erts_atomic32_t *addp, erts_aint32_t i); +ERTS_GLB_INLINE erts_aint32_t erts_atomic32_xchg(erts_atomic32_t *xchgp, + erts_aint32_t new); +ERTS_GLB_INLINE erts_aint32_t erts_atomic32_cmpxchg(erts_atomic32_t *xchgp, + erts_aint32_t new, + erts_aint32_t expected); +ERTS_GLB_INLINE erts_aint32_t erts_atomic32_bor(erts_atomic32_t *var, + erts_aint32_t mask); +ERTS_GLB_INLINE erts_aint32_t erts_atomic32_band(erts_atomic32_t *var, + erts_aint32_t mask); +ERTS_GLB_INLINE erts_aint32_t erts_atomic32_read_acqb(erts_atomic32_t *var); +ERTS_GLB_INLINE void erts_atomic32_set_relb(erts_atomic32_t *var, + erts_aint32_t i); +ERTS_GLB_INLINE void erts_atomic32_dec_relb(erts_atomic32_t *decp); +ERTS_GLB_INLINE erts_aint32_t erts_atomic32_dectest_relb(erts_atomic32_t *decp); +ERTS_GLB_INLINE erts_aint32_t erts_atomic32_cmpxchg_acqb(erts_atomic32_t *xchgp, + erts_aint32_t new, + erts_aint32_t exp); +ERTS_GLB_INLINE erts_aint32_t erts_atomic32_cmpxchg_relb(erts_atomic32_t *xchgp, + erts_aint32_t new, + erts_aint32_t exp); ERTS_GLB_INLINE void erts_spinlock_init_x_opt(erts_spinlock_t *lock, char *name, Eterm extra, @@ -292,7 +327,6 @@ ERTS_GLB_INLINE void erts_write_lock(erts_rwlock_t *lock); ERTS_GLB_INLINE void erts_write_unlock(erts_rwlock_t *lock); ERTS_GLB_INLINE int erts_lc_rwlock_is_rlocked(erts_rwlock_t *lock); ERTS_GLB_INLINE int erts_lc_rwlock_is_rwlocked(erts_rwlock_t *lock); -ERTS_GLB_INLINE void erts_thr_time_now(erts_thr_timeval_t *time); ERTS_GLB_INLINE void erts_tsd_key_create(erts_tsd_key_t *keyp); ERTS_GLB_INLINE void erts_tsd_key_delete(erts_tsd_key_t key); ERTS_GLB_INLINE void erts_tsd_set(erts_tsd_key_t key, void *value); @@ -925,7 +959,7 @@ erts_lc_rwmtx_is_rwlocked(erts_rwmtx_t *mtx) } ERTS_GLB_INLINE void -erts_atomic_init(erts_atomic_t *var, long i) +erts_atomic_init(erts_atomic_t *var, erts_aint_t i) { #ifdef USE_THREADS ethr_atomic_init(var, i); @@ -935,7 +969,7 @@ erts_atomic_init(erts_atomic_t *var, long i) } ERTS_GLB_INLINE void -erts_atomic_set(erts_atomic_t *var, long i) +erts_atomic_set(erts_atomic_t *var, erts_aint_t i) { #ifdef USE_THREADS ethr_atomic_set(var, i); @@ -944,7 +978,7 @@ erts_atomic_set(erts_atomic_t *var, long i) #endif } -ERTS_GLB_INLINE long +ERTS_GLB_INLINE erts_aint_t erts_atomic_read(erts_atomic_t *var) { #ifdef USE_THREADS @@ -954,7 +988,7 @@ erts_atomic_read(erts_atomic_t *var) #endif } -ERTS_GLB_INLINE long +ERTS_GLB_INLINE erts_aint_t erts_atomic_inctest(erts_atomic_t *incp) { #ifdef USE_THREADS @@ -964,7 +998,7 @@ erts_atomic_inctest(erts_atomic_t *incp) #endif } -ERTS_GLB_INLINE long +ERTS_GLB_INLINE erts_aint_t erts_atomic_dectest(erts_atomic_t *decp) { #ifdef USE_THREADS @@ -994,8 +1028,8 @@ erts_atomic_dec(erts_atomic_t *decp) #endif } -ERTS_GLB_INLINE long -erts_atomic_addtest(erts_atomic_t *addp, long i) +ERTS_GLB_INLINE erts_aint_t +erts_atomic_addtest(erts_atomic_t *addp, erts_aint_t i) { #ifdef USE_THREADS return ethr_atomic_add_read(addp, i); @@ -1005,7 +1039,7 @@ erts_atomic_addtest(erts_atomic_t *addp, long i) } ERTS_GLB_INLINE void -erts_atomic_add(erts_atomic_t *addp, long i) +erts_atomic_add(erts_atomic_t *addp, erts_aint_t i) { #ifdef USE_THREADS ethr_atomic_add(addp, i); @@ -1014,59 +1048,58 @@ erts_atomic_add(erts_atomic_t *addp, long i) #endif } -ERTS_GLB_INLINE long -erts_atomic_xchg(erts_atomic_t *xchgp, long new) +ERTS_GLB_INLINE erts_aint_t +erts_atomic_xchg(erts_atomic_t *xchgp, erts_aint_t new) { - long old; #ifdef USE_THREADS return ethr_atomic_xchg(xchgp, new); #else - old = *xchgp; + erts_aint_t old = *xchgp; *xchgp = new; -#endif return old; +#endif } -ERTS_GLB_INLINE long -erts_atomic_cmpxchg(erts_atomic_t *xchgp, long new, long expected) +ERTS_GLB_INLINE erts_aint_t +erts_atomic_cmpxchg(erts_atomic_t *xchgp, erts_aint_t new, erts_aint_t expected) { #ifdef USE_THREADS return ethr_atomic_cmpxchg(xchgp, new, expected); #else - long old = *xchgp; + erts_aint_t old = *xchgp; if (old == expected) *xchgp = new; return old; #endif } -ERTS_GLB_INLINE long -erts_atomic_bor(erts_atomic_t *var, long mask) +ERTS_GLB_INLINE erts_aint_t +erts_atomic_bor(erts_atomic_t *var, erts_aint_t mask) { #ifdef USE_THREADS return ethr_atomic_read_bor(var, mask); #else - long old; + erts_aint_t old; old = *var; *var |= mask; return old; #endif } -ERTS_GLB_INLINE long -erts_atomic_band(erts_atomic_t *var, long mask) +ERTS_GLB_INLINE erts_aint_t +erts_atomic_band(erts_atomic_t *var, erts_aint_t mask) { #ifdef USE_THREADS return ethr_atomic_read_band(var, mask); #else - long old; + erts_aint_t old; old = *var; *var &= mask; return old; #endif } -ERTS_GLB_INLINE long +ERTS_GLB_INLINE erts_aint_t erts_atomic_read_acqb(erts_atomic_t *var) { #ifdef USE_THREADS @@ -1077,7 +1110,7 @@ erts_atomic_read_acqb(erts_atomic_t *var) } ERTS_GLB_INLINE void -erts_atomic_set_relb(erts_atomic_t *var, long i) +erts_atomic_set_relb(erts_atomic_t *var, erts_aint_t i) { #ifdef USE_THREADS ethr_atomic_set_relb(var, i); @@ -1096,7 +1129,7 @@ erts_atomic_dec_relb(erts_atomic_t *decp) #endif } -ERTS_GLB_INLINE long +ERTS_GLB_INLINE erts_aint_t erts_atomic_dectest_relb(erts_atomic_t *decp) { #ifdef USE_THREADS @@ -1106,28 +1139,243 @@ erts_atomic_dectest_relb(erts_atomic_t *decp) #endif } -ERTS_GLB_INLINE long erts_atomic_cmpxchg_acqb(erts_atomic_t *xchgp, - long new, - long exp) +ERTS_GLB_INLINE erts_aint_t erts_atomic_cmpxchg_acqb(erts_atomic_t *xchgp, + erts_aint_t new, + erts_aint_t exp) { #ifdef USE_THREADS return ethr_atomic_cmpxchg_acqb(xchgp, new, exp); #else - long old = *xchgp; + erts_aint_t old = *xchgp; if (old == exp) *xchgp = new; return old; #endif } -ERTS_GLB_INLINE long erts_atomic_cmpxchg_relb(erts_atomic_t *xchgp, - long new, - long exp) +ERTS_GLB_INLINE erts_aint_t erts_atomic_cmpxchg_relb(erts_atomic_t *xchgp, + erts_aint_t new, + erts_aint_t exp) { #ifdef USE_THREADS return ethr_atomic_cmpxchg_relb(xchgp, new, exp); #else - long old = *xchgp; + erts_aint_t old = *xchgp; + if (old == exp) + *xchgp = new; + return old; +#endif +} + +/* atomic32 */ + +ERTS_GLB_INLINE void +erts_atomic32_init(erts_atomic32_t *var, erts_aint32_t i) +{ +#ifdef USE_THREADS + ethr_atomic32_init(var, i); +#else + *var = i; +#endif +} + +ERTS_GLB_INLINE void +erts_atomic32_set(erts_atomic32_t *var, erts_aint32_t i) +{ +#ifdef USE_THREADS + ethr_atomic32_set(var, i); +#else + *var = i; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read(erts_atomic32_t *var) +{ +#ifdef USE_THREADS + return ethr_atomic32_read(var); +#else + return *var; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_inctest(erts_atomic32_t *incp) +{ +#ifdef USE_THREADS + return ethr_atomic32_inc_read(incp); +#else + return ++(*incp); +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_dectest(erts_atomic32_t *decp) +{ +#ifdef USE_THREADS + return ethr_atomic32_dec_read(decp); +#else + return --(*decp); +#endif +} + +ERTS_GLB_INLINE void +erts_atomic32_inc(erts_atomic32_t *incp) +{ +#ifdef USE_THREADS + ethr_atomic32_inc(incp); +#else + ++(*incp); +#endif +} + +ERTS_GLB_INLINE void +erts_atomic32_dec(erts_atomic32_t *decp) +{ +#ifdef USE_THREADS + ethr_atomic32_dec(decp); +#else + --(*decp); +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_addtest(erts_atomic32_t *addp, erts_aint32_t i) +{ +#ifdef USE_THREADS + return ethr_atomic32_add_read(addp, i); +#else + return *addp += i; +#endif +} + +ERTS_GLB_INLINE void +erts_atomic32_add(erts_atomic32_t *addp, erts_aint32_t i) +{ +#ifdef USE_THREADS + ethr_atomic32_add(addp, i); +#else + *addp += i; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_xchg(erts_atomic32_t *xchgp, erts_aint32_t new) +{ +#ifdef USE_THREADS + return ethr_atomic32_xchg(xchgp, new); +#else + erts_aint32_t old = *xchgp; + *xchgp = new; + return old; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_cmpxchg(erts_atomic32_t *xchgp, + erts_aint32_t new, + erts_aint32_t expected) +{ +#ifdef USE_THREADS + return ethr_atomic32_cmpxchg(xchgp, new, expected); +#else + erts_aint32_t old = *xchgp; + if (old == expected) + *xchgp = new; + return old; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_bor(erts_atomic32_t *var, erts_aint32_t mask) +{ +#ifdef USE_THREADS + return ethr_atomic32_read_bor(var, mask); +#else + erts_aint32_t old; + old = *var; + *var |= mask; + return old; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_band(erts_atomic32_t *var, erts_aint32_t mask) +{ +#ifdef USE_THREADS + return ethr_atomic32_read_band(var, mask); +#else + erts_aint32_t old; + old = *var; + *var &= mask; + return old; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_read_acqb(erts_atomic32_t *var) +{ +#ifdef USE_THREADS + return ethr_atomic32_read_acqb(var); +#else + return *var; +#endif +} + +ERTS_GLB_INLINE void +erts_atomic32_set_relb(erts_atomic32_t *var, erts_aint32_t i) +{ +#ifdef USE_THREADS + ethr_atomic32_set_relb(var, i); +#else + *var = i; +#endif +} + +ERTS_GLB_INLINE void +erts_atomic32_dec_relb(erts_atomic32_t *decp) +{ +#ifdef USE_THREADS + ethr_atomic32_dec_relb(decp); +#else + --(*decp); +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_dectest_relb(erts_atomic32_t *decp) +{ +#ifdef USE_THREADS + return ethr_atomic32_dec_read_relb(decp); +#else + return --(*decp); +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_cmpxchg_acqb(erts_atomic32_t *xchgp, + erts_aint32_t new, + erts_aint32_t exp) +{ +#ifdef USE_THREADS + return ethr_atomic32_cmpxchg_acqb(xchgp, new, exp); +#else + erts_aint32_t old = *xchgp; + if (old == exp) + *xchgp = new; + return old; +#endif +} + +ERTS_GLB_INLINE erts_aint32_t +erts_atomic32_cmpxchg_relb(erts_atomic32_t *xchgp, + erts_aint32_t new, + erts_aint32_t exp) +{ +#ifdef USE_THREADS + return ethr_atomic32_cmpxchg_relb(xchgp, new, exp); +#else + erts_aint32_t old = *xchgp; if (old == exp) *xchgp = new; return old; @@ -1428,16 +1676,6 @@ erts_lc_rwlock_is_rwlocked(erts_rwlock_t *lock) } ERTS_GLB_INLINE void -erts_thr_time_now(erts_thr_timeval_t *time) -{ -#ifdef USE_THREADS - int res = ethr_time_now(time); - if (res) - erts_thr_fatal_error(res, "get current time"); -#endif -} - -ERTS_GLB_INLINE void erts_tsd_key_create(erts_tsd_key_t *keyp) { #ifdef USE_THREADS diff --git a/erts/emulator/beam/erl_unicode.c b/erts/emulator/beam/erl_unicode.c index d01a3661f9..545b345a71 100644 --- a/erts/emulator/beam/erl_unicode.c +++ b/erts/emulator/beam/erl_unicode.c @@ -30,6 +30,8 @@ #include "big.h" #include "erl_unicode.h" +#include "erl_unicode_normalize.h" + typedef struct _restart_context { byte *bytes; @@ -54,13 +56,6 @@ static BIF_RETTYPE finalize_list_to_list(Process *p, Uint num_resulting_chars, int state, int left, Eterm tail); -static int analyze_utf8(byte *source, Uint size, - byte **err_pos, Uint *num_chars, int *left); -#define UTF8_OK 0 -#define UTF8_INCOMPLETE 1 -#define UTF8_ERROR 2 -#define UTF8_ANALYZE_MORE 3 - static BIF_RETTYPE characters_to_utf8_trap(BIF_ALIST_3); static BIF_RETTYPE characters_to_list_trap_1(BIF_ALIST_3); static BIF_RETTYPE characters_to_list_trap_2(BIF_ALIST_3); @@ -463,7 +458,7 @@ L_Again: /* Restart with sublist, old listend was pushed on stack */ } objp = list_val(ioterm); obj = CAR(objp); - if (!is_byte(obj)) + if (!is_small(obj)) break; } } else if (is_nil(obj)) { @@ -970,11 +965,11 @@ static int is_valid_utf8(Eterm orig_bin) bytes = erts_get_aligned_binary_bytes(orig_bin, &temp_alloc); } size = binary_size(orig_bin); - ret = analyze_utf8(bytes, + ret = erts_analyze_utf8(bytes, size, &endpos,&numchar,NULL); erts_free_aligned_binary_bytes(temp_alloc); - return (ret == UTF8_OK); + return (ret == ERTS_UTF8_OK); } BIF_RETTYPE unicode_characters_to_binary_2(BIF_ALIST_2) @@ -1084,14 +1079,14 @@ static BIF_RETTYPE build_list_return(Process *p, byte *bytes, int pos, Uint char hp += 2; rest_term = CONS(hp,leftover_bin,rest_term); } - BIF_RET(finalize_list_to_list(p, bytes, rest_term, 0U, pos, characters, UTF8_ERROR, left, NIL)); + BIF_RET(finalize_list_to_list(p, bytes, rest_term, 0U, pos, characters, ERTS_UTF8_ERROR, left, NIL)); } else if (rest_term == NIL && num_leftovers != 0) { Eterm leftover_bin = new_binary(p, leftover, num_leftovers); if (check_leftovers(leftover,num_leftovers) != 0) { - BIF_RET(finalize_list_to_list(p, bytes, leftover_bin, 0U, pos, characters, UTF8_ERROR, + BIF_RET(finalize_list_to_list(p, bytes, leftover_bin, 0U, pos, characters, ERTS_UTF8_ERROR, left, NIL)); } else { - BIF_RET(finalize_list_to_list(p, bytes, leftover_bin, 0U, pos, characters, UTF8_INCOMPLETE, + BIF_RET(finalize_list_to_list(p, bytes, leftover_bin, 0U, pos, characters, ERTS_UTF8_INCOMPLETE, left, NIL)); } } else { /* All OK */ @@ -1107,11 +1102,11 @@ static BIF_RETTYPE build_list_return(Process *p, byte *bytes, int pos, Uint char rc.num_processed_bytes = 0; /* not used */ rc.num_bytes_to_process = pos; rc.num_resulting_chars = characters; - rc.state = UTF8_OK; /* not used */ + rc.state = ERTS_UTF8_OK; /* not used */ BIF_TRAP3(&characters_to_list_trap_1_exp, p, make_magic_bin_for_restart(p,&rc), rest_term, latin1); } else { /* Success */ - BIF_RET(finalize_list_to_list(p, bytes, NIL, 0U, pos, characters, UTF8_OK, left, NIL)); + BIF_RET(finalize_list_to_list(p, bytes, NIL, 0U, pos, characters, ERTS_UTF8_OK, left, NIL)); } } } @@ -1205,7 +1200,7 @@ BIF_RETTYPE unicode_characters_to_list_2(BIF_ALIST_2) * When input to characters_to_list is a plain binary and the format is 'unicode', we do * a faster analyze and size count with this function. */ -static int analyze_utf8(byte *source, Uint size, +int erts_analyze_utf8(byte *source, Uint size, byte **err_pos, Uint *num_chars, int *left) { *err_pos = source; @@ -1216,60 +1211,60 @@ static int analyze_utf8(byte *source, Uint size, --size; } else if (((*source) & ((byte) 0xE0)) == 0xC0) { if (size < 2) { - return UTF8_INCOMPLETE; + return ERTS_UTF8_INCOMPLETE; } if (((source[1] & ((byte) 0xC0)) != 0x80) || ((*source) < 0xC2) /* overlong */) { - return UTF8_ERROR; + return ERTS_UTF8_ERROR; } source += 2; size -= 2; } else if (((*source) & ((byte) 0xF0)) == 0xE0) { if (size < 3) { - return UTF8_INCOMPLETE; + return ERTS_UTF8_INCOMPLETE; } if (((source[1] & ((byte) 0xC0)) != 0x80) || ((source[2] & ((byte) 0xC0)) != 0x80) || (((*source) == 0xE0) && (source[1] < 0xA0)) /* overlong */ ) { - return UTF8_ERROR; + return ERTS_UTF8_ERROR; } if ((((*source) & ((byte) 0xF)) == 0xD) && ((source[1] & 0x20) != 0)) { - return UTF8_ERROR; + return ERTS_UTF8_ERROR; } if (((*source) == 0xEF) && (source[1] == 0xBF) && ((source[2] == 0xBE) || (source[2] == 0xBF))) { - return UTF8_ERROR; + return ERTS_UTF8_ERROR; } source += 3; size -= 3; } else if (((*source) & ((byte) 0xF8)) == 0xF0) { if (size < 4) { - return UTF8_INCOMPLETE; + return ERTS_UTF8_INCOMPLETE; } if (((source[1] & ((byte) 0xC0)) != 0x80) || ((source[2] & ((byte) 0xC0)) != 0x80) || ((source[3] & ((byte) 0xC0)) != 0x80) || (((*source) == 0xF0) && (source[1] < 0x90)) /* overlong */) { - return UTF8_ERROR; + return ERTS_UTF8_ERROR; } if ((((*source) & ((byte)0x7)) > 0x4U) || ((((*source) & ((byte)0x7)) == 0x4U) && ((source[1] & ((byte)0x3F)) > 0xFU))) { - return UTF8_ERROR; + return ERTS_UTF8_ERROR; } source += 4; size -= 4; } else { - return UTF8_ERROR; + return ERTS_UTF8_ERROR; } ++(*num_chars); *err_pos = source; if (left && --(*left) <= 0) { - return UTF8_ANALYZE_MORE; + return ERTS_UTF8_ANALYZE_MORE; } } - return UTF8_OK; + return ERTS_UTF8_OK; } /* @@ -1304,7 +1299,7 @@ static Eterm do_utf8_to_list(Process *p, Uint num, byte *bytes, Uint sz, } else if (((*source) & ((byte) 0xE0)) == 0xC0) { unipoint = (((Uint) ((*source) & ((byte) 0x1F))) << 6) | - ((Uint) (source[1] & ((byte) 0x3F))); + ((Uint) (source[1] & ((byte) 0x3F))); } else if (((*source) & ((byte) 0xF0)) == 0xE0) { unipoint = (((Uint) ((*source) & ((byte) 0xF))) << 12) | @@ -1330,6 +1325,216 @@ static Eterm do_utf8_to_list(Process *p, Uint num, byte *bytes, Uint sz, return ret; } +static int is_candidate(Uint cp) +{ + int index,pos; + if (cp < 768) return 0; + if (cp > 4023) { + if (cp == 12441 || cp == 12442) return 1; + return 0; + } + index = cp / 32 - COMP_CANDIDATE_MAP_OFFSET; + pos = cp % 32; + return !!(comp_candidate_map[index] & (1UL << pos)); +} + +static int hashsearch(int *htab, int htab_size, CompEntry *cv, Uint16 c) +{ + int bucket = c % htab_size; + while (htab[bucket] != -1 && cv[htab[bucket]].c != c) + bucket = (bucket + 1) % htab_size; + return htab[bucket]; +} + +#define TRANSLATE_NO 0 +#define TRANSLATE_MAYBE -1 + +/* The s array is reversed */ +static int translate(Uint16 *s, int slen, Uint16 *res) +{ + /* Go backwards through buffer and match against tree */ + int pos = 0; + CompEntry *cv = compose_tab; + int *hc = hash_compose_tab; + int cvs = compose_tab_size; + int x; + while (pos < slen) { + x = hashsearch(hc,cvs*HASH_SIZE_FACTOR,cv,s[pos]); + if (x < 0) { + return TRANSLATE_NO; + } + if (cv[x].res) { + *res = cv[x].res; + return pos; + } + cvs = cv[x].num_subs; + hc = cv[x].hash; + cv = cv[x].subs; + ++pos; + } + return TRANSLATE_MAYBE; +} + +static void handle_first_norm(Uint16 *savepoints, int *numpointsp, Uint unipoint) +{ + /*erts_fprintf(stderr,"CP = %d, numpoints = %d\n",(int) unipoint,(int) *numpointsp);*/ + *numpointsp = 1; + savepoints[0] = (Uint16) unipoint; +} + +static void cleanup_norm(Eterm **hpp, Uint16 *savepoints, int numpoints, Eterm *retp) +{ + Eterm *hp = *hpp; + int res,i; + Uint16 newpoint; + Eterm ret = *retp; + + ret = CONS(hp,make_small((Uint) savepoints[0]),ret); + hp += 2; + + for (i = 1;i < numpoints;) { + if(!is_candidate(savepoints[i]) || + ((res = translate(savepoints+i,numpoints - i, &newpoint)) <= 0)) { + ret = CONS(hp,make_small((Uint) savepoints[i]),ret); + hp += 2; + ++i; + } else { + ret = CONS(hp,make_small((Uint) newpoint),ret); + hp += 2; + i += res; + } + } + *retp = ret; +} + +static void handle_potential_norm(Eterm **hpp, Uint16 *savepoints, int *numpointsp, Uint unipoint, Eterm *retp) +{ + Eterm *hp = *hpp; + int numpoints = *numpointsp; + int res,i; + Uint16 newpoint; + Eterm ret = *retp; + + /* erts_fprintf(stderr,"CP = %d, numpoints = %d\n",(int) unipoint,(int) numpoints);*/ + if ((unipoint >> 16) == 0) { /* otherwise we're done here */ + savepoints[numpoints++] = (Uint16) unipoint; + res = translate(savepoints,numpoints,&newpoint); + if (res == TRANSLATE_NO) { + ret = CONS(hp,make_small((Uint) savepoints[0]),ret); + hp += 2; + for (i = 1;i < numpoints;) { + if(!is_candidate(savepoints[i]) || + ((res = translate(savepoints+i,numpoints - i, &newpoint)) == 0)) { + ret = CONS(hp,make_small((Uint) savepoints[i]),ret); + hp += 2; + ++i; + } else if (res > 0) { + ret = CONS(hp,make_small((Uint) newpoint),ret); + hp += 2; + i += res; + } else { /* res < 0 */ + /* A "maybe", means we are not done yet */ + int j = 0; + while (i < numpoints) { + savepoints[j++] = savepoints[i++]; + } + numpoints = j; + goto breakaway; + } + } + numpoints = 0; + breakaway: + ; + } else if (res > 0) { + numpoints = 0; + ret = CONS(hp,make_small((Uint) newpoint),ret); + hp += 2; + } /* < 0 means go on */ + } else { + /* Unconditional rollup, this character is larger than 16 bit */ + ret = CONS(hp,make_small((Uint) savepoints[0]),ret); + hp += 2; + + for (i = 1;i < numpoints;) { + if(!is_candidate(savepoints[i]) || + ((res = translate(savepoints+i,numpoints - i, &newpoint)) <= 0)) { + ret = CONS(hp,make_small((Uint) savepoints[i]),ret); + hp += 2; + ++i; + } else { + ret = CONS(hp,make_small((Uint) newpoint),ret); + hp += 2; + i += res; + } + } + ret = CONS(hp,make_small(unipoint),ret); + hp += 2; + numpoints = 0; + } + *hpp = hp; + *numpointsp = numpoints; + *retp = ret; +} + +static Eterm do_utf8_to_list_normalize(Process *p, Uint num, byte *bytes, Uint sz) +{ + Eterm *hp,*hp_end; + Eterm ret; + byte *source; + Uint unipoint; + Uint16 savepoints[4]; + int numpoints = 0; + + ASSERT(num > 0); + + hp = HAlloc(p,num * 2); /* May be to much */ + hp_end = hp + num * 2; + ret = NIL; + source = bytes + sz; + while(--source >= bytes) { + if (((*source) & ((byte) 0x80)) == 0) { + unipoint = (Uint) *source; + } else if (((*source) & ((byte) 0xE0)) == 0xC0) { + unipoint = + (((Uint) ((*source) & ((byte) 0x1F))) << 6) | + ((Uint) (source[1] & ((byte) 0x3F))); + } else if (((*source) & ((byte) 0xF0)) == 0xE0) { + unipoint = + (((Uint) ((*source) & ((byte) 0xF))) << 12) | + (((Uint) (source[1] & ((byte) 0x3F))) << 6) | + ((Uint) (source[2] & ((byte) 0x3F))); + } else if (((*source) & ((byte) 0xF8)) == 0xF0) { + unipoint = + (((Uint) ((*source) & ((byte) 0x7))) << 18) | + (((Uint) (source[1] & ((byte) 0x3F))) << 12) | + (((Uint) (source[2] & ((byte) 0x3F))) << 6) | + ((Uint) (source[3] & ((byte) 0x3F))); + } else { + /* ignore 2#10XXXXXX */ + continue; + } + if (numpoints) { + handle_potential_norm(&hp,savepoints,&numpoints,unipoint,&ret); + continue; + } + /* We are not building up any normalizations yet, look that we shouldn't start... */ + if (is_candidate(unipoint)) { + handle_first_norm(savepoints,&numpoints,unipoint); + continue; + } + ret = CONS(hp,make_small(unipoint),ret); + hp += 2; + } + /* so, we'we looped to the beginning, do we have anything saved? */ + if (numpoints) { + cleanup_norm(&hp,savepoints,numpoints,&ret); + } + if (hp_end != hp) { + HRelease(p,hp_end,hp); + } + return ret; +} + /* * The last step of characters_to_list, build a list from the buffer 'bytes' (created in the same way * as for characters_to_utf8). All sizes are known in advance and most data will be held in a @@ -1378,10 +1583,10 @@ static BIF_RETTYPE finalize_list_to_list(Process *p, */ free_restart(bytes); - if (state == UTF8_INCOMPLETE) { + if (state == ERTS_UTF8_INCOMPLETE) { hp = HAlloc(p,4); ret = TUPLE3(hp,am_incomplete,converted,rest); - } else if (state == UTF8_ERROR) { + } else if (state == ERTS_UTF8_ERROR) { hp = HAlloc(p,4); ret = TUPLE3(hp,am_error,converted,rest); } else { @@ -1408,7 +1613,7 @@ static BIF_RETTYPE characters_to_list_trap_2(BIF_ALIST_3) /* * Hooks into the process of decoding a binary depending on state. - * If last_state is UTF8_ANALYZE_MORE, num_bytes_to_process + * If last_state is ERTS_UTF8_ANALYZE_MORE, num_bytes_to_process * and num_resulting_chars will grow * until we're done analyzing the binary. Then we'll eat * the bytes to process, lowering num_bytes_to_process and num_resulting_chars, @@ -1465,14 +1670,14 @@ static BIF_RETTYPE do_bif_utf8_to_list(Process *p, left = allowed_iterations(p); - if (state == UTF8_ANALYZE_MORE) { - state = analyze_utf8(bytes + num_bytes_to_process, + if (state == ERTS_UTF8_ANALYZE_MORE) { + state = erts_analyze_utf8(bytes + num_bytes_to_process, size - num_bytes_to_process, &endpos,&numchar,&left); cost_to_proc(p,numchar); num_resulting_chars += numchar; num_bytes_to_process = endpos - bytes; - if (state == UTF8_ANALYZE_MORE) { + if (state == ERTS_UTF8_ANALYZE_MORE) { Eterm epos = erts_make_integer(num_bytes_to_process,p); Eterm enumchar = erts_make_integer(num_resulting_chars,p); erts_free_aligned_binary_bytes(temp_alloc); @@ -1528,7 +1733,7 @@ static BIF_RETTYPE do_bif_utf8_to_list(Process *p, ErlSubBin *sb; Eterm orig; Uint offset; - ASSERT(state != UTF8_OK); + ASSERT(state != ERTS_UTF8_OK); hp = HAlloc(p, ERL_SUB_BIN_SIZE); sb = (ErlSubBin *) hp; ERTS_GET_REAL_BIN(orig_bin, orig, offset, bitoffs, bitsize); @@ -1544,14 +1749,14 @@ static BIF_RETTYPE do_bif_utf8_to_list(Process *p, /* Done */ - if (state == UTF8_INCOMPLETE) { + if (state == ERTS_UTF8_INCOMPLETE) { if (check_leftovers(bytes + num_bytes_to_process + num_processed_bytes, b_sz) != 0) { goto error_return; } hp = HAlloc(p,4); ret = TUPLE3(hp,am_incomplete,converted,rest); - } else if (state == UTF8_ERROR) { + } else if (state == ERTS_UTF8_ERROR) { error_return: hp = HAlloc(p,4); ret = TUPLE3(hp,am_error,converted,rest); @@ -1589,7 +1794,7 @@ static BIF_RETTYPE characters_to_list_trap_3(BIF_ALIST_3) 0U, /* nothing processed yet */ num_bytes_to_process, num_resulting_chars, - UTF8_ANALYZE_MORE, /* always this state here */ + ERTS_UTF8_ANALYZE_MORE, /* always this state here */ NIL); /* Nothing built -> no tail yet */ } @@ -1642,7 +1847,7 @@ static BIF_RETTYPE utf8_to_list(BIF_ALIST_1) BIF_ERROR(BIF_P,BADARG); } return do_bif_utf8_to_list(BIF_P, BIF_ARG_1, 0U, 0U, 0U, - UTF8_ANALYZE_MORE,NIL); + ERTS_UTF8_ANALYZE_MORE,NIL); } @@ -1728,8 +1933,8 @@ binary_to_atom(Process* p, Eterm bin, Eterm enc, int must_exist) Uint n; int reds_left = bin_size+1; /* Number of reductions left. */ - if (analyze_utf8(bytes, bin_size, &err_pos, - &n, &reds_left) == UTF8_OK) { + if (erts_analyze_utf8(bytes, bin_size, &err_pos, + &n, &reds_left) == ERTS_UTF8_OK) { /* * Correct UTF-8 encoding, but too many characters to * fit in an atom. @@ -1813,3 +2018,616 @@ BIF_RETTYPE binary_to_existing_atom_2(BIF_ALIST_2) { return binary_to_atom(BIF_P, BIF_ARG_1, BIF_ARG_2, 1); } + +/********************************************************** + * Simpler non-interruptable routines for UTF-8 and + * Windowish UTF-16 (restricted) + **********************************************************/ +/* + * This function is the heart of the Unicode support for + * open_port - spawn_executable. It converts both the name + * of the executable and the arguments according to the same rules + * as for filename conversion. That means as if your arguments are + * to be raw, you supply binaries, else unicode characters are allowed up to + * the encoding maximum (256 of the unicode max). + * Depending on the filename encoding standard, the vector is then + * converted to whatever is used, which might mean win_utf16 if on windows. + * Do not peek into the argument vector or filenam with ordinary + * string routines, that will certainly fail on some OS. + */ + +char *erts_convert_filename_to_native(Eterm name, ErtsAlcType_t alloc_type, int allow_empty) +{ + int encoding = erts_get_native_filename_encoding(); + char* name_buf = NULL; + + if (is_atom(name) || is_list(name) || (allow_empty && is_nil(name))) { + Sint need; + if ((need = erts_native_filename_need(name,encoding)) < 0) { + return NULL; + } + if (encoding == ERL_FILENAME_WIN_WCHAR) { + need += 2; + } else { + ++need; + } + name_buf = (char *) erts_alloc(alloc_type, need); + erts_native_filename_put(name,encoding,(byte *)name_buf); + name_buf[need-1] = 0; + if (encoding == ERL_FILENAME_WIN_WCHAR) { + name_buf[need-2] = 0; + } + } else if (is_binary(name)) { + byte *temp_alloc = NULL; + byte *bytes; + byte *err_pos; + Uint size,num_chars; + + size = binary_size(name); + bytes = erts_get_aligned_binary_bytes(name, &temp_alloc); + if (encoding != ERL_FILENAME_WIN_WCHAR) { + /*Add 0 termination only*/ + name_buf = (char *) erts_alloc(alloc_type, size+1); + memcpy(name_buf,bytes,size); + name_buf[size]=0; + } else if (erts_analyze_utf8(bytes,size,&err_pos,&num_chars,NULL) != ERTS_UTF8_OK || + erts_get_user_requested_filename_encoding() == ERL_FILENAME_LATIN1) { + byte *p; + /* What to do now? Maybe latin1, so just take byte for byte instead */ + name_buf = (char *) erts_alloc(alloc_type, (size+1)*2); + p = (byte *) name_buf; + while (size--) { + *p++ = *bytes++; + *p++ = 0; + } + *p++ = 0; + *p++ = 0; + } else { /* WIN_WCHAR and valid UTF8 */ + name_buf = (char *) erts_alloc(alloc_type, (num_chars+1)*2); + erts_copy_utf8_to_utf16_little((byte *) name_buf, bytes, num_chars); + name_buf[num_chars*2] = 0; + name_buf[num_chars*2+1] = 0; + } + erts_free_aligned_binary_bytes(temp_alloc); + } else { + return NULL; + } + return name_buf; +} + + +Sint erts_native_filename_need(Eterm ioterm, int encoding) +{ + Eterm *objp; + Eterm obj; + DECLARE_ESTACK(stack); + Sint need = 0; + + if (is_atom(ioterm)) { + Atom* ap; + int i; + ap = atom_tab(atom_val(ioterm)); + switch (encoding) { + case ERL_FILENAME_LATIN1: + need = ap->len; + break; + case ERL_FILENAME_UTF8_MAC: + case ERL_FILENAME_UTF8: + for (i = 0; i < ap->len; i++) { + need += (ap->name[i] >= 0x80) ? 2 : 1; + } + break; + case ERL_FILENAME_WIN_WCHAR: + need = 2*(ap->len); + break; + default: + need = -1; + } + DESTROY_ESTACK(stack); + return need; + } + + if (is_nil(ioterm)) { + DESTROY_ESTACK(stack); + return need; + } + if (!is_list(ioterm)) { + DESTROY_ESTACK(stack); + return (Sint) -1; + } + /* OK a list, needs to be processed in order, handling each flat list-level + as they occur, just like io_list_to_binary would */ + ESTACK_PUSH(stack,ioterm); + while (!ESTACK_ISEMPTY(stack)) { + ioterm = ESTACK_POP(stack); + if (is_nil(ioterm)) { + /* ignore empty lists */ + continue; + } + if(is_list(ioterm)) { +L_Again: /* Restart with sublist, old listend was pushed on stack */ + objp = list_val(ioterm); + obj = CAR(objp); + for(;;) { /* loop over one flat list of bytes and binaries + until sublist or list end is encountered */ + if (is_small(obj)) { /* Always small */ + for(;;) { + Uint x = unsigned_val(obj); + switch (encoding) { + case ERL_FILENAME_LATIN1: + if (x > 255) { + DESTROY_ESTACK(stack); + return ((Sint) -1); + } + need += 1; + break; + case ERL_FILENAME_UTF8_MAC: + case ERL_FILENAME_UTF8: + if (x < 0x80) { + need +=1; + } else if (x < 0x800) { + need += 2; + } else if (x < 0x10000) { + if ((x >= 0xD800 && x <= 0xDFFF) || + (x == 0xFFFE) || + (x == 0xFFFF)) { /* Invalid unicode range */ + DESTROY_ESTACK(stack); + return ((Sint) -1); + } + need += 3; + } else if (x < 0x110000) { + need += 4; + } else { + DESTROY_ESTACK(stack); + return ((Sint) -1); + } + break; + case ERL_FILENAME_WIN_WCHAR: + if (x <= 0xffff) { + need += 2; + break; + } /* else fall throug to error */ + default: + DESTROY_ESTACK(stack); + return ((Sint) -1); + } + + /* everything else will give badarg later + in the process, so we dont check */ + ioterm = CDR(objp); + if (!is_list(ioterm)) { + break; + } + objp = list_val(ioterm); + obj = CAR(objp); + if (!is_small(obj)) + break; + } + } else if (is_nil(obj)) { + ioterm = CDR(objp); + if (!is_list(ioterm)) { + break; + } + objp = list_val(ioterm); + obj = CAR(objp); + } else if (is_list(obj)) { + /* push rest of list for later processing, start + again with sublist */ + ESTACK_PUSH(stack,CDR(objp)); + ioterm = obj; + goto L_Again; + } else { + DESTROY_ESTACK(stack); + return ((Sint) -1); + } + if (is_nil(ioterm) || !is_list(ioterm)) { + break; + } + } /* for(;;) */ + } /* is_list(ioterm) */ + + if (!is_list(ioterm) && !is_nil(ioterm)) { + /* inproper list end */ + DESTROY_ESTACK(stack); + return ((Sint) -1); + } + } /* while not estack empty */ + DESTROY_ESTACK(stack); + return need; +} + +void erts_native_filename_put(Eterm ioterm, int encoding, byte *p) +{ + Eterm *objp; + Eterm obj; + DECLARE_ESTACK(stack); + + if (is_atom(ioterm)) { + Atom* ap; + int i; + ap = atom_tab(atom_val(ioterm)); + switch (encoding) { + case ERL_FILENAME_LATIN1: + for (i = 0; i < ap->len; i++) { + *p++ = ap->name[i]; + } + break; + case ERL_FILENAME_UTF8_MAC: + case ERL_FILENAME_UTF8: + for (i = 0; i < ap->len; i++) { + if(ap->name[i] < 0x80) { + *p++ = ap->name[i]; + } else { + *p++ = (((ap->name[i]) >> 6) | ((byte) 0xC0)); + *p++ = (((ap->name[i]) & 0x3F) | ((byte) 0x80)); + } + } + break; + case ERL_FILENAME_WIN_WCHAR: + for (i = 0; i < ap->len; i++) { + /* Little endian */ + *p++ = ap->name[i]; + *p++ = 0; + } + break; + default: + ASSERT(0); + } + DESTROY_ESTACK(stack); + return; + } + + if (is_nil(ioterm)) { + DESTROY_ESTACK(stack); + return; + } + ASSERT(is_list(ioterm)); + /* OK a list, needs to be processed in order, handling each flat list-level + as they occur, just like io_list_to_binary would */ + ESTACK_PUSH(stack,ioterm); + while (!ESTACK_ISEMPTY(stack)) { + ioterm = ESTACK_POP(stack); + if (is_nil(ioterm)) { + /* ignore empty lists */ + continue; + } + if(is_list(ioterm)) { +L_Again: /* Restart with sublist, old listend was pushed on stack */ + objp = list_val(ioterm); + obj = CAR(objp); + for(;;) { /* loop over one flat list of bytes and binaries + until sublist or list end is encountered */ + if (is_small(obj)) { /* Always small */ + for(;;) { + Uint x = unsigned_val(obj); + switch (encoding) { + case ERL_FILENAME_LATIN1: + ASSERT( x < 256); + *p++ = (byte) x; + break; + case ERL_FILENAME_UTF8_MAC: + case ERL_FILENAME_UTF8: + if (x < 0x80) { + *p++ = (byte) x; + } + else if (x < 0x800) { + *p++ = (((byte) (x >> 6)) | + ((byte) 0xC0)); + *p++ = (((byte) (x & 0x3F)) | + ((byte) 0x80)); + } else if (x < 0x10000) { + ASSERT(!((x >= 0xD800 && x <= 0xDFFF) || + (x == 0xFFFE) || + (x == 0xFFFF))); + *p++ = (((byte) (x >> 12)) | + ((byte) 0xE0)); + *p++ = ((((byte) (x >> 6)) & 0x3F) | + ((byte) 0x80)); + *p++ = (((byte) (x & 0x3F)) | + ((byte) 0x80)); + } else { + ASSERT(x < 0x110000); + *p++ = (((byte) (x >> 18)) | + ((byte) 0xF0)); + *p++ = ((((byte) (x >> 12)) & 0x3F) | + ((byte) 0x80)); + *p++ = ((((byte) (x >> 6)) & 0x3F) | + ((byte) 0x80)); + *p++ = (((byte) (x & 0x3F)) | + ((byte) 0x80)); + } + break; + case ERL_FILENAME_WIN_WCHAR: + ASSERT(x <= 0xFFFF); + *p++ = (byte) (x & 0xFFU); + *p++ = (byte) ((x >> 8) & 0xFFU); + break; + default: + ASSERT(0); + } + + /* everything else will give badarg later + in the process, so we dont check */ + ioterm = CDR(objp); + if (!is_list(ioterm)) { + break; + } + objp = list_val(ioterm); + obj = CAR(objp); + if (!is_small(obj)) + break; + } + } else if (is_nil(obj)) { + ioterm = CDR(objp); + if (!is_list(ioterm)) { + break; + } + objp = list_val(ioterm); + obj = CAR(objp); + } else if (is_list(obj)) { + /* push rest of list for later processing, start + again with sublist */ + ESTACK_PUSH(stack,CDR(objp)); + ioterm = obj; + goto L_Again; + } else { + ASSERT(0); + } + if (is_nil(ioterm) || !is_list(ioterm)) { + break; + } + } /* for(;;) */ + } /* is_list(ioterm) */ + + ASSERT(is_list(ioterm) || is_nil(ioterm)); + } /* while not estack empty */ + DESTROY_ESTACK(stack); + return; +} +void erts_copy_utf8_to_utf16_little(byte *target, byte *bytes, int num_chars) +{ + Uint unipoint; + + while (num_chars--) { + if (((*bytes) & ((byte) 0x80)) == 0) { + unipoint = (Uint) *bytes; + ++bytes; + } else if (((*bytes) & ((byte) 0xE0)) == 0xC0) { + unipoint = + (((Uint) ((*bytes) & ((byte) 0x1F))) << 6) | + ((Uint) (bytes[1] & ((byte) 0x3F))); + bytes += 2; + } else if (((*bytes) & ((byte) 0xF0)) == 0xE0) { + unipoint = + (((Uint) ((*bytes) & ((byte) 0xF))) << 12) | + (((Uint) (bytes[1] & ((byte) 0x3F))) << 6) | + ((Uint) (bytes[2] & ((byte) 0x3F))); + bytes +=3; + } else if (((*bytes) & ((byte) 0xF8)) == 0xF0) { + unipoint = + (((Uint) ((*bytes) & ((byte) 0x7))) << 18) | + (((Uint) (bytes[1] & ((byte) 0x3F))) << 12) | + (((Uint) (bytes[2] & ((byte) 0x3F))) << 6) | + ((Uint) (bytes[3] & ((byte) 0x3F))); + bytes += 4; + } else { + erl_exit(1,"Internal unicode error in prim_file:internal_name2native/1"); + } + *target++ = (byte) (unipoint & 0xFF); + *target++ = (byte) ((unipoint >> 8) & 0xFF); + } +} + +/* + * This internal bif converts a filename to whatever format is suitable for the file driver + * It also adds zero termination so that prim_file needn't bother with the character encoding + * of the file driver + */ +BIF_RETTYPE prim_file_internal_name2native_1(BIF_ALIST_1) +{ + int encoding = erts_get_native_filename_encoding(); + Sint need; + Eterm bin_term; + byte* bin_p; + /* Prim file explicitly does not allow atoms, although we could + very well cope with it. Instead of letting 'file' handle them, + it would probably be more efficient to handle them here. Subject to + change in R15. */ + if (is_atom(BIF_ARG_1)) { + BIF_ERROR(BIF_P,BADARG); + } + if (is_binary(BIF_ARG_1)) { + byte *temp_alloc = NULL; + byte *bytes; + byte *err_pos; + Uint size,num_chars; + /* Uninterpreted encoding except if windows widechar, in case we convert from + utf8 to win_wchar */ + size = binary_size(BIF_ARG_1); + bytes = erts_get_aligned_binary_bytes(BIF_ARG_1, &temp_alloc); + if (encoding != ERL_FILENAME_WIN_WCHAR) { + /*Add 0 termination only*/ + bin_term = new_binary(BIF_P, NULL, size+1); + bin_p = binary_bytes(bin_term); + memcpy(bin_p,bytes,size); + bin_p[size]=0; + erts_free_aligned_binary_bytes(temp_alloc); + BIF_RET(bin_term); + } + /* In a wchar world, the emulator flags only affect how + binaries are interpreted when sent from the user. */ + /* Determine real length and create a new binary */ + if (erts_analyze_utf8(bytes,size,&err_pos,&num_chars,NULL) != ERTS_UTF8_OK || + erts_get_user_requested_filename_encoding() == ERL_FILENAME_LATIN1) { + /* What to do now? Maybe latin1, so just take byte for byte instead */ + bin_term = new_binary(BIF_P, 0, (size+1)*2); + bin_p = binary_bytes(bin_term); + while (size--) { + *bin_p++ = *bytes++; + *bin_p++ = 0; + } + *bin_p++ = 0; + *bin_p++ = 0; + erts_free_aligned_binary_bytes(temp_alloc); + BIF_RET(bin_term); + } + /* OK, UTF8 ok, number of characters is in num_chars */ + bin_term = new_binary(BIF_P, 0, (num_chars+1)*2); + bin_p = binary_bytes(bin_term); + erts_copy_utf8_to_utf16_little(bin_p, bytes, num_chars); + /* zero termination */ + bin_p[num_chars*2] = 0; + bin_p[num_chars*2+1] = 0; + erts_free_aligned_binary_bytes(temp_alloc); + BIF_RET(bin_term); + } /* binary */ + + + if ((need = erts_native_filename_need(BIF_ARG_1,encoding)) < 0) { + BIF_ERROR(BIF_P,BADARG); + } + if (encoding == ERL_FILENAME_WIN_WCHAR) { + need += 2; + } else { + ++need; + } + + bin_term = new_binary(BIF_P, 0, need); + bin_p = binary_bytes(bin_term); + erts_native_filename_put(BIF_ARG_1,encoding,bin_p); + bin_p[need-1] = 0; + if (encoding == ERL_FILENAME_WIN_WCHAR) { + bin_p[need-2] = 0; + } + BIF_RET(bin_term); +} + +BIF_RETTYPE prim_file_internal_native2name_1(BIF_ALIST_1) +{ + Eterm real_bin; + Uint offset; + Uint size,num_chars; + Uint bitsize; + Uint bitoffs; + Eterm *hp; + byte *temp_alloc = NULL; + byte *bytes; + byte *err_pos; + Uint num_built; /* characters */ + Uint num_eaten; /* bytes */ + Eterm ret; + int mac = 0; + + if (is_not_binary(BIF_ARG_1)) { + BIF_ERROR(BIF_P,BADARG); + } + size = binary_size(BIF_ARG_1); + ERTS_GET_REAL_BIN(BIF_ARG_1, real_bin, offset, bitoffs, bitsize); + if (bitsize != 0) { + BIF_ERROR(BIF_P,BADARG); + } + if (size == 0) { + BIF_RET(NIL); + } + switch (erts_get_native_filename_encoding()) { + case ERL_FILENAME_LATIN1: + hp = HAlloc(BIF_P, 2 * size); + bytes = binary_bytes(real_bin)+offset; + + BIF_RET(erts_bin_bytes_to_list(NIL, hp, bytes, size, bitoffs)); + case ERL_FILENAME_UTF8_MAC: + mac = 1; + case ERL_FILENAME_UTF8: + bytes = erts_get_aligned_binary_bytes(BIF_ARG_1, &temp_alloc); + if (erts_analyze_utf8(bytes,size,&err_pos,&num_chars,NULL) != ERTS_UTF8_OK) { + erts_free_aligned_binary_bytes(temp_alloc); + goto noconvert; + } + num_built = 0; + num_eaten = 0; + if (mac) { + ret = do_utf8_to_list_normalize(BIF_P, num_chars, bytes, size); + } else { + ret = do_utf8_to_list(BIF_P, num_chars, bytes, size, num_chars, &num_built, &num_eaten, NIL); + } + erts_free_aligned_binary_bytes(temp_alloc); + BIF_RET(ret); + case ERL_FILENAME_WIN_WCHAR: + bytes = erts_get_aligned_binary_bytes(BIF_ARG_1, &temp_alloc); + if ((size % 2) != 0) { /* Panic fixup to avoid crashing the emulator */ + size--; + hp = HAlloc(BIF_P, size+2); + ret = CONS(hp,make_small((Uint) bytes[size]),NIL); + hp += 2; + } else { + hp = HAlloc(BIF_P, size); + ret = NIL; + } + bytes += size-1; + while (size > 0) { + Uint x = ((Uint) *bytes--) << 8; + x |= ((Uint) *bytes--); + size -= 2; + ret = CONS(hp,make_small(x),ret); + hp += 2; + } + erts_free_aligned_binary_bytes(temp_alloc); + BIF_RET(ret); + default: + goto noconvert; + } + noconvert: + BIF_RET(BIF_ARG_1); +} + +BIF_RETTYPE prim_file_internal_normalize_utf8_1(BIF_ALIST_1) +{ + Eterm real_bin; + Uint offset; + Uint size,num_chars; + Uint bitsize; + Uint bitoffs; + Eterm ret; + byte *temp_alloc = NULL; + byte *bytes; + byte *err_pos; + + if (is_not_binary(BIF_ARG_1)) { + BIF_ERROR(BIF_P,BADARG); + } + size = binary_size(BIF_ARG_1); + ERTS_GET_REAL_BIN(BIF_ARG_1, real_bin, offset, bitoffs, bitsize); + if (bitsize != 0) { + BIF_ERROR(BIF_P,BADARG); + } + if (size == 0) { + BIF_RET(NIL); + } + bytes = erts_get_aligned_binary_bytes(BIF_ARG_1, &temp_alloc); + if (erts_analyze_utf8(bytes,size,&err_pos,&num_chars,NULL) != ERTS_UTF8_OK) { + erts_free_aligned_binary_bytes(temp_alloc); + BIF_ERROR(BIF_P,BADARG); + } + ret = do_utf8_to_list_normalize(BIF_P, num_chars, bytes, size); + erts_free_aligned_binary_bytes(temp_alloc); + BIF_RET(ret); +} + +BIF_RETTYPE file_native_name_encoding_0(BIF_ALIST_0) +{ + switch (erts_get_native_filename_encoding()) { + case ERL_FILENAME_LATIN1: + BIF_RET(am_latin1); + case ERL_FILENAME_UTF8_MAC: + case ERL_FILENAME_UTF8: + BIF_RET(am_utf8); + case ERL_FILENAME_WIN_WCHAR: + if (erts_get_user_requested_filename_encoding() == ERL_FILENAME_LATIN1) { + BIF_RET(am_latin1); + } else { + BIF_RET(am_utf8); + } + default: + BIF_RET(am_undefined); + } +} diff --git a/erts/emulator/beam/erl_unicode_normalize.h b/erts/emulator/beam/erl_unicode_normalize.h new file mode 100644 index 0000000000..fb0a111ca2 --- /dev/null +++ b/erts/emulator/beam/erl_unicode_normalize.h @@ -0,0 +1,1687 @@ +/* +* %CopyrightBegin% +* +* Copyright Ericsson AB 1999-2010. All Rights Reserved. +* +* The contents of this file are subject to the Erlang Public License, +* Version 1.1, (the "License"); you may not use this file except in +* compliance with the License. You should have received a copy of the +* Erlang Public License along with this software. If not, it can be +* retrieved online at http://www.erlang.org/. +* +* Software distributed under the License is distributed on an "AS IS" +* basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +* the License for the specific language governing rights and limitations +* under the License. +* +* %CopyrightEnd% +*/ +/* +* This file is automatically generated by dec.erl, do not edit manually +*/ +#define HASH_SIZE_FACTOR 2 +typedef struct _compose_entry { + Uint16 c; + Uint16 res; + Uint16 num_subs; + struct _compose_entry *subs; + int *hash; +} CompEntry; + +static int compose_tab_size = 61; +static int hash_compose_tab_0_15[12] = +{-1,3,-1,5,-1,0,4,2,-1,1,-1,-1}; /* hash_compose_tab_0_15 */ +static CompEntry compose_tab_0_15[] = { +{65, 7846, 0, NULL, NULL}, +{69, 7872, 0, NULL, NULL}, +{79, 7890, 0, NULL, NULL}, +{97, 7847, 0, NULL, NULL}, +{101, 7873, 0, NULL, NULL}, +{111, 7891, 0, NULL, NULL} +}; /* compose_tab_0_15 */ +static int hash_compose_tab_0_16[8] = +{3,-1,-1,-1,-1,0,2,1}; /* hash_compose_tab_0_16 */ +static CompEntry compose_tab_0_16[] = { +{69, 7700, 0, NULL, NULL}, +{79, 7760, 0, NULL, NULL}, +{101, 7701, 0, NULL, NULL}, +{111, 7761, 0, NULL, NULL} +}; /* compose_tab_0_16 */ +static int hash_compose_tab_0_17[4] = +{-1,0,1,-1}; /* hash_compose_tab_0_17 */ +static CompEntry compose_tab_0_17[] = { +{65, 7856, 0, NULL, NULL}, +{97, 7857, 0, NULL, NULL} +}; /* compose_tab_0_17 */ +static int hash_compose_tab_0_18[8] = +{-1,2,-1,-1,-1,0,1,3}; /* hash_compose_tab_0_18 */ +static CompEntry compose_tab_0_18[] = { +{85, 475, 0, NULL, NULL}, +{117, 476, 0, NULL, NULL}, +{953, 8146, 0, NULL, NULL}, +{965, 8162, 0, NULL, NULL} +}; /* compose_tab_0_18 */ +static int hash_compose_tab_0_19_0[12] = +{-1,0,2,4,-1,-1,-1,1,-1,3,5,-1}; /* hash_compose_tab_0_19_0 */ +static CompEntry compose_tab_0_19_0[] = { +{913, 8074, 0, NULL, NULL}, +{919, 8090, 0, NULL, NULL}, +{937, 8106, 0, NULL, NULL}, +{945, 8066, 0, NULL, NULL}, +{951, 8082, 0, NULL, NULL}, +{969, 8098, 0, NULL, NULL} +}; /* compose_tab_0_19_0 */ +static int hash_compose_tab_0_19[28] = +{9,10,-1,5,-1,-1,-1,11,-1,-1,-1,-1,-1,6,12,-1,-1,1,13,-1,-1,2,7,3,-1,0,4,8}; /* hash_compose_tab_0_19 */ +static CompEntry compose_tab_0_19[] = { +{837, 0, 6, compose_tab_0_19_0, hash_compose_tab_0_19_0}, +{913, 7946, 0, NULL, NULL}, +{917, 7962, 0, NULL, NULL}, +{919, 7978, 0, NULL, NULL}, +{921, 7994, 0, NULL, NULL}, +{927, 8010, 0, NULL, NULL}, +{937, 8042, 0, NULL, NULL}, +{945, 7938, 0, NULL, NULL}, +{949, 7954, 0, NULL, NULL}, +{951, 7970, 0, NULL, NULL}, +{953, 7986, 0, NULL, NULL}, +{959, 8002, 0, NULL, NULL}, +{965, 8018, 0, NULL, NULL}, +{969, 8034, 0, NULL, NULL} +}; /* compose_tab_0_19 */ +static int hash_compose_tab_0_20_0[12] = +{-1,0,2,4,-1,-1,-1,1,-1,3,5,-1}; /* hash_compose_tab_0_20_0 */ +static CompEntry compose_tab_0_20_0[] = { +{913, 8075, 0, NULL, NULL}, +{919, 8091, 0, NULL, NULL}, +{937, 8107, 0, NULL, NULL}, +{945, 8067, 0, NULL, NULL}, +{951, 8083, 0, NULL, NULL}, +{969, 8099, 0, NULL, NULL} +}; /* compose_tab_0_20_0 */ +static int hash_compose_tab_0_20[30] = +{-1,-1,-1,6,-1,13,-1,7,-1,14,-1,-1,-1,1,-1,8,-1,2,-1,3,9,4,10,11,-1,-1,-1,0,5, + 12}; /* hash_compose_tab_0_20 */ +static CompEntry compose_tab_0_20[] = { +{837, 0, 6, compose_tab_0_20_0, hash_compose_tab_0_20_0}, +{913, 7947, 0, NULL, NULL}, +{917, 7963, 0, NULL, NULL}, +{919, 7979, 0, NULL, NULL}, +{921, 7995, 0, NULL, NULL}, +{927, 8011, 0, NULL, NULL}, +{933, 8027, 0, NULL, NULL}, +{937, 8043, 0, NULL, NULL}, +{945, 7939, 0, NULL, NULL}, +{949, 7955, 0, NULL, NULL}, +{951, 7971, 0, NULL, NULL}, +{953, 7987, 0, NULL, NULL}, +{959, 8003, 0, NULL, NULL}, +{965, 8019, 0, NULL, NULL}, +{969, 8035, 0, NULL, NULL} +}; /* compose_tab_0_20 */ +static int hash_compose_tab_0_21[8] = +{2,-1,-1,-1,-1,1,3,0}; /* hash_compose_tab_0_21 */ +static CompEntry compose_tab_0_21[] = { +{79, 7900, 0, NULL, NULL}, +{85, 7914, 0, NULL, NULL}, +{111, 7901, 0, NULL, NULL}, +{117, 7915, 0, NULL, NULL} +}; /* compose_tab_0_21 */ +static int hash_compose_tab_0_22[6] = +{-1,-1,-1,0,1,2}; /* hash_compose_tab_0_22 */ +static CompEntry compose_tab_0_22[] = { +{945, 8114, 0, NULL, NULL}, +{951, 8130, 0, NULL, NULL}, +{969, 8178, 0, NULL, NULL} +}; /* compose_tab_0_22 */ +static int hash_compose_tab_0[78] = +{38,3,29,-1,-1,-1,-1,4,19,5,20,6,14,30,31,21,32,33,37,7,-1,-1,-1,8,34,-1,-1,9, + -1,35,-1,-1,-1,10,36,-1,-1,-1,-1,11,-1,12,-1,13,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,23,-1,22,-1,24,-1,25,-1,26,-1,0,-1,-1,15,1,16,27,17,2,18,28,-1,-1}; /* hash_compose_tab_0 */ +static CompEntry compose_tab_0[] = { +{65, 192, 0, NULL, NULL}, +{69, 200, 0, NULL, NULL}, +{73, 204, 0, NULL, NULL}, +{79, 210, 0, NULL, NULL}, +{85, 217, 0, NULL, NULL}, +{87, 7808, 0, NULL, NULL}, +{89, 7922, 0, NULL, NULL}, +{97, 224, 0, NULL, NULL}, +{101, 232, 0, NULL, NULL}, +{105, 236, 0, NULL, NULL}, +{111, 242, 0, NULL, NULL}, +{117, 249, 0, NULL, NULL}, +{119, 7809, 0, NULL, NULL}, +{121, 7923, 0, NULL, NULL}, +{168, 8173, 0, NULL, NULL}, +{770, 0, 6, compose_tab_0_15, hash_compose_tab_0_15}, +{772, 0, 4, compose_tab_0_16, hash_compose_tab_0_16}, +{774, 0, 2, compose_tab_0_17, hash_compose_tab_0_17}, +{776, 0, 4, compose_tab_0_18, hash_compose_tab_0_18}, +{787, 0, 14, compose_tab_0_19, hash_compose_tab_0_19}, +{788, 0, 15, compose_tab_0_20, hash_compose_tab_0_20}, +{795, 0, 4, compose_tab_0_21, hash_compose_tab_0_21}, +{837, 0, 3, compose_tab_0_22, hash_compose_tab_0_22}, +{913, 8122, 0, NULL, NULL}, +{917, 8136, 0, NULL, NULL}, +{919, 8138, 0, NULL, NULL}, +{921, 8154, 0, NULL, NULL}, +{927, 8184, 0, NULL, NULL}, +{933, 8170, 0, NULL, NULL}, +{937, 8186, 0, NULL, NULL}, +{945, 8048, 0, NULL, NULL}, +{949, 8050, 0, NULL, NULL}, +{951, 8052, 0, NULL, NULL}, +{953, 8054, 0, NULL, NULL}, +{959, 8056, 0, NULL, NULL}, +{965, 8058, 0, NULL, NULL}, +{969, 8060, 0, NULL, NULL}, +{8127, 8141, 0, NULL, NULL}, +{8190, 8157, 0, NULL, NULL} +}; /* compose_tab_0 */ +static int hash_compose_tab_1_39[12] = +{-1,3,-1,5,-1,0,4,2,-1,1,-1,-1}; /* hash_compose_tab_1_39 */ +static CompEntry compose_tab_1_39[] = { +{65, 7844, 0, NULL, NULL}, +{69, 7870, 0, NULL, NULL}, +{79, 7888, 0, NULL, NULL}, +{97, 7845, 0, NULL, NULL}, +{101, 7871, 0, NULL, NULL}, +{111, 7889, 0, NULL, NULL} +}; /* compose_tab_1_39 */ +static int hash_compose_tab_1_40[8] = +{2,-1,-1,-1,-1,1,3,0}; /* hash_compose_tab_1_40 */ +static CompEntry compose_tab_1_40[] = { +{79, 7756, 0, NULL, NULL}, +{85, 7800, 0, NULL, NULL}, +{111, 7757, 0, NULL, NULL}, +{117, 7801, 0, NULL, NULL} +}; /* compose_tab_1_40 */ +static int hash_compose_tab_1_41[8] = +{3,-1,-1,-1,-1,0,2,1}; /* hash_compose_tab_1_41 */ +static CompEntry compose_tab_1_41[] = { +{69, 7702, 0, NULL, NULL}, +{79, 7762, 0, NULL, NULL}, +{101, 7703, 0, NULL, NULL}, +{111, 7763, 0, NULL, NULL} +}; /* compose_tab_1_41 */ +static int hash_compose_tab_1_42[4] = +{-1,0,1,-1}; /* hash_compose_tab_1_42 */ +static CompEntry compose_tab_1_42[] = { +{65, 7854, 0, NULL, NULL}, +{97, 7855, 0, NULL, NULL} +}; /* compose_tab_1_42 */ +static int hash_compose_tab_1_43[12] = +{-1,0,1,-1,-1,4,5,-1,-1,2,3,-1}; /* hash_compose_tab_1_43 */ +static CompEntry compose_tab_1_43[] = { +{73, 7726, 0, NULL, NULL}, +{85, 471, 0, NULL, NULL}, +{105, 7727, 0, NULL, NULL}, +{117, 472, 0, NULL, NULL}, +{953, 8147, 0, NULL, NULL}, +{965, 8163, 0, NULL, NULL} +}; /* compose_tab_1_43 */ +static int hash_compose_tab_1_44[4] = +{-1,0,1,-1}; /* hash_compose_tab_1_44 */ +static CompEntry compose_tab_1_44[] = { +{65, 506, 0, NULL, NULL}, +{97, 507, 0, NULL, NULL} +}; /* compose_tab_1_44 */ +static int hash_compose_tab_1_45_0[12] = +{-1,0,2,4,-1,-1,-1,1,-1,3,5,-1}; /* hash_compose_tab_1_45_0 */ +static CompEntry compose_tab_1_45_0[] = { +{913, 8076, 0, NULL, NULL}, +{919, 8092, 0, NULL, NULL}, +{937, 8108, 0, NULL, NULL}, +{945, 8068, 0, NULL, NULL}, +{951, 8084, 0, NULL, NULL}, +{969, 8100, 0, NULL, NULL} +}; /* compose_tab_1_45_0 */ +static int hash_compose_tab_1_45[28] = +{9,10,-1,5,-1,-1,-1,11,-1,-1,-1,-1,-1,6,12,-1,-1,1,13,-1,-1,2,7,3,-1,0,4,8}; /* hash_compose_tab_1_45 */ +static CompEntry compose_tab_1_45[] = { +{837, 0, 6, compose_tab_1_45_0, hash_compose_tab_1_45_0}, +{913, 7948, 0, NULL, NULL}, +{917, 7964, 0, NULL, NULL}, +{919, 7980, 0, NULL, NULL}, +{921, 7996, 0, NULL, NULL}, +{927, 8012, 0, NULL, NULL}, +{937, 8044, 0, NULL, NULL}, +{945, 7940, 0, NULL, NULL}, +{949, 7956, 0, NULL, NULL}, +{951, 7972, 0, NULL, NULL}, +{953, 7988, 0, NULL, NULL}, +{959, 8004, 0, NULL, NULL}, +{965, 8020, 0, NULL, NULL}, +{969, 8036, 0, NULL, NULL} +}; /* compose_tab_1_45 */ +static int hash_compose_tab_1_46_0[12] = +{-1,0,2,4,-1,-1,-1,1,-1,3,5,-1}; /* hash_compose_tab_1_46_0 */ +static CompEntry compose_tab_1_46_0[] = { +{913, 8077, 0, NULL, NULL}, +{919, 8093, 0, NULL, NULL}, +{937, 8109, 0, NULL, NULL}, +{945, 8069, 0, NULL, NULL}, +{951, 8085, 0, NULL, NULL}, +{969, 8101, 0, NULL, NULL} +}; /* compose_tab_1_46_0 */ +static int hash_compose_tab_1_46[30] = +{-1,-1,-1,6,-1,13,-1,7,-1,14,-1,-1,-1,1,-1,8,-1,2,-1,3,9,4,10,11,-1,-1,-1,0,5, + 12}; /* hash_compose_tab_1_46 */ +static CompEntry compose_tab_1_46[] = { +{837, 0, 6, compose_tab_1_46_0, hash_compose_tab_1_46_0}, +{913, 7949, 0, NULL, NULL}, +{917, 7965, 0, NULL, NULL}, +{919, 7981, 0, NULL, NULL}, +{921, 7997, 0, NULL, NULL}, +{927, 8013, 0, NULL, NULL}, +{933, 8029, 0, NULL, NULL}, +{937, 8045, 0, NULL, NULL}, +{945, 7941, 0, NULL, NULL}, +{949, 7957, 0, NULL, NULL}, +{951, 7973, 0, NULL, NULL}, +{953, 7989, 0, NULL, NULL}, +{959, 8005, 0, NULL, NULL}, +{965, 8021, 0, NULL, NULL}, +{969, 8037, 0, NULL, NULL} +}; /* compose_tab_1_46 */ +static int hash_compose_tab_1_47[8] = +{2,-1,-1,-1,-1,1,3,0}; /* hash_compose_tab_1_47 */ +static CompEntry compose_tab_1_47[] = { +{79, 7898, 0, NULL, NULL}, +{85, 7912, 0, NULL, NULL}, +{111, 7899, 0, NULL, NULL}, +{117, 7913, 0, NULL, NULL} +}; /* compose_tab_1_47 */ +static int hash_compose_tab_1_48[4] = +{1,-1,-1,0}; /* hash_compose_tab_1_48 */ +static CompEntry compose_tab_1_48[] = { +{67, 7688, 0, NULL, NULL}, +{99, 7689, 0, NULL, NULL} +}; /* compose_tab_1_48 */ +static int hash_compose_tab_1_49[6] = +{-1,-1,-1,0,1,2}; /* hash_compose_tab_1_49 */ +static CompEntry compose_tab_1_49[] = { +{945, 8116, 0, NULL, NULL}, +{951, 8132, 0, NULL, NULL}, +{959, 8180, 0, NULL, NULL} +}; /* compose_tab_1_49 */ +static int hash_compose_tab_1[140] = +{-1,-1,-1,-1,-1,-1,-1,68,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,34,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,35,-1,-1,-1,-1,64,-1,0,-1,1,-1,2,39,3,40,4,41,5,6,7, + 8,9,10,36,11,12,42,13,43,14,44,15,16,37,45,46,50,47,51,17,52,18,53,19,54,20, + 55,21,56,22,23,24,25,26,27,38,28,29,48,30,57,31,58,32,33,59,60,61,62,65,66, + 63,67,69,-1,-1,-1,-1,-1,49,-1,-1}; /* hash_compose_tab_1 */ +static CompEntry compose_tab_1[] = { +{65, 193, 0, NULL, NULL}, +{67, 262, 0, NULL, NULL}, +{69, 201, 0, NULL, NULL}, +{71, 500, 0, NULL, NULL}, +{73, 205, 0, NULL, NULL}, +{75, 7728, 0, NULL, NULL}, +{76, 313, 0, NULL, NULL}, +{77, 7742, 0, NULL, NULL}, +{78, 323, 0, NULL, NULL}, +{79, 211, 0, NULL, NULL}, +{80, 7764, 0, NULL, NULL}, +{82, 340, 0, NULL, NULL}, +{83, 346, 0, NULL, NULL}, +{85, 218, 0, NULL, NULL}, +{87, 7810, 0, NULL, NULL}, +{89, 221, 0, NULL, NULL}, +{90, 377, 0, NULL, NULL}, +{97, 225, 0, NULL, NULL}, +{99, 263, 0, NULL, NULL}, +{101, 233, 0, NULL, NULL}, +{103, 501, 0, NULL, NULL}, +{105, 237, 0, NULL, NULL}, +{107, 7729, 0, NULL, NULL}, +{108, 314, 0, NULL, NULL}, +{109, 7743, 0, NULL, NULL}, +{110, 324, 0, NULL, NULL}, +{111, 243, 0, NULL, NULL}, +{112, 7765, 0, NULL, NULL}, +{114, 341, 0, NULL, NULL}, +{115, 347, 0, NULL, NULL}, +{117, 250, 0, NULL, NULL}, +{119, 7811, 0, NULL, NULL}, +{121, 253, 0, NULL, NULL}, +{122, 378, 0, NULL, NULL}, +{168, 8174, 0, NULL, NULL}, +{198, 508, 0, NULL, NULL}, +{216, 510, 0, NULL, NULL}, +{230, 509, 0, NULL, NULL}, +{248, 511, 0, NULL, NULL}, +{770, 0, 6, compose_tab_1_39, hash_compose_tab_1_39}, +{771, 0, 4, compose_tab_1_40, hash_compose_tab_1_40}, +{772, 0, 4, compose_tab_1_41, hash_compose_tab_1_41}, +{774, 0, 2, compose_tab_1_42, hash_compose_tab_1_42}, +{776, 0, 6, compose_tab_1_43, hash_compose_tab_1_43}, +{778, 0, 2, compose_tab_1_44, hash_compose_tab_1_44}, +{787, 0, 14, compose_tab_1_45, hash_compose_tab_1_45}, +{788, 0, 15, compose_tab_1_46, hash_compose_tab_1_46}, +{795, 0, 4, compose_tab_1_47, hash_compose_tab_1_47}, +{807, 0, 2, compose_tab_1_48, hash_compose_tab_1_48}, +{837, 0, 3, compose_tab_1_49, hash_compose_tab_1_49}, +{913, 8123, 0, NULL, NULL}, +{917, 8137, 0, NULL, NULL}, +{919, 8139, 0, NULL, NULL}, +{921, 8155, 0, NULL, NULL}, +{927, 8185, 0, NULL, NULL}, +{933, 8171, 0, NULL, NULL}, +{937, 8187, 0, NULL, NULL}, +{945, 8049, 0, NULL, NULL}, +{949, 8051, 0, NULL, NULL}, +{951, 8053, 0, NULL, NULL}, +{953, 8055, 0, NULL, NULL}, +{959, 8057, 0, NULL, NULL}, +{965, 8059, 0, NULL, NULL}, +{969, 8061, 0, NULL, NULL}, +{1043, 1027, 0, NULL, NULL}, +{1050, 1036, 0, NULL, NULL}, +{1075, 1107, 0, NULL, NULL}, +{1082, 1116, 0, NULL, NULL}, +{8127, 8142, 0, NULL, NULL}, +{8190, 8158, 0, NULL, NULL} +}; /* compose_tab_1 */ +static int hash_compose_tab_2_26[12] = +{-1,3,-1,5,-1,0,4,2,-1,1,-1,-1}; /* hash_compose_tab_2_26 */ +static CompEntry compose_tab_2_26[] = { +{65, 7852, 0, NULL, NULL}, +{69, 7878, 0, NULL, NULL}, +{79, 7896, 0, NULL, NULL}, +{97, 7853, 0, NULL, NULL}, +{101, 7879, 0, NULL, NULL}, +{111, 7897, 0, NULL, NULL} +}; /* compose_tab_2_26 */ +static int hash_compose_tab_2[54] = +{-1,-1,-1,20,-1,-1,-1,21,-1,22,-1,0,23,1,24,2,25,3,4,5,6,-1,-1,-1,-1,7,-1,-1, + -1,8,-1,9,-1,10,-1,11,12,-1,-1,-1,-1,-1,-1,13,-1,14,-1,15,26,16,17,18,19,-1}; /* hash_compose_tab_2 */ +static CompEntry compose_tab_2[] = { +{65, 194, 0, NULL, NULL}, +{67, 264, 0, NULL, NULL}, +{69, 202, 0, NULL, NULL}, +{71, 284, 0, NULL, NULL}, +{72, 292, 0, NULL, NULL}, +{73, 206, 0, NULL, NULL}, +{74, 308, 0, NULL, NULL}, +{79, 212, 0, NULL, NULL}, +{83, 348, 0, NULL, NULL}, +{85, 219, 0, NULL, NULL}, +{87, 372, 0, NULL, NULL}, +{89, 374, 0, NULL, NULL}, +{90, 7824, 0, NULL, NULL}, +{97, 226, 0, NULL, NULL}, +{99, 265, 0, NULL, NULL}, +{101, 234, 0, NULL, NULL}, +{103, 285, 0, NULL, NULL}, +{104, 293, 0, NULL, NULL}, +{105, 238, 0, NULL, NULL}, +{106, 309, 0, NULL, NULL}, +{111, 244, 0, NULL, NULL}, +{115, 349, 0, NULL, NULL}, +{117, 251, 0, NULL, NULL}, +{119, 373, 0, NULL, NULL}, +{121, 375, 0, NULL, NULL}, +{122, 7825, 0, NULL, NULL}, +{803, 0, 6, compose_tab_2_26, hash_compose_tab_2_26} +}; /* compose_tab_2 */ +static int hash_compose_tab_3_16[12] = +{-1,3,-1,5,-1,0,4,2,-1,1,-1,-1}; /* hash_compose_tab_3_16 */ +static CompEntry compose_tab_3_16[] = { +{65, 7850, 0, NULL, NULL}, +{69, 7876, 0, NULL, NULL}, +{79, 7894, 0, NULL, NULL}, +{97, 7851, 0, NULL, NULL}, +{101, 7877, 0, NULL, NULL}, +{111, 7895, 0, NULL, NULL} +}; /* compose_tab_3_16 */ +static int hash_compose_tab_3_17[4] = +{-1,0,1,-1}; /* hash_compose_tab_3_17 */ +static CompEntry compose_tab_3_17[] = { +{65, 7860, 0, NULL, NULL}, +{97, 7861, 0, NULL, NULL} +}; /* compose_tab_3_17 */ +static int hash_compose_tab_3_18[8] = +{2,-1,-1,-1,-1,1,3,0}; /* hash_compose_tab_3_18 */ +static CompEntry compose_tab_3_18[] = { +{79, 7904, 0, NULL, NULL}, +{85, 7918, 0, NULL, NULL}, +{111, 7905, 0, NULL, NULL}, +{117, 7919, 0, NULL, NULL} +}; /* compose_tab_3_18 */ +static int hash_compose_tab_3[38] = +{-1,-1,3,4,13,14,-1,15,-1,5,6,16,-1,7,17,-1,-1,-1,-1,-1,-1,8,-1,-1,-1,9,-1,0, + -1,10,-1,1,-1,-1,11,2,12,18}; /* hash_compose_tab_3 */ +static CompEntry compose_tab_3[] = { +{65, 195, 0, NULL, NULL}, +{69, 7868, 0, NULL, NULL}, +{73, 296, 0, NULL, NULL}, +{78, 209, 0, NULL, NULL}, +{79, 213, 0, NULL, NULL}, +{85, 360, 0, NULL, NULL}, +{86, 7804, 0, NULL, NULL}, +{89, 7928, 0, NULL, NULL}, +{97, 227, 0, NULL, NULL}, +{101, 7869, 0, NULL, NULL}, +{105, 297, 0, NULL, NULL}, +{110, 241, 0, NULL, NULL}, +{111, 245, 0, NULL, NULL}, +{117, 361, 0, NULL, NULL}, +{118, 7805, 0, NULL, NULL}, +{121, 7929, 0, NULL, NULL}, +{770, 0, 6, compose_tab_3_16, hash_compose_tab_3_16}, +{774, 0, 2, compose_tab_3_17, hash_compose_tab_3_17}, +{795, 0, 4, compose_tab_3_18, hash_compose_tab_3_18} +}; /* compose_tab_3 */ +static int hash_compose_tab_4_14[4] = +{-1,0,1,-1}; /* hash_compose_tab_4_14 */ +static CompEntry compose_tab_4_14[] = { +{65, 480, 0, NULL, NULL}, +{97, 481, 0, NULL, NULL} +}; /* compose_tab_4_14 */ +static int hash_compose_tab_4_15[8] = +{-1,0,2,-1,-1,1,3,-1}; /* hash_compose_tab_4_15 */ +static CompEntry compose_tab_4_15[] = { +{65, 478, 0, NULL, NULL}, +{85, 469, 0, NULL, NULL}, +{97, 479, 0, NULL, NULL}, +{117, 470, 0, NULL, NULL} +}; /* compose_tab_4_15 */ +static int hash_compose_tab_4_16[8] = +{-1,-1,1,3,0,2,-1,-1}; /* hash_compose_tab_4_16 */ +static CompEntry compose_tab_4_16[] = { +{76, 7736, 0, NULL, NULL}, +{82, 7772, 0, NULL, NULL}, +{108, 7737, 0, NULL, NULL}, +{114, 7773, 0, NULL, NULL} +}; /* compose_tab_4_16 */ +static int hash_compose_tab_4_17[4] = +{1,-1,-1,0}; /* hash_compose_tab_4_17 */ +static CompEntry compose_tab_4_17[] = { +{79, 492, 0, NULL, NULL}, +{111, 493, 0, NULL, NULL} +}; /* compose_tab_4_17 */ +static int hash_compose_tab_4[56] = +{-1,22,-1,-1,-1,11,13,-1,-1,0,-1,-1,-1,1,23,2,26,3,18,16,-1,-1,-1,4,17,19,-1, + 27,-1,5,12,-1,-1,-1,-1,-1,-1,20,-1,-1,24,6,-1,-1,-1,7,-1,8,14,9,15,21,25,-1, + -1,10}; /* hash_compose_tab_4 */ +static CompEntry compose_tab_4[] = { +{65, 256, 0, NULL, NULL}, +{69, 274, 0, NULL, NULL}, +{71, 7712, 0, NULL, NULL}, +{73, 298, 0, NULL, NULL}, +{79, 332, 0, NULL, NULL}, +{85, 362, 0, NULL, NULL}, +{97, 257, 0, NULL, NULL}, +{101, 275, 0, NULL, NULL}, +{103, 7713, 0, NULL, NULL}, +{105, 299, 0, NULL, NULL}, +{111, 333, 0, NULL, NULL}, +{117, 363, 0, NULL, NULL}, +{198, 482, 0, NULL, NULL}, +{230, 483, 0, NULL, NULL}, +{775, 0, 2, compose_tab_4_14, hash_compose_tab_4_14}, +{776, 0, 4, compose_tab_4_15, hash_compose_tab_4_15}, +{803, 0, 4, compose_tab_4_16, hash_compose_tab_4_16}, +{808, 0, 2, compose_tab_4_17, hash_compose_tab_4_17}, +{913, 8121, 0, NULL, NULL}, +{921, 8153, 0, NULL, NULL}, +{933, 8169, 0, NULL, NULL}, +{945, 8113, 0, NULL, NULL}, +{953, 8145, 0, NULL, NULL}, +{965, 8161, 0, NULL, NULL}, +{1048, 1250, 0, NULL, NULL}, +{1059, 1262, 0, NULL, NULL}, +{1080, 1251, 0, NULL, NULL}, +{1091, 1263, 0, NULL, NULL} +}; /* compose_tab_4 */ +static int hash_compose_tab_5_12[4] = +{-1,0,1,-1}; /* hash_compose_tab_5_12 */ +static CompEntry compose_tab_5_12[] = { +{65, 7862, 0, NULL, NULL}, +{97, 7863, 0, NULL, NULL} +}; /* compose_tab_5_12 */ +static int hash_compose_tab_5_13[4] = +{-1,0,1,-1}; /* hash_compose_tab_5_13 */ +static CompEntry compose_tab_5_13[] = { +{69, 7708, 0, NULL, NULL}, +{101, 7709, 0, NULL, NULL} +}; /* compose_tab_5_13 */ +static int hash_compose_tab_5[60] = +{28,-1,-1,-1,-1,0,19,-1,-1,1,-1,2,29,3,14,-1,-1,-1,-1,4,20,15,-1,12,-1,5,21, + 13,22,23,-1,-1,-1,16,-1,-1,-1,6,-1,24,-1,7,-1,8,-1,9,17,-1,-1,-1,-1,10,25,18, + -1,-1,-1,11,26,27}; /* hash_compose_tab_5 */ +static CompEntry compose_tab_5[] = { +{65, 258, 0, NULL, NULL}, +{69, 276, 0, NULL, NULL}, +{71, 286, 0, NULL, NULL}, +{73, 300, 0, NULL, NULL}, +{79, 334, 0, NULL, NULL}, +{85, 364, 0, NULL, NULL}, +{97, 259, 0, NULL, NULL}, +{101, 277, 0, NULL, NULL}, +{103, 287, 0, NULL, NULL}, +{105, 301, 0, NULL, NULL}, +{111, 335, 0, NULL, NULL}, +{117, 365, 0, NULL, NULL}, +{803, 0, 2, compose_tab_5_12, hash_compose_tab_5_12}, +{807, 0, 2, compose_tab_5_13, hash_compose_tab_5_13}, +{913, 8120, 0, NULL, NULL}, +{921, 8152, 0, NULL, NULL}, +{933, 8168, 0, NULL, NULL}, +{945, 8112, 0, NULL, NULL}, +{953, 8144, 0, NULL, NULL}, +{965, 8160, 0, NULL, NULL}, +{1040, 1232, 0, NULL, NULL}, +{1045, 1238, 0, NULL, NULL}, +{1046, 1217, 0, NULL, NULL}, +{1048, 1049, 0, NULL, NULL}, +{1059, 1038, 0, NULL, NULL}, +{1072, 1233, 0, NULL, NULL}, +{1077, 1239, 0, NULL, NULL}, +{1078, 1218, 0, NULL, NULL}, +{1080, 1081, 0, NULL, NULL}, +{1091, 1118, 0, NULL, NULL} +}; /* compose_tab_5 */ +static int hash_compose_tab_6_36[4] = +{1,-1,-1,0}; /* hash_compose_tab_6_36 */ +static CompEntry compose_tab_6_36[] = { +{83, 7780, 0, NULL, NULL}, +{115, 7781, 0, NULL, NULL} +}; /* compose_tab_6_36 */ +static int hash_compose_tab_6_38[4] = +{1,-1,-1,0}; /* hash_compose_tab_6_38 */ +static CompEntry compose_tab_6_38[] = { +{83, 7782, 0, NULL, NULL}, +{115, 7783, 0, NULL, NULL} +}; /* compose_tab_6_38 */ +static int hash_compose_tab_6_39[4] = +{1,-1,-1,0}; /* hash_compose_tab_6_39 */ +static CompEntry compose_tab_6_39[] = { +{83, 7784, 0, NULL, NULL}, +{115, 7785, 0, NULL, NULL} +}; /* compose_tab_6_39 */ +static int hash_compose_tab_6[80] = +{10,-1,11,12,13,39,-1,14,15,16,17,-1,-1,-1,-1,-1,-1,-1,18,19,20,21,22,23,24, + -1,-1,-1,-1,25,26,-1,27,-1,28,29,30,-1,-1,31,32,33,34,-1,-1,-1,-1,-1,-1,36, + -1,-1,-1,-1,37,-1,-1,-1,-1,-1,38,-1,-1,35,-1,-1,0,1,2,3,4,5,6,7,-1,-1,-1,8,9, + -1}; /* hash_compose_tab_6 */ +static CompEntry compose_tab_6[] = { +{66, 7682, 0, NULL, NULL}, +{67, 266, 0, NULL, NULL}, +{68, 7690, 0, NULL, NULL}, +{69, 278, 0, NULL, NULL}, +{70, 7710, 0, NULL, NULL}, +{71, 288, 0, NULL, NULL}, +{72, 7714, 0, NULL, NULL}, +{73, 304, 0, NULL, NULL}, +{77, 7744, 0, NULL, NULL}, +{78, 7748, 0, NULL, NULL}, +{80, 7766, 0, NULL, NULL}, +{82, 7768, 0, NULL, NULL}, +{83, 7776, 0, NULL, NULL}, +{84, 7786, 0, NULL, NULL}, +{87, 7814, 0, NULL, NULL}, +{88, 7818, 0, NULL, NULL}, +{89, 7822, 0, NULL, NULL}, +{90, 379, 0, NULL, NULL}, +{98, 7683, 0, NULL, NULL}, +{99, 267, 0, NULL, NULL}, +{100, 7691, 0, NULL, NULL}, +{101, 279, 0, NULL, NULL}, +{102, 7711, 0, NULL, NULL}, +{103, 289, 0, NULL, NULL}, +{104, 7715, 0, NULL, NULL}, +{109, 7745, 0, NULL, NULL}, +{110, 7749, 0, NULL, NULL}, +{112, 7767, 0, NULL, NULL}, +{114, 7769, 0, NULL, NULL}, +{115, 7777, 0, NULL, NULL}, +{116, 7787, 0, NULL, NULL}, +{119, 7815, 0, NULL, NULL}, +{120, 7819, 0, NULL, NULL}, +{121, 7823, 0, NULL, NULL}, +{122, 380, 0, NULL, NULL}, +{383, 7835, 0, NULL, NULL}, +{769, 0, 2, compose_tab_6_36, hash_compose_tab_6_36}, +{774, 784, 0, NULL, NULL}, +{780, 0, 2, compose_tab_6_38, hash_compose_tab_6_38}, +{803, 0, 2, compose_tab_6_39, hash_compose_tab_6_39} +}; /* compose_tab_6 */ +static int hash_compose_tab_7_23[4] = +{1,-1,-1,0}; /* hash_compose_tab_7_23 */ +static CompEntry compose_tab_7_23[] = { +{79, 7758, 0, NULL, NULL}, +{111, 7759, 0, NULL, NULL} +}; /* compose_tab_7_23 */ +static int hash_compose_tab_7_24[4] = +{-1,0,1,-1}; /* hash_compose_tab_7_24 */ +static CompEntry compose_tab_7_24[] = { +{85, 7802, 0, NULL, NULL}, +{117, 7803, 0, NULL, NULL} +}; /* compose_tab_7_24 */ +static int hash_compose_tab_7[100] = +{48,10,21,-1,11,12,-1,-1,-1,-1,49,13,-1,-1,-1,20,14,15,-1,16,17,18,25,-1,-1, + -1,-1,-1,-1,22,30,-1,-1,26,-1,-1,-1,-1,-1,-1,31,-1,-1,-1,-1,32,33,34,35,-1, + -1,-1,-1,27,36,-1,-1,-1,-1,37,-1,-1,-1,38,-1,0,28,39,-1,1,-1,23,2,3,24,40,-1, + 41,29,4,42,43,44,-1,-1,5,45,6,7,8,-1,46,-1,-1,-1,47,-1,9,-1,19}; /* hash_compose_tab_7 */ +static CompEntry compose_tab_7[] = { +{65, 196, 0, NULL, NULL}, +{69, 203, 0, NULL, NULL}, +{72, 7718, 0, NULL, NULL}, +{73, 207, 0, NULL, NULL}, +{79, 214, 0, NULL, NULL}, +{85, 220, 0, NULL, NULL}, +{87, 7812, 0, NULL, NULL}, +{88, 7820, 0, NULL, NULL}, +{89, 376, 0, NULL, NULL}, +{97, 228, 0, NULL, NULL}, +{101, 235, 0, NULL, NULL}, +{104, 7719, 0, NULL, NULL}, +{105, 239, 0, NULL, NULL}, +{111, 246, 0, NULL, NULL}, +{116, 7831, 0, NULL, NULL}, +{117, 252, 0, NULL, NULL}, +{119, 7813, 0, NULL, NULL}, +{120, 7821, 0, NULL, NULL}, +{121, 255, 0, NULL, NULL}, +{399, 1242, 0, NULL, NULL}, +{415, 1258, 0, NULL, NULL}, +{601, 1243, 0, NULL, NULL}, +{629, 1259, 0, NULL, NULL}, +{771, 0, 2, compose_tab_7_23, hash_compose_tab_7_23}, +{772, 0, 2, compose_tab_7_24, hash_compose_tab_7_24}, +{921, 938, 0, NULL, NULL}, +{933, 939, 0, NULL, NULL}, +{953, 970, 0, NULL, NULL}, +{965, 971, 0, NULL, NULL}, +{978, 980, 0, NULL, NULL}, +{1030, 1031, 0, NULL, NULL}, +{1040, 1234, 0, NULL, NULL}, +{1045, 1025, 0, NULL, NULL}, +{1046, 1244, 0, NULL, NULL}, +{1047, 1246, 0, NULL, NULL}, +{1048, 1252, 0, NULL, NULL}, +{1054, 1254, 0, NULL, NULL}, +{1059, 1264, 0, NULL, NULL}, +{1063, 1268, 0, NULL, NULL}, +{1067, 1272, 0, NULL, NULL}, +{1072, 1235, 0, NULL, NULL}, +{1077, 1105, 0, NULL, NULL}, +{1078, 1245, 0, NULL, NULL}, +{1079, 1247, 0, NULL, NULL}, +{1080, 1253, 0, NULL, NULL}, +{1086, 1255, 0, NULL, NULL}, +{1091, 1265, 0, NULL, NULL}, +{1095, 1269, 0, NULL, NULL}, +{1099, 1273, 0, NULL, NULL}, +{1110, 1111, 0, NULL, NULL} +}; /* compose_tab_7 */ +static int hash_compose_tab_8_12[12] = +{-1,3,-1,5,-1,0,4,2,-1,1,-1,-1}; /* hash_compose_tab_8_12 */ +static CompEntry compose_tab_8_12[] = { +{65, 7848, 0, NULL, NULL}, +{69, 7874, 0, NULL, NULL}, +{79, 7892, 0, NULL, NULL}, +{97, 7849, 0, NULL, NULL}, +{101, 7875, 0, NULL, NULL}, +{111, 7893, 0, NULL, NULL} +}; /* compose_tab_8_12 */ +static int hash_compose_tab_8_13[4] = +{-1,0,1,-1}; /* hash_compose_tab_8_13 */ +static CompEntry compose_tab_8_13[] = { +{65, 7858, 0, NULL, NULL}, +{97, 7859, 0, NULL, NULL} +}; /* compose_tab_8_13 */ +static int hash_compose_tab_8_14[8] = +{2,-1,-1,-1,-1,1,3,0}; /* hash_compose_tab_8_14 */ +static CompEntry compose_tab_8_14[] = { +{79, 7902, 0, NULL, NULL}, +{85, 7916, 0, NULL, NULL}, +{111, 7903, 0, NULL, NULL}, +{117, 7917, 0, NULL, NULL} +}; /* compose_tab_8_14 */ +static int hash_compose_tab_8[30] = +{-1,11,-1,-1,-1,0,-1,6,-1,1,-1,7,-1,2,-1,8,14,-1,-1,3,12,9,-1,-1,13,4,-1,10, + -1,5}; /* hash_compose_tab_8 */ +static CompEntry compose_tab_8[] = { +{65, 7842, 0, NULL, NULL}, +{69, 7866, 0, NULL, NULL}, +{73, 7880, 0, NULL, NULL}, +{79, 7886, 0, NULL, NULL}, +{85, 7910, 0, NULL, NULL}, +{89, 7926, 0, NULL, NULL}, +{97, 7843, 0, NULL, NULL}, +{101, 7867, 0, NULL, NULL}, +{105, 7881, 0, NULL, NULL}, +{111, 7887, 0, NULL, NULL}, +{117, 7911, 0, NULL, NULL}, +{121, 7927, 0, NULL, NULL}, +{770, 0, 6, compose_tab_8_12, hash_compose_tab_8_12}, +{774, 0, 2, compose_tab_8_13, hash_compose_tab_8_13}, +{795, 0, 4, compose_tab_8_14, hash_compose_tab_8_14} +}; /* compose_tab_8 */ +static int hash_compose_tab_9[12] = +{-1,1,2,5,-1,0,-1,-1,-1,3,-1,4}; /* hash_compose_tab_9 */ +static CompEntry compose_tab_9[] = { +{65, 197, 0, NULL, NULL}, +{85, 366, 0, NULL, NULL}, +{97, 229, 0, NULL, NULL}, +{117, 367, 0, NULL, NULL}, +{119, 7832, 0, NULL, NULL}, +{121, 7833, 0, NULL, NULL} +}; /* compose_tab_9 */ +static int hash_compose_tab_10[12] = +{-1,1,-1,2,4,-1,-1,0,-1,3,-1,5}; /* hash_compose_tab_10 */ +static CompEntry compose_tab_10[] = { +{79, 336, 0, NULL, NULL}, +{85, 368, 0, NULL, NULL}, +{111, 337, 0, NULL, NULL}, +{117, 369, 0, NULL, NULL}, +{1059, 1266, 0, NULL, NULL}, +{1091, 1267, 0, NULL, NULL} +}; /* compose_tab_10 */ +static int hash_compose_tab_11_33[4] = +{-1,0,1,-1}; /* hash_compose_tab_11_33 */ +static CompEntry compose_tab_11_33[] = { +{85, 473, 0, NULL, NULL}, +{117, 474, 0, NULL, NULL} +}; /* compose_tab_11_33 */ +static int hash_compose_tab_11[68] = +{2,3,-1,4,-1,5,-1,6,7,-1,8,9,-1,-1,10,11,12,13,-1,-1,-1,-1,14,-1,-1,-1,-1,-1, + 33,15,-1,16,17,18,31,19,-1,20,21,22,23,-1,24,25,-1,-1,26,27,28,29,32,-1,-1, + -1,30,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,-1,1}; /* hash_compose_tab_11 */ +static CompEntry compose_tab_11[] = { +{65, 461, 0, NULL, NULL}, +{67, 268, 0, NULL, NULL}, +{68, 270, 0, NULL, NULL}, +{69, 282, 0, NULL, NULL}, +{71, 486, 0, NULL, NULL}, +{73, 463, 0, NULL, NULL}, +{75, 488, 0, NULL, NULL}, +{76, 317, 0, NULL, NULL}, +{78, 327, 0, NULL, NULL}, +{79, 465, 0, NULL, NULL}, +{82, 344, 0, NULL, NULL}, +{83, 352, 0, NULL, NULL}, +{84, 356, 0, NULL, NULL}, +{85, 467, 0, NULL, NULL}, +{90, 381, 0, NULL, NULL}, +{97, 462, 0, NULL, NULL}, +{99, 269, 0, NULL, NULL}, +{100, 271, 0, NULL, NULL}, +{101, 283, 0, NULL, NULL}, +{103, 487, 0, NULL, NULL}, +{105, 464, 0, NULL, NULL}, +{106, 496, 0, NULL, NULL}, +{107, 489, 0, NULL, NULL}, +{108, 318, 0, NULL, NULL}, +{110, 328, 0, NULL, NULL}, +{111, 466, 0, NULL, NULL}, +{114, 345, 0, NULL, NULL}, +{115, 353, 0, NULL, NULL}, +{116, 357, 0, NULL, NULL}, +{117, 468, 0, NULL, NULL}, +{122, 382, 0, NULL, NULL}, +{439, 494, 0, NULL, NULL}, +{658, 495, 0, NULL, NULL}, +{776, 0, 2, compose_tab_11_33, hash_compose_tab_11_33} +}; /* compose_tab_11 */ +static int hash_compose_tab_12_1[4] = +{-1,0,1,-1}; /* hash_compose_tab_12_1 */ +static CompEntry compose_tab_12_1[] = { +{953, 912, 0, NULL, NULL}, +{965, 944, 0, NULL, NULL} +}; /* compose_tab_12_1 */ +static int hash_compose_tab_12[34] = +{11,4,12,5,-1,-1,-1,13,-1,6,-1,-1,-1,14,-1,7,-1,15,-1,8,-1,-1,-1,-1,-1,-1,16, + 9,1,2,-1,10,0,3}; /* hash_compose_tab_12 */ +static CompEntry compose_tab_12[] = { +{168, 901, 0, NULL, NULL}, +{776, 0, 2, compose_tab_12_1, hash_compose_tab_12_1}, +{913, 902, 0, NULL, NULL}, +{917, 904, 0, NULL, NULL}, +{919, 905, 0, NULL, NULL}, +{921, 906, 0, NULL, NULL}, +{927, 908, 0, NULL, NULL}, +{933, 910, 0, NULL, NULL}, +{937, 911, 0, NULL, NULL}, +{945, 940, 0, NULL, NULL}, +{949, 941, 0, NULL, NULL}, +{951, 942, 0, NULL, NULL}, +{953, 943, 0, NULL, NULL}, +{959, 972, 0, NULL, NULL}, +{965, 973, 0, NULL, NULL}, +{969, 974, 0, NULL, NULL}, +{978, 979, 0, NULL, NULL} +}; /* compose_tab_12 */ +static int hash_compose_tab_13[28] = +{-1,5,10,-1,-1,11,-1,-1,-1,0,-1,-1,-1,1,6,-1,-1,2,7,-1,12,8,13,3,-1,-1,4,9}; /* hash_compose_tab_13 */ +static CompEntry compose_tab_13[] = { +{65, 512, 0, NULL, NULL}, +{69, 516, 0, NULL, NULL}, +{73, 520, 0, NULL, NULL}, +{79, 524, 0, NULL, NULL}, +{82, 528, 0, NULL, NULL}, +{85, 532, 0, NULL, NULL}, +{97, 513, 0, NULL, NULL}, +{101, 517, 0, NULL, NULL}, +{105, 521, 0, NULL, NULL}, +{111, 525, 0, NULL, NULL}, +{114, 529, 0, NULL, NULL}, +{117, 533, 0, NULL, NULL}, +{1140, 1142, 0, NULL, NULL}, +{1141, 1143, 0, NULL, NULL} +}; /* compose_tab_13 */ +static int hash_compose_tab_14[24] = +{-1,2,6,-1,-1,7,-1,3,-1,8,4,-1,-1,5,-1,9,-1,0,10,-1,-1,1,11,-1}; /* hash_compose_tab_14 */ +static CompEntry compose_tab_14[] = { +{65, 514, 0, NULL, NULL}, +{69, 518, 0, NULL, NULL}, +{73, 522, 0, NULL, NULL}, +{79, 526, 0, NULL, NULL}, +{82, 530, 0, NULL, NULL}, +{85, 534, 0, NULL, NULL}, +{97, 515, 0, NULL, NULL}, +{101, 519, 0, NULL, NULL}, +{105, 523, 0, NULL, NULL}, +{111, 527, 0, NULL, NULL}, +{114, 531, 0, NULL, NULL}, +{117, 535, 0, NULL, NULL} +}; /* compose_tab_14 */ +static int hash_compose_tab_15_0[12] = +{-1,0,2,4,-1,-1,-1,1,-1,3,5,-1}; /* hash_compose_tab_15_0 */ +static CompEntry compose_tab_15_0[] = { +{913, 8072, 0, NULL, NULL}, +{919, 8088, 0, NULL, NULL}, +{937, 8104, 0, NULL, NULL}, +{945, 8064, 0, NULL, NULL}, +{951, 8080, 0, NULL, NULL}, +{969, 8096, 0, NULL, NULL} +}; /* compose_tab_15_0 */ +static int hash_compose_tab_15[30] = +{-1,12,-1,-1,-1,13,-1,6,-1,14,-1,-1,-1,1,-1,7,-1,2,-1,3,8,4,9,10,-1,-1,-1,0,5, + 11}; /* hash_compose_tab_15 */ +static CompEntry compose_tab_15[] = { +{837, 0, 6, compose_tab_15_0, hash_compose_tab_15_0}, +{913, 7944, 0, NULL, NULL}, +{917, 7960, 0, NULL, NULL}, +{919, 7976, 0, NULL, NULL}, +{921, 7992, 0, NULL, NULL}, +{927, 8008, 0, NULL, NULL}, +{937, 8040, 0, NULL, NULL}, +{945, 7936, 0, NULL, NULL}, +{949, 7952, 0, NULL, NULL}, +{951, 7968, 0, NULL, NULL}, +{953, 7984, 0, NULL, NULL}, +{959, 8000, 0, NULL, NULL}, +{961, 8164, 0, NULL, NULL}, +{965, 8016, 0, NULL, NULL}, +{969, 8032, 0, NULL, NULL} +}; /* compose_tab_15 */ +static int hash_compose_tab_16_0[12] = +{-1,0,2,4,-1,-1,-1,1,-1,3,5,-1}; /* hash_compose_tab_16_0 */ +static CompEntry compose_tab_16_0[] = { +{913, 8073, 0, NULL, NULL}, +{919, 8089, 0, NULL, NULL}, +{937, 8105, 0, NULL, NULL}, +{945, 8065, 0, NULL, NULL}, +{951, 8081, 0, NULL, NULL}, +{969, 8097, 0, NULL, NULL} +}; /* compose_tab_16_0 */ +static int hash_compose_tab_16[34] = +{11,3,12,4,-1,-1,-1,13,-1,5,14,6,-1,15,-1,7,-1,16,-1,8,-1,0,-1,-1,-1,-1,-1,9, + -1,1,-1,10,-1,2}; /* hash_compose_tab_16 */ +static CompEntry compose_tab_16[] = { +{837, 0, 6, compose_tab_16_0, hash_compose_tab_16_0}, +{913, 7945, 0, NULL, NULL}, +{917, 7961, 0, NULL, NULL}, +{919, 7977, 0, NULL, NULL}, +{921, 7993, 0, NULL, NULL}, +{927, 8009, 0, NULL, NULL}, +{929, 8172, 0, NULL, NULL}, +{933, 8025, 0, NULL, NULL}, +{937, 8041, 0, NULL, NULL}, +{945, 7937, 0, NULL, NULL}, +{949, 7953, 0, NULL, NULL}, +{951, 7969, 0, NULL, NULL}, +{953, 7985, 0, NULL, NULL}, +{959, 8001, 0, NULL, NULL}, +{961, 8165, 0, NULL, NULL}, +{965, 8017, 0, NULL, NULL}, +{969, 8033, 0, NULL, NULL} +}; /* compose_tab_16 */ +static int hash_compose_tab_17[8] = +{2,-1,-1,-1,-1,1,3,0}; /* hash_compose_tab_17 */ +static CompEntry compose_tab_17[] = { +{79, 416, 0, NULL, NULL}, +{85, 431, 0, NULL, NULL}, +{111, 417, 0, NULL, NULL}, +{117, 432, 0, NULL, NULL} +}; /* compose_tab_17 */ +static int hash_compose_tab_18_38[8] = +{2,-1,-1,-1,-1,1,3,0}; /* hash_compose_tab_18_38 */ +static CompEntry compose_tab_18_38[] = { +{79, 7906, 0, NULL, NULL}, +{85, 7920, 0, NULL, NULL}, +{111, 7907, 0, NULL, NULL}, +{117, 7921, 0, NULL, NULL} +}; /* compose_tab_18_38 */ +static int hash_compose_tab_18[78] = +{9,10,-1,-1,11,12,13,14,15,16,-1,17,18,-1,-1,38,-1,-1,-1,19,20,-1,21,22,-1,-1, + 23,24,-1,25,26,27,28,29,-1,-1,30,31,32,33,34,35,-1,36,37,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,1,-1,2,3,-1,-1,4,5,-1,6,7,8}; /* hash_compose_tab_18 */ +static CompEntry compose_tab_18[] = { +{65, 7840, 0, NULL, NULL}, +{66, 7684, 0, NULL, NULL}, +{68, 7692, 0, NULL, NULL}, +{69, 7864, 0, NULL, NULL}, +{72, 7716, 0, NULL, NULL}, +{73, 7882, 0, NULL, NULL}, +{75, 7730, 0, NULL, NULL}, +{76, 7734, 0, NULL, NULL}, +{77, 7746, 0, NULL, NULL}, +{78, 7750, 0, NULL, NULL}, +{79, 7884, 0, NULL, NULL}, +{82, 7770, 0, NULL, NULL}, +{83, 7778, 0, NULL, NULL}, +{84, 7788, 0, NULL, NULL}, +{85, 7908, 0, NULL, NULL}, +{86, 7806, 0, NULL, NULL}, +{87, 7816, 0, NULL, NULL}, +{89, 7924, 0, NULL, NULL}, +{90, 7826, 0, NULL, NULL}, +{97, 7841, 0, NULL, NULL}, +{98, 7685, 0, NULL, NULL}, +{100, 7693, 0, NULL, NULL}, +{101, 7865, 0, NULL, NULL}, +{104, 7717, 0, NULL, NULL}, +{105, 7883, 0, NULL, NULL}, +{107, 7731, 0, NULL, NULL}, +{108, 7735, 0, NULL, NULL}, +{109, 7747, 0, NULL, NULL}, +{110, 7751, 0, NULL, NULL}, +{111, 7885, 0, NULL, NULL}, +{114, 7771, 0, NULL, NULL}, +{115, 7779, 0, NULL, NULL}, +{116, 7789, 0, NULL, NULL}, +{117, 7909, 0, NULL, NULL}, +{118, 7807, 0, NULL, NULL}, +{119, 7817, 0, NULL, NULL}, +{121, 7925, 0, NULL, NULL}, +{122, 7827, 0, NULL, NULL}, +{795, 0, 4, compose_tab_18_38, hash_compose_tab_18_38} +}; /* compose_tab_18 */ +static int hash_compose_tab_19[4] = +{-1,0,1,-1}; /* hash_compose_tab_19 */ +static CompEntry compose_tab_19[] = { +{85, 7794, 0, NULL, NULL}, +{117, 7795, 0, NULL, NULL} +}; /* compose_tab_19 */ +static int hash_compose_tab_20[4] = +{-1,0,1,-1}; /* hash_compose_tab_20 */ +static CompEntry compose_tab_20[] = { +{65, 7680, 0, NULL, NULL}, +{97, 7681, 0, NULL, NULL} +}; /* compose_tab_20 */ +static int hash_compose_tab_21[40] = +{-1,-1,7,8,9,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,10,11,-1,-1,12,13,-1, + -1,0,1,14,15,2,3,16,17,4,5,18,6,19}; /* hash_compose_tab_21 */ +static CompEntry compose_tab_21[] = { +{67, 199, 0, NULL, NULL}, +{68, 7696, 0, NULL, NULL}, +{71, 290, 0, NULL, NULL}, +{72, 7720, 0, NULL, NULL}, +{75, 310, 0, NULL, NULL}, +{76, 315, 0, NULL, NULL}, +{78, 325, 0, NULL, NULL}, +{82, 342, 0, NULL, NULL}, +{83, 350, 0, NULL, NULL}, +{84, 354, 0, NULL, NULL}, +{99, 231, 0, NULL, NULL}, +{100, 7697, 0, NULL, NULL}, +{103, 291, 0, NULL, NULL}, +{104, 7721, 0, NULL, NULL}, +{107, 311, 0, NULL, NULL}, +{108, 316, 0, NULL, NULL}, +{110, 326, 0, NULL, NULL}, +{114, 343, 0, NULL, NULL}, +{115, 351, 0, NULL, NULL}, +{116, 355, 0, NULL, NULL} +}; /* compose_tab_21 */ +static int hash_compose_tab_22[20] = +{-1,6,-1,-1,-1,0,4,7,-1,1,-1,8,-1,2,-1,-1,-1,5,9,3}; /* hash_compose_tab_22 */ +static CompEntry compose_tab_22[] = { +{65, 260, 0, NULL, NULL}, +{69, 280, 0, NULL, NULL}, +{73, 302, 0, NULL, NULL}, +{79, 490, 0, NULL, NULL}, +{85, 370, 0, NULL, NULL}, +{97, 261, 0, NULL, NULL}, +{101, 281, 0, NULL, NULL}, +{105, 303, 0, NULL, NULL}, +{111, 491, 0, NULL, NULL}, +{117, 371, 0, NULL, NULL} +}; /* compose_tab_22 */ +static int hash_compose_tab_23[24] = +{-1,-1,-1,-1,2,6,3,7,-1,-1,-1,-1,4,5,8,9,-1,-1,-1,-1,0,1,10,11}; /* hash_compose_tab_23 */ +static CompEntry compose_tab_23[] = { +{68, 7698, 0, NULL, NULL}, +{69, 7704, 0, NULL, NULL}, +{76, 7740, 0, NULL, NULL}, +{78, 7754, 0, NULL, NULL}, +{84, 7792, 0, NULL, NULL}, +{85, 7798, 0, NULL, NULL}, +{100, 7699, 0, NULL, NULL}, +{101, 7705, 0, NULL, NULL}, +{108, 7741, 0, NULL, NULL}, +{110, 7755, 0, NULL, NULL}, +{116, 7793, 0, NULL, NULL}, +{117, 7799, 0, NULL, NULL} +}; /* compose_tab_23 */ +static int hash_compose_tab_24[4] = +{0,1,-1,-1}; /* hash_compose_tab_24 */ +static CompEntry compose_tab_24[] = { +{72, 7722, 0, NULL, NULL}, +{104, 7723, 0, NULL, NULL} +}; /* compose_tab_24 */ +static int hash_compose_tab_25[12] = +{-1,1,2,-1,-1,3,-1,-1,-1,0,4,5}; /* hash_compose_tab_25 */ +static CompEntry compose_tab_25[] = { +{69, 7706, 0, NULL, NULL}, +{73, 7724, 0, NULL, NULL}, +{85, 7796, 0, NULL, NULL}, +{101, 7707, 0, NULL, NULL}, +{105, 7725, 0, NULL, NULL}, +{117, 7797, 0, NULL, NULL} +}; /* compose_tab_25 */ +static int hash_compose_tab_26[34] = +{1,-1,10,-1,-1,11,12,2,3,13,4,-1,14,-1,5,15,6,-1,-1,-1,16,-1,7,-1,-1,-1,-1,-1, + -1,-1,8,-1,0,9}; /* hash_compose_tab_26 */ +static CompEntry compose_tab_26[] = { +{66, 7686, 0, NULL, NULL}, +{68, 7694, 0, NULL, NULL}, +{75, 7732, 0, NULL, NULL}, +{76, 7738, 0, NULL, NULL}, +{78, 7752, 0, NULL, NULL}, +{82, 7774, 0, NULL, NULL}, +{84, 7790, 0, NULL, NULL}, +{90, 7828, 0, NULL, NULL}, +{98, 7687, 0, NULL, NULL}, +{100, 7695, 0, NULL, NULL}, +{104, 7830, 0, NULL, NULL}, +{107, 7733, 0, NULL, NULL}, +{108, 7739, 0, NULL, NULL}, +{110, 7753, 0, NULL, NULL}, +{114, 7775, 0, NULL, NULL}, +{116, 7791, 0, NULL, NULL}, +{122, 7829, 0, NULL, NULL} +}; /* compose_tab_26 */ +static int hash_compose_tab_27_1[4] = +{-1,0,1,-1}; /* hash_compose_tab_27_1 */ +static CompEntry compose_tab_27_1[] = { +{953, 8151, 0, NULL, NULL}, +{965, 8167, 0, NULL, NULL} +}; /* compose_tab_27_1 */ +static int hash_compose_tab_27_2_0[12] = +{-1,0,2,4,-1,-1,-1,1,-1,3,5,-1}; /* hash_compose_tab_27_2_0 */ +static CompEntry compose_tab_27_2_0[] = { +{913, 8078, 0, NULL, NULL}, +{919, 8094, 0, NULL, NULL}, +{937, 8110, 0, NULL, NULL}, +{945, 8070, 0, NULL, NULL}, +{951, 8086, 0, NULL, NULL}, +{969, 8102, 0, NULL, NULL} +}; /* compose_tab_27_2_0 */ +static int hash_compose_tab_27_2[20] = +{-1,3,-1,-1,-1,5,8,-1,-1,9,-1,6,-1,1,7,-1,-1,0,4,2}; /* hash_compose_tab_27_2 */ +static CompEntry compose_tab_27_2[] = { +{837, 0, 6, compose_tab_27_2_0, hash_compose_tab_27_2_0}, +{913, 7950, 0, NULL, NULL}, +{919, 7982, 0, NULL, NULL}, +{921, 7998, 0, NULL, NULL}, +{937, 8046, 0, NULL, NULL}, +{945, 7942, 0, NULL, NULL}, +{951, 7974, 0, NULL, NULL}, +{953, 7990, 0, NULL, NULL}, +{965, 8022, 0, NULL, NULL}, +{969, 8038, 0, NULL, NULL} +}; /* compose_tab_27_2 */ +static int hash_compose_tab_27_3_0[12] = +{-1,0,2,4,-1,-1,-1,1,-1,3,5,-1}; /* hash_compose_tab_27_3_0 */ +static CompEntry compose_tab_27_3_0[] = { +{913, 8079, 0, NULL, NULL}, +{919, 8095, 0, NULL, NULL}, +{937, 8111, 0, NULL, NULL}, +{945, 8071, 0, NULL, NULL}, +{951, 8087, 0, NULL, NULL}, +{969, 8103, 0, NULL, NULL} +}; /* compose_tab_27_3_0 */ +static int hash_compose_tab_27_3[22] = +{-1,0,10,-1,-1,7,-1,8,-1,4,-1,1,-1,5,-1,-1,-1,2,-1,3,9,6}; /* hash_compose_tab_27_3 */ +static CompEntry compose_tab_27_3[] = { +{837, 0, 6, compose_tab_27_3_0, hash_compose_tab_27_3_0}, +{913, 7951, 0, NULL, NULL}, +{919, 7983, 0, NULL, NULL}, +{921, 7999, 0, NULL, NULL}, +{933, 8031, 0, NULL, NULL}, +{937, 8047, 0, NULL, NULL}, +{945, 7943, 0, NULL, NULL}, +{951, 7975, 0, NULL, NULL}, +{953, 7991, 0, NULL, NULL}, +{965, 8023, 0, NULL, NULL}, +{969, 8039, 0, NULL, NULL} +}; /* compose_tab_27_3 */ +static int hash_compose_tab_27_4[6] = +{-1,-1,-1,0,1,2}; /* hash_compose_tab_27_4 */ +static CompEntry compose_tab_27_4[] = { +{945, 8119, 0, NULL, NULL}, +{951, 8135, 0, NULL, NULL}, +{969, 8183, 0, NULL, NULL} +}; /* compose_tab_27_4 */ +static int hash_compose_tab_27[24] = +{0,-1,-1,-1,-1,8,11,-1,1,5,9,-1,-1,-1,-1,6,10,7,-1,2,3,4,-1,-1}; /* hash_compose_tab_27 */ +static CompEntry compose_tab_27[] = { +{168, 8129, 0, NULL, NULL}, +{776, 0, 2, compose_tab_27_1, hash_compose_tab_27_1}, +{787, 0, 10, compose_tab_27_2, hash_compose_tab_27_2}, +{788, 0, 11, compose_tab_27_3, hash_compose_tab_27_3}, +{837, 0, 3, compose_tab_27_4, hash_compose_tab_27_4}, +{945, 8118, 0, NULL, NULL}, +{951, 8134, 0, NULL, NULL}, +{953, 8150, 0, NULL, NULL}, +{965, 8166, 0, NULL, NULL}, +{969, 8182, 0, NULL, NULL}, +{8127, 8143, 0, NULL, NULL}, +{8190, 8159, 0, NULL, NULL} +}; /* compose_tab_27 */ +static int hash_compose_tab_28[12] = +{-1,0,2,4,-1,-1,-1,1,-1,3,5,-1}; /* hash_compose_tab_28 */ +static CompEntry compose_tab_28[] = { +{913, 8124, 0, NULL, NULL}, +{919, 8140, 0, NULL, NULL}, +{937, 8188, 0, NULL, NULL}, +{945, 8115, 0, NULL, NULL}, +{951, 8131, 0, NULL, NULL}, +{969, 8179, 0, NULL, NULL} +}; /* compose_tab_28 */ +static int hash_compose_tab_29[4] = +{0,-1,1,-1}; /* hash_compose_tab_29 */ +static CompEntry compose_tab_29[] = { +{1488, 64302, 0, NULL, NULL}, +{1522, 64287, 0, NULL, NULL} +}; /* compose_tab_29 */ +static int hash_compose_tab_30[2] = +{0,-1}; /* hash_compose_tab_30 */ +static CompEntry compose_tab_30[] = { +{1488, 64303, 0, NULL, NULL} +}; /* compose_tab_30 */ +static int hash_compose_tab_31[2] = +{-1,0}; /* hash_compose_tab_31 */ +static CompEntry compose_tab_31[] = { +{1493, 64331, 0, NULL, NULL} +}; /* compose_tab_31 */ +static int hash_compose_tab_32[44] = +{7,8,9,10,11,-1,12,-1,13,14,-1,15,16,-1,17,18,19,20,21,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,1,2,3,4,5,6,-1}; /* hash_compose_tab_32 */ +static CompEntry compose_tab_32[] = { +{1488, 64304, 0, NULL, NULL}, +{1489, 64305, 0, NULL, NULL}, +{1490, 64306, 0, NULL, NULL}, +{1491, 64307, 0, NULL, NULL}, +{1492, 64308, 0, NULL, NULL}, +{1493, 64309, 0, NULL, NULL}, +{1494, 64310, 0, NULL, NULL}, +{1496, 64312, 0, NULL, NULL}, +{1497, 64313, 0, NULL, NULL}, +{1498, 64314, 0, NULL, NULL}, +{1499, 64315, 0, NULL, NULL}, +{1500, 64316, 0, NULL, NULL}, +{1502, 64318, 0, NULL, NULL}, +{1504, 64320, 0, NULL, NULL}, +{1505, 64321, 0, NULL, NULL}, +{1507, 64323, 0, NULL, NULL}, +{1508, 64324, 0, NULL, NULL}, +{1510, 64326, 0, NULL, NULL}, +{1511, 64327, 0, NULL, NULL}, +{1512, 64328, 0, NULL, NULL}, +{1513, 64329, 0, NULL, NULL}, +{1514, 64330, 0, NULL, NULL} +}; /* compose_tab_32 */ +static int hash_compose_tab_33[6] = +{-1,0,2,-1,-1,1}; /* hash_compose_tab_33 */ +static CompEntry compose_tab_33[] = { +{1489, 64332, 0, NULL, NULL}, +{1499, 64333, 0, NULL, NULL}, +{1508, 64334, 0, NULL, NULL} +}; /* compose_tab_33 */ +static int hash_compose_tab_34_0[2] = +{-1,0}; /* hash_compose_tab_34_0 */ +static CompEntry compose_tab_34_0[] = { +{1513, 64300, 0, NULL, NULL} +}; /* compose_tab_34_0 */ +static int hash_compose_tab_34[4] = +{0,1,-1,-1}; /* hash_compose_tab_34 */ +static CompEntry compose_tab_34[] = { +{1468, 0, 1, compose_tab_34_0, hash_compose_tab_34_0}, +{1513, 64298, 0, NULL, NULL} +}; /* compose_tab_34 */ +static int hash_compose_tab_35_0[2] = +{-1,0}; /* hash_compose_tab_35_0 */ +static CompEntry compose_tab_35_0[] = { +{1513, 64301, 0, NULL, NULL} +}; /* compose_tab_35_0 */ +static int hash_compose_tab_35[4] = +{0,1,-1,-1}; /* hash_compose_tab_35 */ +static CompEntry compose_tab_35[] = { +{1468, 0, 1, compose_tab_35_0, hash_compose_tab_35_0}, +{1513, 64299, 0, NULL, NULL} +}; /* compose_tab_35 */ +static int hash_compose_tab_36[22] = +{3,10,-1,-1,-1,4,5,-1,-1,-1,-1,-1,6,-1,-1,0,1,2,7,8,9,-1}; /* hash_compose_tab_36 */ +static CompEntry compose_tab_36[] = { +{2325, 2392, 0, NULL, NULL}, +{2326, 2393, 0, NULL, NULL}, +{2327, 2394, 0, NULL, NULL}, +{2332, 2395, 0, NULL, NULL}, +{2337, 2396, 0, NULL, NULL}, +{2338, 2397, 0, NULL, NULL}, +{2344, 2345, 0, NULL, NULL}, +{2347, 2398, 0, NULL, NULL}, +{2351, 2399, 0, NULL, NULL}, +{2352, 2353, 0, NULL, NULL}, +{2355, 2356, 0, NULL, NULL} +}; /* compose_tab_36 */ +static int hash_compose_tab_37[8] = +{-1,0,1,-1,2,-1,-1,3}; /* hash_compose_tab_37 */ +static CompEntry compose_tab_37[] = { +{2465, 2524, 0, NULL, NULL}, +{2466, 2525, 0, NULL, NULL}, +{2476, 2480, 0, NULL, NULL}, +{2479, 2527, 0, NULL, NULL} +}; /* compose_tab_37 */ +static int hash_compose_tab_38[2] = +{-1,0}; /* hash_compose_tab_38 */ +static CompEntry compose_tab_38[] = { +{2503, 2507, 0, NULL, NULL} +}; /* compose_tab_38 */ +static int hash_compose_tab_39[2] = +{-1,0}; /* hash_compose_tab_39 */ +static CompEntry compose_tab_39[] = { +{2503, 2508, 0, NULL, NULL} +}; /* compose_tab_39 */ +static int hash_compose_tab_40[10] = +{-1,-1,0,1,3,4,-1,-1,2,-1}; /* hash_compose_tab_40 */ +static CompEntry compose_tab_40[] = { +{2582, 2649, 0, NULL, NULL}, +{2583, 2650, 0, NULL, NULL}, +{2588, 2651, 0, NULL, NULL}, +{2593, 2652, 0, NULL, NULL}, +{2603, 2654, 0, NULL, NULL} +}; /* compose_tab_40 */ +static int hash_compose_tab_41[6] = +{1,2,-1,-1,-1,0}; /* hash_compose_tab_41 */ +static CompEntry compose_tab_41[] = { +{2849, 2908, 0, NULL, NULL}, +{2850, 2909, 0, NULL, NULL}, +{2863, 2911, 0, NULL, NULL} +}; /* compose_tab_41 */ +static int hash_compose_tab_42[2] = +{-1,0}; /* hash_compose_tab_42 */ +static CompEntry compose_tab_42[] = { +{2887, 2891, 0, NULL, NULL} +}; /* compose_tab_42 */ +static int hash_compose_tab_43[2] = +{-1,0}; /* hash_compose_tab_43 */ +static CompEntry compose_tab_43[] = { +{2887, 2888, 0, NULL, NULL} +}; /* compose_tab_43 */ +static int hash_compose_tab_44[2] = +{-1,0}; /* hash_compose_tab_44 */ +static CompEntry compose_tab_44[] = { +{2887, 2892, 0, NULL, NULL} +}; /* compose_tab_44 */ +static int hash_compose_tab_45[4] = +{-1,-1,0,1}; /* hash_compose_tab_45 */ +static CompEntry compose_tab_45[] = { +{3014, 3018, 0, NULL, NULL}, +{3015, 3019, 0, NULL, NULL} +}; /* compose_tab_45 */ +static int hash_compose_tab_46[4] = +{-1,-1,0,1}; /* hash_compose_tab_46 */ +static CompEntry compose_tab_46[] = { +{2962, 2964, 0, NULL, NULL}, +{3014, 3020, 0, NULL, NULL} +}; /* compose_tab_46 */ +static int hash_compose_tab_47[2] = +{0,-1}; /* hash_compose_tab_47 */ +static CompEntry compose_tab_47[] = { +{3142, 3144, 0, NULL, NULL} +}; /* compose_tab_47 */ +static int hash_compose_tab_48[2] = +{0,-1}; /* hash_compose_tab_48 */ +static CompEntry compose_tab_48[] = { +{3270, 3274, 0, NULL, NULL} +}; /* compose_tab_48 */ +static int hash_compose_tab_49_1[2] = +{0,-1}; /* hash_compose_tab_49_1 */ +static CompEntry compose_tab_49_1[] = { +{3270, 3275, 0, NULL, NULL} +}; /* compose_tab_49_1 */ +static int hash_compose_tab_49[6] = +{2,-1,1,-1,-1,0}; /* hash_compose_tab_49 */ +static CompEntry compose_tab_49[] = { +{3263, 3264, 0, NULL, NULL}, +{3266, 0, 1, compose_tab_49_1, hash_compose_tab_49_1}, +{3270, 3271, 0, NULL, NULL} +}; /* compose_tab_49 */ +static int hash_compose_tab_50[2] = +{0,-1}; /* hash_compose_tab_50 */ +static CompEntry compose_tab_50[] = { +{3270, 3272, 0, NULL, NULL} +}; /* compose_tab_50 */ +static int hash_compose_tab_51[4] = +{-1,-1,0,1}; /* hash_compose_tab_51 */ +static CompEntry compose_tab_51[] = { +{3398, 3402, 0, NULL, NULL}, +{3399, 3403, 0, NULL, NULL} +}; /* compose_tab_51 */ +static int hash_compose_tab_52[2] = +{0,-1}; /* hash_compose_tab_52 */ +static CompEntry compose_tab_52[] = { +{3398, 3404, 0, NULL, NULL} +}; /* compose_tab_52 */ +static int hash_compose_tab_53[2] = +{-1,0}; /* hash_compose_tab_53 */ +static CompEntry compose_tab_53[] = { +{3661, 3635, 0, NULL, NULL} +}; /* compose_tab_53 */ +static int hash_compose_tab_54[2] = +{-1,0}; /* hash_compose_tab_54 */ +static CompEntry compose_tab_54[] = { +{3789, 3763, 0, NULL, NULL} +}; /* compose_tab_54 */ +static int hash_compose_tab_55_2[4] = +{-1,-1,0,1}; /* hash_compose_tab_55_2 */ +static CompEntry compose_tab_55_2[] = { +{4018, 3959, 0, NULL, NULL}, +{4019, 3961, 0, NULL, NULL} +}; /* compose_tab_55_2 */ +static int hash_compose_tab_55[6] = +{0,-1,1,2,-1,-1}; /* hash_compose_tab_55 */ +static CompEntry compose_tab_55[] = { +{3954, 3955, 0, NULL, NULL}, +{3956, 3957, 0, NULL, NULL}, +{3968, 0, 2, compose_tab_55_2, hash_compose_tab_55_2} +}; /* compose_tab_55 */ +static int hash_compose_tab_56[4] = +{-1,-1,0,1}; /* hash_compose_tab_56 */ +static CompEntry compose_tab_56[] = { +{4018, 3958, 0, NULL, NULL}, +{4019, 3960, 0, NULL, NULL} +}; /* compose_tab_56 */ +static int hash_compose_tab_57[4] = +{0,1,-1,-1}; /* hash_compose_tab_57 */ +static CompEntry compose_tab_57[] = { +{3904, 3945, 0, NULL, NULL}, +{3984, 4025, 0, NULL, NULL} +}; /* compose_tab_57 */ +static int hash_compose_tab_58[20] = +{-1,2,7,-1,-1,-1,0,3,5,8,-1,4,9,-1,-1,-1,1,6,-1,-1}; /* hash_compose_tab_58 */ +static CompEntry compose_tab_58[] = { +{3906, 3907, 0, NULL, NULL}, +{3916, 3917, 0, NULL, NULL}, +{3921, 3922, 0, NULL, NULL}, +{3926, 3927, 0, NULL, NULL}, +{3931, 3932, 0, NULL, NULL}, +{3986, 3987, 0, NULL, NULL}, +{3996, 3997, 0, NULL, NULL}, +{4001, 4002, 0, NULL, NULL}, +{4006, 4007, 0, NULL, NULL}, +{4011, 4012, 0, NULL, NULL} +}; /* compose_tab_58 */ +static int hash_compose_tab_59[96] = +{33,12,34,-1,13,35,14,36,15,37,-1,-1,-1,-1,-1,16,38,-1,17,39,-1,18,40,-1,19, + 41,-1,20,42,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,43,44,45, + 46,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,21,47,-1,-1,-1,-1,-1,-1,-1,0,22,-1,-1,-1,1, + 23,2,24,3,25,4,26,5,27,6,28,7,29,8,30,9,31,10,32,11}; /* hash_compose_tab_59 */ +static CompEntry compose_tab_59[] = { +{12358, 12436, 0, NULL, NULL}, +{12363, 12364, 0, NULL, NULL}, +{12365, 12366, 0, NULL, NULL}, +{12367, 12368, 0, NULL, NULL}, +{12369, 12370, 0, NULL, NULL}, +{12371, 12372, 0, NULL, NULL}, +{12373, 12374, 0, NULL, NULL}, +{12375, 12376, 0, NULL, NULL}, +{12377, 12378, 0, NULL, NULL}, +{12379, 12380, 0, NULL, NULL}, +{12381, 12382, 0, NULL, NULL}, +{12383, 12384, 0, NULL, NULL}, +{12385, 12386, 0, NULL, NULL}, +{12388, 12389, 0, NULL, NULL}, +{12390, 12391, 0, NULL, NULL}, +{12392, 12393, 0, NULL, NULL}, +{12399, 12400, 0, NULL, NULL}, +{12402, 12403, 0, NULL, NULL}, +{12405, 12406, 0, NULL, NULL}, +{12408, 12409, 0, NULL, NULL}, +{12411, 12412, 0, NULL, NULL}, +{12445, 12446, 0, NULL, NULL}, +{12454, 12532, 0, NULL, NULL}, +{12459, 12460, 0, NULL, NULL}, +{12461, 12462, 0, NULL, NULL}, +{12463, 12464, 0, NULL, NULL}, +{12465, 12466, 0, NULL, NULL}, +{12467, 12468, 0, NULL, NULL}, +{12469, 12470, 0, NULL, NULL}, +{12471, 12472, 0, NULL, NULL}, +{12473, 12474, 0, NULL, NULL}, +{12475, 12476, 0, NULL, NULL}, +{12477, 12478, 0, NULL, NULL}, +{12479, 12480, 0, NULL, NULL}, +{12481, 12482, 0, NULL, NULL}, +{12484, 12485, 0, NULL, NULL}, +{12486, 12487, 0, NULL, NULL}, +{12488, 12489, 0, NULL, NULL}, +{12495, 12496, 0, NULL, NULL}, +{12498, 12499, 0, NULL, NULL}, +{12501, 12502, 0, NULL, NULL}, +{12504, 12505, 0, NULL, NULL}, +{12507, 12508, 0, NULL, NULL}, +{12527, 12535, 0, NULL, NULL}, +{12528, 12536, 0, NULL, NULL}, +{12529, 12537, 0, NULL, NULL}, +{12530, 12538, 0, NULL, NULL}, +{12541, 12542, 0, NULL, NULL} +}; /* compose_tab_59 */ +static int hash_compose_tab_60[20] = +{-1,7,1,-1,8,2,-1,9,3,-1,-1,4,-1,-1,-1,5,-1,-1,6,0}; /* hash_compose_tab_60 */ +static CompEntry compose_tab_60[] = { +{12399, 12401, 0, NULL, NULL}, +{12402, 12404, 0, NULL, NULL}, +{12405, 12407, 0, NULL, NULL}, +{12408, 12410, 0, NULL, NULL}, +{12411, 12413, 0, NULL, NULL}, +{12495, 12497, 0, NULL, NULL}, +{12498, 12500, 0, NULL, NULL}, +{12501, 12503, 0, NULL, NULL}, +{12504, 12506, 0, NULL, NULL}, +{12507, 12509, 0, NULL, NULL} +}; /* compose_tab_60 */ +static int hash_compose_tab[122] = +{30,31,52,60,32,-1,-1,33,-1,34,35,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1, + -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,0,1,2,3,4,-1,5,6,7,8,9,10,11,12,36,13,37,14, + 38,15,16,55,40,-1,-1,-1,-1,17,56,-1,-1,-1,-1,-1,41,18,19,20,42,21,22,-1,45, + 39,-1,23,24,-1,25,26,-1,-1,-1,-1,-1,-1,-1,-1,48,-1,43,44,51,53,-1,-1,27,46, + 54,28,-1,-1,47,-1,-1,-1,-1,49,50,-1,-1,57,-1,58,59,29}; /* hash_compose_tab */ +static CompEntry compose_tab[] = { +{768, 0, 39, compose_tab_0, hash_compose_tab_0}, +{769, 0, 70, compose_tab_1, hash_compose_tab_1}, +{770, 0, 27, compose_tab_2, hash_compose_tab_2}, +{771, 0, 19, compose_tab_3, hash_compose_tab_3}, +{772, 0, 28, compose_tab_4, hash_compose_tab_4}, +{774, 0, 30, compose_tab_5, hash_compose_tab_5}, +{775, 0, 40, compose_tab_6, hash_compose_tab_6}, +{776, 0, 50, compose_tab_7, hash_compose_tab_7}, +{777, 0, 15, compose_tab_8, hash_compose_tab_8}, +{778, 0, 6, compose_tab_9, hash_compose_tab_9}, +{779, 0, 6, compose_tab_10, hash_compose_tab_10}, +{780, 0, 34, compose_tab_11, hash_compose_tab_11}, +{781, 0, 17, compose_tab_12, hash_compose_tab_12}, +{783, 0, 14, compose_tab_13, hash_compose_tab_13}, +{785, 0, 12, compose_tab_14, hash_compose_tab_14}, +{787, 0, 15, compose_tab_15, hash_compose_tab_15}, +{788, 0, 17, compose_tab_16, hash_compose_tab_16}, +{795, 0, 4, compose_tab_17, hash_compose_tab_17}, +{803, 0, 39, compose_tab_18, hash_compose_tab_18}, +{804, 0, 2, compose_tab_19, hash_compose_tab_19}, +{805, 0, 2, compose_tab_20, hash_compose_tab_20}, +{807, 0, 20, compose_tab_21, hash_compose_tab_21}, +{808, 0, 10, compose_tab_22, hash_compose_tab_22}, +{813, 0, 12, compose_tab_23, hash_compose_tab_23}, +{814, 0, 2, compose_tab_24, hash_compose_tab_24}, +{816, 0, 6, compose_tab_25, hash_compose_tab_25}, +{817, 0, 17, compose_tab_26, hash_compose_tab_26}, +{834, 0, 12, compose_tab_27, hash_compose_tab_27}, +{837, 0, 6, compose_tab_28, hash_compose_tab_28}, +{1463, 0, 2, compose_tab_29, hash_compose_tab_29}, +{1464, 0, 1, compose_tab_30, hash_compose_tab_30}, +{1465, 0, 1, compose_tab_31, hash_compose_tab_31}, +{1468, 0, 22, compose_tab_32, hash_compose_tab_32}, +{1471, 0, 3, compose_tab_33, hash_compose_tab_33}, +{1473, 0, 2, compose_tab_34, hash_compose_tab_34}, +{1474, 0, 2, compose_tab_35, hash_compose_tab_35}, +{2364, 0, 11, compose_tab_36, hash_compose_tab_36}, +{2492, 0, 4, compose_tab_37, hash_compose_tab_37}, +{2494, 0, 1, compose_tab_38, hash_compose_tab_38}, +{2519, 0, 1, compose_tab_39, hash_compose_tab_39}, +{2620, 0, 5, compose_tab_40, hash_compose_tab_40}, +{2876, 0, 3, compose_tab_41, hash_compose_tab_41}, +{2878, 0, 1, compose_tab_42, hash_compose_tab_42}, +{2902, 0, 1, compose_tab_43, hash_compose_tab_43}, +{2903, 0, 1, compose_tab_44, hash_compose_tab_44}, +{3006, 0, 2, compose_tab_45, hash_compose_tab_45}, +{3031, 0, 2, compose_tab_46, hash_compose_tab_46}, +{3158, 0, 1, compose_tab_47, hash_compose_tab_47}, +{3266, 0, 1, compose_tab_48, hash_compose_tab_48}, +{3285, 0, 3, compose_tab_49, hash_compose_tab_49}, +{3286, 0, 1, compose_tab_50, hash_compose_tab_50}, +{3390, 0, 2, compose_tab_51, hash_compose_tab_51}, +{3415, 0, 1, compose_tab_52, hash_compose_tab_52}, +{3634, 0, 1, compose_tab_53, hash_compose_tab_53}, +{3762, 0, 1, compose_tab_54, hash_compose_tab_54}, +{3953, 0, 3, compose_tab_55, hash_compose_tab_55}, +{3968, 0, 2, compose_tab_56, hash_compose_tab_56}, +{4021, 0, 2, compose_tab_57, hash_compose_tab_57}, +{4023, 0, 10, compose_tab_58, hash_compose_tab_58}, +{12441, 0, 48, compose_tab_59, hash_compose_tab_59}, +{12442, 0, 10, compose_tab_60, hash_compose_tab_60} +}; /* compose_tab */ +#define COMP_CANDIDATE_MAP_OFFSET 24 +static Uint32 comp_candidate_map[] = { + 0x081ABFDFU, + 0x000361B8U, + 0x00000024U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x93800000U, + 0x00000006U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x10000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x50000000U, + 0x00800000U, + 0x00000000U, + 0x00000000U, + 0x10000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x50000000U, + 0x00C00000U, + 0x00000000U, + 0x00000000U, + 0x40000000U, + 0x00800000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00400000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00600004U, + 0x00000000U, + 0x00000000U, + 0x40000000U, + 0x00800000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00040000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00040000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00000000U, + 0x00020000U, + 0x00000001U, + 0x00A00000U +}; diff --git a/erts/emulator/beam/global.h b/erts/emulator/beam/global.h index 89c6625550..c948af14ae 100644 --- a/erts/emulator/beam/global.h +++ b/erts/emulator/beam/global.h @@ -544,7 +544,7 @@ ERTS_GLB_INLINE void erts_may_save_closed_port(Port *prt) if (prt->snapshot != erts_smp_atomic_read(&erts_ports_snapshot)) { /* Dead ports are added from the end of the snapshot buffer */ Eterm* tombstone = (Eterm*) erts_smp_atomic_addtest(&erts_dead_ports_ptr, - -(long)sizeof(Eterm)); + -(erts_aint_t)sizeof(Eterm)); ASSERT(tombstone+1 != NULL); ASSERT(prt->snapshot == (Uint32) erts_smp_atomic_read(&erts_ports_snapshot) - 1); *tombstone = prt->id; @@ -563,7 +563,7 @@ extern Uint display_items; /* no of items to display in traces etc */ extern Uint display_loads; /* print info about loaded modules */ extern int erts_backtrace_depth; -extern erts_smp_atomic_t erts_max_gen_gcs; +extern erts_smp_atomic32_t erts_max_gen_gcs; extern int erts_disable_tolerant_timeofday; @@ -1206,7 +1206,7 @@ ERTS_GLB_INLINE void erts_smp_port_unlock(Port *prt) { #ifdef ERTS_SMP - long refc; + erts_aint_t refc; erts_smp_mtx_unlock(prt->lock); refc = erts_smp_atomic_dectest(&prt->refc); ASSERT(refc >= 0); @@ -1425,29 +1425,29 @@ void erl_drv_thr_init(void); /* time.c */ -ERTS_GLB_INLINE long do_time_read_and_reset(void); +ERTS_GLB_INLINE erts_aint_t do_time_read_and_reset(void); #ifdef ERTS_TIMER_THREAD ERTS_GLB_INLINE int next_time(void); -ERTS_GLB_INLINE void bump_timer(long); +ERTS_GLB_INLINE void bump_timer(erts_aint_t); #else -int next_time(void); -void bump_timer(long); +erts_aint_t next_time(void); +void bump_timer(erts_aint_t); extern erts_smp_atomic_t do_time; /* set at clock interrupt */ -ERTS_GLB_INLINE void do_time_add(long); +ERTS_GLB_INLINE void do_time_add(erts_aint_t); #endif #if ERTS_GLB_INLINE_INCL_FUNC_DEF #ifdef ERTS_TIMER_THREAD -ERTS_GLB_INLINE long do_time_read_and_reset(void) { return 0; } -ERTS_GLB_INLINE int next_time(void) { return -1; } -ERTS_GLB_INLINE void bump_timer(long ignore) { } +ERTS_GLB_INLINE erts_aint_t do_time_read_and_reset(void) { return 0; } +ERTS_GLB_INLINE erts_aint_t next_time(void) { return -1; } +ERTS_GLB_INLINE void bump_timer(erts_aint_t ignore) { } #else -ERTS_GLB_INLINE long do_time_read_and_reset(void) +ERTS_GLB_INLINE erts_aint_t do_time_read_and_reset(void) { return erts_smp_atomic_xchg(&do_time, 0L); } -ERTS_GLB_INLINE void do_time_add(long elapsed) +ERTS_GLB_INLINE void do_time_add(erts_aint_t elapsed) { erts_smp_atomic_add(&do_time, elapsed); } @@ -1596,6 +1596,19 @@ Sint erts_binary_set_loop_limit(Sint limit); /* erl_unicode.c */ void erts_init_unicode(void); Sint erts_unicode_set_loop_limit(Sint limit); + +void erts_native_filename_put(Eterm ioterm, int encoding, byte *p) ; +Sint erts_native_filename_need(Eterm ioterm, int encoding); +void erts_copy_utf8_to_utf16_little(byte *target, byte *bytes, int num_chars); +int erts_analyze_utf8(byte *source, Uint size, + byte **err_pos, Uint *num_chars, int *left); +char *erts_convert_filename_to_native(Eterm name, ErtsAlcType_t alloc_type, int allow_empty); + +#define ERTS_UTF8_OK 0 +#define ERTS_UTF8_INCOMPLETE 1 +#define ERTS_UTF8_ERROR 2 +#define ERTS_UTF8_ANALYZE_MORE 3 + /* erl_trace.c */ void erts_init_trace(void); void erts_trace_check_exiting(Eterm exiting); diff --git a/erts/emulator/beam/io.c b/erts/emulator/beam/io.c index 9ed92bbe03..f6c6a01fb2 100644 --- a/erts/emulator/beam/io.c +++ b/erts/emulator/beam/io.c @@ -428,7 +428,7 @@ setup_port(Port* prt, Eterm pid, erts_driver_t *driver, old_name = prt->name; prt->name = new_name; #ifdef ERTS_SMP - erts_smp_atomic_set(&prt->run_queue, (long) runq); + erts_smp_atomic_set(&prt->run_queue, (erts_aint_t) runq); #endif ASSERT(!prt->drv_ptr); prt->drv_ptr = driver; @@ -1297,7 +1297,7 @@ void init_io(void) erts_port[i].port_data_lock = NULL; } - erts_smp_atomic_init(&erts_ports_snapshot, (long) 0); + erts_smp_atomic_init(&erts_ports_snapshot, (erts_aint_t) 0); last_port_num = 0; erts_smp_spinlock_init(&get_free_port_lck, "get_free_port"); @@ -3252,7 +3252,7 @@ int driver_output_binary(ErlDrvPort ix, char* hbuf, int hlen, return 0; prt->bytes_in += (hlen + len); - erts_smp_atomic_add(&erts_bytes_in, (long) (hlen + len)); + erts_smp_atomic_add(&erts_bytes_in, (erts_aint_t) (hlen + len)); if (prt->status & ERTS_PORT_SFLG_DISTRIBUTION) { return erts_net_message(prt, prt->dist_entry, @@ -3287,7 +3287,7 @@ int driver_output2(ErlDrvPort ix, char* hbuf, int hlen, char* buf, int len) return 0; prt->bytes_in += (hlen + len); - erts_smp_atomic_add(&erts_bytes_in, (long) (hlen + len)); + erts_smp_atomic_add(&erts_bytes_in, (erts_aint_t) (hlen + len)); if (prt->status & ERTS_PORT_SFLG_DISTRIBUTION) { if (len == 0) return erts_net_message(prt, @@ -3364,7 +3364,7 @@ int driver_outputv(ErlDrvPort ix, char* hbuf, int hlen, ErlIOVec* vec, int skip) /* XXX handle distribution !!! */ prt->bytes_in += (hlen + size); - erts_smp_atomic_add(&erts_bytes_in, (long) (hlen + size)); + erts_smp_atomic_add(&erts_bytes_in, (erts_aint_t) (hlen + size)); deliver_vec_message(prt, prt->connected, hbuf, hlen, binv, iov, n, size); return 0; } @@ -3408,25 +3408,25 @@ int len; * reference count on driver binaries... */ -long +ErlDrvSInt driver_binary_get_refc(ErlDrvBinary *dbp) { Binary* bp = ErlDrvBinary2Binary(dbp); - return erts_refc_read(&bp->refc, 1); + return (ErlDrvSInt) erts_refc_read(&bp->refc, 1); } -long +ErlDrvSInt driver_binary_inc_refc(ErlDrvBinary *dbp) { Binary* bp = ErlDrvBinary2Binary(dbp); - return erts_refc_inctest(&bp->refc, 2); + return (ErlDrvSInt) erts_refc_inctest(&bp->refc, 2); } -long +ErlDrvSInt driver_binary_dec_refc(ErlDrvBinary *dbp) { Binary* bp = ErlDrvBinary2Binary(dbp); - return erts_refc_dectest(&bp->refc, 1); + return (ErlDrvSInt) erts_refc_dectest(&bp->refc, 1); } @@ -3541,12 +3541,12 @@ pdl_init_refc(ErlDrvPDL pdl) erts_atomic_init(&pdl->refc, 1); } -static ERTS_INLINE long +static ERTS_INLINE ErlDrvSInt pdl_read_refc(ErlDrvPDL pdl) { - long refc = erts_atomic_read(&pdl->refc); + erts_aint_t refc = erts_atomic_read(&pdl->refc); ERTS_LC_ASSERT(refc >= 0); - return refc; + return (ErlDrvSInt) refc; } static ERTS_INLINE void @@ -3556,12 +3556,12 @@ pdl_inc_refc(ErlDrvPDL pdl) ERTS_LC_ASSERT(driver_pdl_get_refc(pdl) > 1); } -static ERTS_INLINE long +static ERTS_INLINE ErlDrvSInt pdl_inctest_refc(ErlDrvPDL pdl) { - long refc = erts_atomic_inctest(&pdl->refc); + erts_aint_t refc = erts_atomic_inctest(&pdl->refc); ERTS_LC_ASSERT(refc > 1); - return refc; + return (ErlDrvSInt) refc; } #if 0 /* unused */ @@ -3573,12 +3573,12 @@ pdl_dec_refc(ErlDrvPDL pdl) } #endif -static ERTS_INLINE long +static ERTS_INLINE ErlDrvSInt pdl_dectest_refc(ErlDrvPDL pdl) { - long refc = erts_atomic_dectest(&pdl->refc); + erts_aint_t refc = erts_atomic_dectest(&pdl->refc); ERTS_LC_ASSERT(refc >= 0); - return refc; + return (ErlDrvSInt) refc; } static ERTS_INLINE void pdl_destroy(ErlDrvPDL pdl) @@ -3649,7 +3649,7 @@ driver_pdl_lock(ErlDrvPDL pdl) void driver_pdl_unlock(ErlDrvPDL pdl) { - long refc; + ErlDrvSInt refc; #ifdef HARDDEBUG erts_fprintf(stderr, "driver_pdl_unlock(0x%08X)\r\n",(unsigned) pdl); #endif @@ -3659,28 +3659,30 @@ driver_pdl_unlock(ErlDrvPDL pdl) pdl_destroy(pdl); } -long +ErlDrvSInt driver_pdl_get_refc(ErlDrvPDL pdl) { return pdl_read_refc(pdl); } -long +ErlDrvSInt driver_pdl_inc_refc(ErlDrvPDL pdl) { - long refc = pdl_inctest_refc(pdl); + ErlDrvSInt refc = pdl_inctest_refc(pdl); #ifdef HARDDEBUG - erts_fprintf(stderr, "driver_pdl_inc_refc(0x%08X) -> %ld\r\n",(unsigned) pdl, refc); + erts_fprintf(stderr, "driver_pdl_inc_refc(%p) -> %bpd\r\n", + pdl, refc); #endif return refc; } -long +ErlDrvSInt driver_pdl_dec_refc(ErlDrvPDL pdl) { - long refc = pdl_dectest_refc(pdl); + ErlDrvSInt refc = pdl_dectest_refc(pdl); #ifdef HARDDEBUG - erts_fprintf(stderr, "driver_pdl_dec_refc(0x%08X) -> %ld\r\n",(unsigned) pdl, refc); + erts_fprintf(stderr, "driver_pdl_dec_refc(%p) -> %bpd\r\n", + pdl, refc); #endif if (!refc) pdl_destroy(pdl); diff --git a/erts/emulator/beam/sys.h b/erts/emulator/beam/sys.h index 0d15272aa8..5a4dad0a28 100644 --- a/erts/emulator/beam/sys.h +++ b/erts/emulator/beam/sys.h @@ -728,11 +728,11 @@ typedef enum { } erts_activity_error_t; typedef struct { - erts_smp_atomic_t do_block; + erts_smp_atomic32_t do_block; struct { - erts_smp_atomic_t wait; - erts_smp_atomic_t gc; - erts_smp_atomic_t io; + erts_smp_atomic32_t wait; + erts_smp_atomic32_t gc; + erts_smp_atomic32_t io; } in_activity; } erts_system_block_state_t; @@ -883,7 +883,7 @@ ERTS_GLB_INLINE int erts_smp_pending_system_block(void) { #ifdef ERTS_SMP - return erts_smp_atomic_read(&erts_system_block_state.do_block); + return (int) erts_smp_atomic32_read(&erts_system_block_state.do_block); #else return 0; #endif @@ -919,7 +919,7 @@ erts_smp_set_activity(erts_activity_t old_activity, case ERTS_ACTIVITY_UNDEFINED: break; case ERTS_ACTIVITY_WAIT: - erts_smp_atomic_dec(&erts_system_block_state.in_activity.wait); + erts_smp_atomic32_dec(&erts_system_block_state.in_activity.wait); if (locked) { /* You are not allowed to leave activity waiting * without supplying the possibility to block @@ -930,10 +930,10 @@ erts_smp_set_activity(erts_activity_t old_activity, } break; case ERTS_ACTIVITY_GC: - erts_smp_atomic_dec(&erts_system_block_state.in_activity.gc); + erts_smp_atomic32_dec(&erts_system_block_state.in_activity.gc); break; case ERTS_ACTIVITY_IO: - erts_smp_atomic_dec(&erts_system_block_state.in_activity.io); + erts_smp_atomic32_dec(&erts_system_block_state.in_activity.io); break; default: erts_set_activity_error(ERTS_ACT_ERR_LEAVE_UNKNOWN_ACTIVITY, @@ -949,13 +949,13 @@ erts_smp_set_activity(erts_activity_t old_activity, case ERTS_ACTIVITY_UNDEFINED: break; case ERTS_ACTIVITY_WAIT: - erts_smp_atomic_inc(&erts_system_block_state.in_activity.wait); + erts_smp_atomic32_inc(&erts_system_block_state.in_activity.wait); break; case ERTS_ACTIVITY_GC: - erts_smp_atomic_inc(&erts_system_block_state.in_activity.gc); + erts_smp_atomic32_inc(&erts_system_block_state.in_activity.gc); break; case ERTS_ACTIVITY_IO: - erts_smp_atomic_inc(&erts_system_block_state.in_activity.io); + erts_smp_atomic32_inc(&erts_system_block_state.in_activity.io); break; default: erts_set_activity_error(ERTS_ACT_ERR_ENTER_UNKNOWN_ACTIVITY, @@ -990,27 +990,31 @@ erts_smp_set_activity(erts_activity_t old_activity, typedef erts_smp_atomic_t erts_refc_t; -ERTS_GLB_INLINE void erts_refc_init(erts_refc_t *refcp, long val); -ERTS_GLB_INLINE void erts_refc_inc(erts_refc_t *refcp, long min_val); -ERTS_GLB_INLINE long erts_refc_inctest(erts_refc_t *refcp, long min_val); -ERTS_GLB_INLINE void erts_refc_dec(erts_refc_t *refcp, long min_val); -ERTS_GLB_INLINE long erts_refc_dectest(erts_refc_t *refcp, long min_val); -ERTS_GLB_INLINE void erts_refc_add(erts_refc_t *refcp, long diff, long min_val); -ERTS_GLB_INLINE long erts_refc_read(erts_refc_t *refcp, long min_val); +ERTS_GLB_INLINE void erts_refc_init(erts_refc_t *refcp, erts_aint_t val); +ERTS_GLB_INLINE void erts_refc_inc(erts_refc_t *refcp, erts_aint_t min_val); +ERTS_GLB_INLINE erts_aint_t erts_refc_inctest(erts_refc_t *refcp, + erts_aint_t min_val); +ERTS_GLB_INLINE void erts_refc_dec(erts_refc_t *refcp, erts_aint_t min_val); +ERTS_GLB_INLINE erts_aint_t erts_refc_dectest(erts_refc_t *refcp, + erts_aint_t min_val); +ERTS_GLB_INLINE void erts_refc_add(erts_refc_t *refcp, erts_aint_t diff, + erts_aint_t min_val); +ERTS_GLB_INLINE erts_aint_t erts_refc_read(erts_refc_t *refcp, + erts_aint_t min_val); #if ERTS_GLB_INLINE_INCL_FUNC_DEF ERTS_GLB_INLINE void -erts_refc_init(erts_refc_t *refcp, long val) +erts_refc_init(erts_refc_t *refcp, erts_aint_t val) { erts_smp_atomic_init((erts_smp_atomic_t *) refcp, val); } ERTS_GLB_INLINE void -erts_refc_inc(erts_refc_t *refcp, long min_val) +erts_refc_inc(erts_refc_t *refcp, erts_aint_t min_val) { #ifdef ERTS_REFC_DEBUG - long val = erts_smp_atomic_inctest((erts_smp_atomic_t *) refcp); + erts_aint_t val = erts_smp_atomic_inctest((erts_smp_atomic_t *) refcp); if (val < min_val) erl_exit(ERTS_ABORT_EXIT, "erts_refc_inc(): Bad refc found (refc=%ld < %ld)!\n", @@ -1020,10 +1024,10 @@ erts_refc_inc(erts_refc_t *refcp, long min_val) #endif } -ERTS_GLB_INLINE long -erts_refc_inctest(erts_refc_t *refcp, long min_val) +ERTS_GLB_INLINE erts_aint_t +erts_refc_inctest(erts_refc_t *refcp, erts_aint_t min_val) { - long val = erts_smp_atomic_inctest((erts_smp_atomic_t *) refcp); + erts_aint_t val = erts_smp_atomic_inctest((erts_smp_atomic_t *) refcp); #ifdef ERTS_REFC_DEBUG if (val < min_val) erl_exit(ERTS_ABORT_EXIT, @@ -1034,10 +1038,10 @@ erts_refc_inctest(erts_refc_t *refcp, long min_val) } ERTS_GLB_INLINE void -erts_refc_dec(erts_refc_t *refcp, long min_val) +erts_refc_dec(erts_refc_t *refcp, erts_aint_t min_val) { #ifdef ERTS_REFC_DEBUG - long val = erts_smp_atomic_dectest((erts_smp_atomic_t *) refcp); + erts_aint_t val = erts_smp_atomic_dectest((erts_smp_atomic_t *) refcp); if (val < min_val) erl_exit(ERTS_ABORT_EXIT, "erts_refc_dec(): Bad refc found (refc=%ld < %ld)!\n", @@ -1047,10 +1051,10 @@ erts_refc_dec(erts_refc_t *refcp, long min_val) #endif } -ERTS_GLB_INLINE long -erts_refc_dectest(erts_refc_t *refcp, long min_val) +ERTS_GLB_INLINE erts_aint_t +erts_refc_dectest(erts_refc_t *refcp, erts_aint_t min_val) { - long val = erts_smp_atomic_dectest((erts_smp_atomic_t *) refcp); + erts_aint_t val = erts_smp_atomic_dectest((erts_smp_atomic_t *) refcp); #ifdef ERTS_REFC_DEBUG if (val < min_val) erl_exit(ERTS_ABORT_EXIT, @@ -1061,10 +1065,10 @@ erts_refc_dectest(erts_refc_t *refcp, long min_val) } ERTS_GLB_INLINE void -erts_refc_add(erts_refc_t *refcp, long diff, long min_val) +erts_refc_add(erts_refc_t *refcp, erts_aint_t diff, erts_aint_t min_val) { #ifdef ERTS_REFC_DEBUG - long val = erts_smp_atomic_addtest((erts_smp_atomic_t *) refcp, diff); + erts_aint_t val = erts_smp_atomic_addtest((erts_smp_atomic_t *) refcp, diff); if (val < min_val) erl_exit(ERTS_ABORT_EXIT, "erts_refc_add(%ld): Bad refc found (refc=%ld < %ld)!\n", @@ -1074,10 +1078,10 @@ erts_refc_add(erts_refc_t *refcp, long diff, long min_val) #endif } -ERTS_GLB_INLINE long -erts_refc_read(erts_refc_t *refcp, long min_val) +ERTS_GLB_INLINE erts_aint_t +erts_refc_read(erts_refc_t *refcp, erts_aint_t min_val) { - long val = erts_smp_atomic_read((erts_smp_atomic_t *) refcp); + erts_aint_t val = erts_smp_atomic_read((erts_smp_atomic_t *) refcp); #ifdef ERTS_REFC_DEBUG if (val < min_val) erl_exit(ERTS_ABORT_EXIT, @@ -1253,6 +1257,22 @@ char* win32_errorstr(int); #endif +/************************************************************************ + * Find out the native filename encoding of the process (look at locale of + * Unix processes and just do UTF16 on windows + ************************************************************************/ +#define ERL_FILENAME_UNKNOWN 0 +#define ERL_FILENAME_LATIN1 1 +#define ERL_FILENAME_UTF8 2 +#define ERL_FILENAME_UTF8_MAC 3 +#define ERL_FILENAME_WIN_WCHAR 4 + +int erts_get_native_filename_encoding(void); +/* The set function is only to be used by erl_init! */ +void erts_set_user_requested_filename_encoding(int encoding); +int erts_get_user_requested_filename_encoding(void); + +void erts_init_sys_common_misc(void); #endif diff --git a/erts/emulator/beam/time.c b/erts/emulator/beam/time.c index 53d39aef0e..9cb6ea34ef 100644 --- a/erts/emulator/beam/time.c +++ b/erts/emulator/beam/time.c @@ -107,12 +107,12 @@ static int itime; /* Constant after init */ #if defined(ERTS_TIMER_THREAD) static SysTimeval time_start; /* start of current time interval */ -static long ticks_end; /* time_start+ticks_end == time_wakeup */ -static long ticks_latest; /* delta from time_start at latest time update*/ +static erts_aint_t ticks_end; /* time_start+ticks_end == time_wakeup */ +static erts_aint_t ticks_latest; /* delta from time_start at latest time update*/ -static ERTS_INLINE long time_gettimeofday(SysTimeval *now) +static ERTS_INLINE erts_aint_t time_gettimeofday(SysTimeval *now) { - long elapsed; + erts_aint_t elapsed; erts_get_timeval(now); now->tv_usec = 1000 * (now->tv_usec / 1000); /* ms resolution */ @@ -122,25 +122,25 @@ static ERTS_INLINE long time_gettimeofday(SysTimeval *now) return elapsed; } -static long do_time_update(void) +static erts_aint_t do_time_update(void) { SysTimeval now; - long elapsed; + erts_aint_t elapsed; elapsed = time_gettimeofday(&now); ticks_latest = elapsed; return elapsed; } -static ERTS_INLINE long do_time_read(void) +static ERTS_INLINE erts_aint_t do_time_read(void) { return ticks_latest; } -static long do_time_reset(void) +static erts_aint_t do_time_reset(void) { SysTimeval now; - long elapsed; + erts_aint_t elapsed; elapsed = time_gettimeofday(&now); time_start = now; @@ -156,20 +156,29 @@ static ERTS_INLINE void do_time_init(void) #else erts_smp_atomic_t do_time; /* set at clock interrupt */ -static ERTS_INLINE long do_time_read(void) { return erts_smp_atomic_read(&do_time); } -static ERTS_INLINE long do_time_update(void) { return do_time_read(); } -static ERTS_INLINE void do_time_init(void) { erts_smp_atomic_init(&do_time, 0L); } +static ERTS_INLINE erts_aint_t do_time_read(void) +{ + return erts_smp_atomic_read(&do_time); +} +static ERTS_INLINE erts_aint_t do_time_update(void) +{ + return do_time_read(); +} +static ERTS_INLINE void do_time_init(void) +{ + erts_smp_atomic_init(&do_time, (erts_aint_t) 0); +} #endif /* get the time (in units of itime) to the next timeout, or -1 if there are no timeouts */ -static int next_time_internal(void) /* PRE: tiw_lock taken by caller */ +static erts_aint_t next_time_internal(void) /* PRE: tiw_lock taken by caller */ { int i, tm, nto; unsigned int min; ErlTimer* p; - long dt; + erts_aint_t dt; if (tiw_nto == 0) return -1; /* no timeouts in wheel */ @@ -204,9 +213,9 @@ static int next_time_internal(void) /* PRE: tiw_lock taken by caller */ #if !defined(ERTS_TIMER_THREAD) /* Private export to erl_time_sup.c */ -int next_time(void) +erts_aint_t next_time(void) { - int ret; + erts_aint_t ret; erts_smp_mtx_lock(&tiw_lock); (void)do_time_update(); @@ -216,12 +225,12 @@ int next_time(void) } #endif -static ERTS_INLINE void bump_timer_internal(long dt) /* PRE: tiw_lock is write-locked */ +static ERTS_INLINE void bump_timer_internal(erts_aint_t dt) /* PRE: tiw_lock is write-locked */ { Uint keep_pos; Uint count; ErlTimer *p, **prev, *timeout_head, **timeout_tail; - Uint dtime = (unsigned long)dt; + Uint dtime = (Uint) dt; /* no need to bump the position if there aren't any timeouts */ if (tiw_nto == 0) { @@ -287,7 +296,7 @@ static void timer_thread_bump_timer(void) bump_timer_internal(do_time_reset()); } #else -void bump_timer(long dt) /* dt is value from do_time */ +void bump_timer(erts_aint_t dt) /* dt is value from do_time */ { erts_smp_mtx_lock(&tiw_lock); bump_timer_internal(dt); @@ -305,8 +314,8 @@ static struct erts_iwait *timer_thread_iwait; static int timer_thread_setup_delay(SysTimeval *rem_time) { - long elapsed; - int ticks; + erts_aint_t elapsed; + erts_aint_t ticks; erts_smp_mtx_lock(&tiw_lock); elapsed = do_time_update(); @@ -496,7 +505,7 @@ Uint time_left(ErlTimer *p) { Uint left; - long dt; + erts_aint_t dt; erts_smp_mtx_lock(&tiw_lock); @@ -517,7 +526,7 @@ time_left(ErlTimer *p) erts_smp_mtx_unlock(&tiw_lock); - return left * itime; + return (Uint) left * itime; } #ifdef DEBUG diff --git a/erts/emulator/beam/utils.c b/erts/emulator/beam/utils.c index ab5e8b5d4a..2bf283d9ec 100644 --- a/erts/emulator/beam/utils.c +++ b/erts/emulator/beam/utils.c @@ -3637,19 +3637,19 @@ erts_set_activity_error(erts_activity_error_t error, char *file, int line) } -static ERTS_INLINE int +static ERTS_INLINE erts_aint32_t threads_not_under_control(void) { - int res = system_block_state.threads_to_block; + erts_aint32_t res = system_block_state.threads_to_block; /* Waiting is always an allowed activity... */ - res -= erts_smp_atomic_read(&erts_system_block_state.in_activity.wait); + res -= erts_smp_atomic32_read(&erts_system_block_state.in_activity.wait); if (system_block_state.allowed_activities & ERTS_BS_FLG_ALLOW_GC) - res -= erts_smp_atomic_read(&erts_system_block_state.in_activity.gc); + res -= erts_smp_atomic32_read(&erts_system_block_state.in_activity.gc); if (system_block_state.allowed_activities & ERTS_BS_FLG_ALLOW_IO) - res -= erts_smp_atomic_read(&erts_system_block_state.in_activity.io); + res -= erts_smp_atomic32_read(&erts_system_block_state.in_activity.io); if (res < 0) { ASSERT(0); @@ -3709,7 +3709,7 @@ erts_block_system(Uint32 allowed_activities) } else { - erts_smp_atomic_inc(&erts_system_block_state.do_block); + erts_smp_atomic32_inc(&erts_system_block_state.do_block); /* Someone else might be waiting for us to block... */ if (do_block) { @@ -3761,11 +3761,11 @@ erts_emergency_block_system(long timeout, Uint32 allowed_activities) another_blocker = erts_smp_pending_system_block(); system_block_state.emergency = 1; - erts_smp_atomic_inc(&erts_system_block_state.do_block); + erts_smp_atomic32_inc(&erts_system_block_state.do_block); if (another_blocker) { if (is_blocker()) { - erts_smp_atomic_dec(&erts_system_block_state.do_block); + erts_smp_atomic32_dec(&erts_system_block_state.do_block); res = 0; goto done; } @@ -3822,7 +3822,7 @@ erts_release_system(void) if (system_block_state.recursive_block) system_block_state.recursive_block--; else { - do_block = erts_smp_atomic_dectest(&erts_system_block_state.do_block); + do_block = erts_smp_atomic32_dectest(&erts_system_block_state.do_block); system_block_state.have_blocker = 0; if (is_blockable_thread()) system_block_state.threads_to_block++; @@ -3957,10 +3957,10 @@ erts_system_block_init(void) /* Global state... */ - erts_smp_atomic_init(&erts_system_block_state.do_block, 0L); - erts_smp_atomic_init(&erts_system_block_state.in_activity.wait, 0L); - erts_smp_atomic_init(&erts_system_block_state.in_activity.gc, 0L); - erts_smp_atomic_init(&erts_system_block_state.in_activity.io, 0L); + erts_smp_atomic32_init(&erts_system_block_state.do_block, 0); + erts_smp_atomic32_init(&erts_system_block_state.in_activity.wait, 0); + erts_smp_atomic32_init(&erts_system_block_state.in_activity.gc, 0); + erts_smp_atomic32_init(&erts_system_block_state.in_activity.io, 0); /* Make sure blockable threads unregister when exiting... */ erts_smp_install_exit_handler(erts_unregister_blockable_thread); diff --git a/erts/emulator/drivers/common/efile_drv.c b/erts/emulator/drivers/common/efile_drv.c index c450f10f48..786fa7da77 100644 --- a/erts/emulator/drivers/common/efile_drv.c +++ b/erts/emulator/drivers/common/efile_drv.c @@ -67,6 +67,8 @@ #define FILE_RESP_LDATA 6 #define FILE_RESP_N2DATA 7 #define FILE_RESP_EOF 8 +#define FILE_RESP_FNAME 9 +#define FILE_RESP_ALL_DATA 10 /* Options */ @@ -109,11 +111,11 @@ void erl_exit(int n, char *fmt, ...); static ErlDrvSysInfo sys_info; -/*#define TRACE 1*/ +/* #define TRACE 1 */ #ifdef TRACE -# define TRACE_C(c) (putchar(c)) -# define TRACE_S(s) (fputs((s), stdout)) -# define TRACE_F(args) (printf args) +# define TRACE_C(c) do { putchar(c); fflush(stdout); } while (0) +# define TRACE_S(s) do { fputs((s), stdout); fflush(stdout); } while (0) +# define TRACE_F(args) do { printf args ;fflush(stdout); } while (0) #else # define TRACE_C(c) ((void)(0)) # define TRACE_S(s) ((void)(0)) @@ -137,24 +139,54 @@ static ErlDrvSysInfo sys_info; #define MUTEX_UNLOCK(m) #endif - - #if 0 /* Experimental, for forcing all file operations to use the same thread. */ -static unsigned file_fixed_key = 1; -#define KEY(desc) (&file_fixed_key) + static unsigned file_fixed_key = 1; +# define KEY(desc) (&file_fixed_key) #else -#define KEY(desc) (&(desc)->key) +# define KEY(desc) (&(desc)->key) #endif +#ifdef FILENAMES_16BIT +# define FILENAME_BYTELEN(Str) filename_len_16bit(Str) +# define FILENAME_COPY(To,From) filename_cpy_16bit((To),(From)) +# define FILENAME_CHARSIZE 2 + + static int filename_len_16bit(char *str) + { + char *p = str; + while(*p != '\0' || p[1] != '\0') { + p += 2; + } + return (p - str); + } + + static void filename_cpy_16bit(char *to, char *from) + { + while(*from != '\0' || from[1] != '\0') { + *to++ = *from++; + *to++ = *from++; + } + *to++ = *from++; + *to++ = *from++; + } + +#else +# define FILENAME_BYTELEN(Str) strlen(Str) +# define FILENAME_COPY(To,From) strcpy(To,From) +# define FILENAME_CHARSIZE 1 +#endif -#if MAXPATHLEN >= BUFSIZ -#define RESBUFSIZE MAXPATHLEN+1 +#if (MAXPATHLEN+1)*FILENAME_CHARSIZE+1 > BUFSIZ +# define RESBUFSIZE ((MAXPATHLEN+1)*FILENAME_CHARSIZE+1) #else -#define RESBUFSIZE BUFSIZ +# define RESBUFSIZE BUFSIZ #endif + + + #define GET_TIME(i, b) \ (i).year = get_int32((b) + 0 * 4); \ (i).month = get_int32((b) + 1 * 4); \ @@ -286,9 +318,9 @@ struct t_preadv { }; #define READDIR_BUFSIZE (8*1024) -#if READDIR_BUFSIZE < (2*MAXPATHLEN) -#undef READDIR_BUFSIZE -#define READDIR_BUFSIZE (2*MAXPATHLEN) +#if READDIR_BUFSIZE < (FILENAME_CHARSIZE*2*(MAXPATHLEN+1)) +# undef READDIR_BUFSIZE +# define READDIR_BUFSIZE (FILENAME_CHARSIZE*2*(MAXPATHLEN+1)) #endif struct t_readdir_buf { @@ -369,6 +401,7 @@ struct t_data }; + #define EF_ALLOC(S) driver_alloc((S)) #define EF_REALLOC(P, S) driver_realloc((P), (S)) #define EF_SAFE_ALLOC(S) ef_safe_alloc((S)) @@ -1288,7 +1321,7 @@ static void invoke_writev(void *data) { p < size && iovcnt < iovlen; p += iov0[iovcnt++].iov_len) ; - iov = EF_ALLOC(sizeof(SysIOVec)*iovcnt); + iov = EF_SAFE_ALLOC(sizeof(SysIOVec)*iovcnt); memcpy(iov,iov0,iovcnt*sizeof(SysIOVec)); MUTEX_UNLOCK(d->c.writev.q_mtx); /* Let go of lock until we deque from original vector */ @@ -1368,7 +1401,7 @@ static void invoke_readlink(void *data) d->result_ok = efile_readlink(&d->errInfo, d->b, resbuf+1, RESBUFSIZE-1); if (d->result_ok != 0) - strcpy((char *) d->b + 1, resbuf+1); + FILENAME_COPY((char *) d->b + 1, resbuf+1); } static void invoke_altname(void *data) @@ -1380,7 +1413,7 @@ static void invoke_altname(void *data) d->result_ok = efile_altname(&d->errInfo, d->b, resbuf+1, RESBUFSIZE-1); if (d->result_ok != 0) - strcpy((char *) d->b + 1, resbuf+1); + FILENAME_COPY((char *) d->b + 1, resbuf+1); } static void invoke_pwritev(void *data) { @@ -1405,7 +1438,7 @@ static void invoke_pwritev(void *data) { /* Lock the queue just for a while, we don't want it locked during write */ MUTEX_LOCK(c->q_mtx); iov0 = driver_peekq(c->port, &iovlen); - iov = EF_ALLOC(sizeof(SysIOVec)*iovlen); + iov = EF_SAFE_ALLOC(sizeof(SysIOVec)*iovlen); memcpy(iov,iov0,sizeof(SysIOVec)*iovlen); MUTEX_UNLOCK(c->q_mtx); @@ -1499,7 +1532,7 @@ static void invoke_link(void *data) char *new_name; d->again = 0; - new_name = name+strlen(name)+1; + new_name = name+FILENAME_BYTELEN(name)+FILENAME_CHARSIZE; d->result_ok = efile_link(&d->errInfo, name, new_name); } @@ -1510,7 +1543,7 @@ static void invoke_symlink(void *data) char *new_name; d->again = 0; - new_name = name+strlen(name)+1; + new_name = name+FILENAME_BYTELEN(name)+FILENAME_CHARSIZE; d->result_ok = efile_symlink(&d->errInfo, name, new_name); } @@ -1521,7 +1554,7 @@ static void invoke_rename(void *data) char *new_name; d->again = 0; - new_name = name+strlen(name)+1; + new_name = name+FILENAME_BYTELEN(name)+FILENAME_CHARSIZE; d->result_ok = efile_rename(&d->errInfo, name, new_name); } @@ -1569,13 +1602,15 @@ static void invoke_readdir(void *data) int s; char *p = NULL; int buf_sz = 0; + size_t tmp_bs; d->again = 0; d->errInfo.posix_errno = 0; while (1) { char *str; - if (buf_sz < (4 /* sz */ + 1 /* cmd */ + MAXPATHLEN + 1 /* '\0' */)) { + if (buf_sz < (4 /* sz */ + 1 /* cmd */ + + FILENAME_CHARSIZE*(MAXPATHLEN + 1))) { struct t_readdir_buf *b; if (p) { put_int32(0, p); /* EOB */ @@ -1591,18 +1626,18 @@ static void invoke_readdir(void *data) buf_sz = READDIR_BUFSIZE - 4/* EOB */; } - p[4] = FILE_RESP_OK; + p[4] = FILE_RESP_FNAME; buf_sz -= 4 + 1; str = p + 4 + 1; ASSERT(buf_sz >= MAXPATHLEN + 1); - s = efile_readdir(&d->errInfo, d->b, &d->dir_handle, str, buf_sz); + tmp_bs = buf_sz; + s = efile_readdir(&d->errInfo, d->b, &d->dir_handle, str, &tmp_bs); if (s) { - int str_sz = strlen(str); - int sz = str_sz + 1; - put_int32(sz, p); - p += 4 + sz; - buf_sz -= str_sz; + put_int32(tmp_bs + 1 /* 1 byte for opcode */, p); + p += 4 + tmp_bs + 1; + ASSERT(p == (str + tmp_bs)); + buf_sz -= tmp_bs; } else { put_int32(1, p); @@ -1911,7 +1946,7 @@ file_async_ready(ErlDrvData e, ErlDrvThreadData data) if (!d->result_ok) reply_error(desc, &d->errInfo); else { - header[0] = FILE_RESP_OK; + header[0] = FILE_RESP_ALL_DATA; TRACE_C('R'); driver_output_binary(desc->port, header, 1, d->c.read_file.binp, @@ -1968,10 +2003,10 @@ file_async_ready(ErlDrvData e, ErlDrvThreadData data) if (!d->result_ok) reply_error(desc, &d->errInfo); else { - resbuf[0] = FILE_RESP_OK; - length = 1+strlen((char*) resbuf+1); + resbuf[0] = FILE_RESP_FNAME; + length = 1+FILENAME_BYTELEN((char*) resbuf+1); TRACE_C('R'); - driver_output2(desc->port, resbuf, length, NULL, 0); + driver_output2(desc->port, resbuf, 1, resbuf+1, length-1); } free_data(data); break; @@ -2031,13 +2066,18 @@ file_async_ready(ErlDrvData e, ErlDrvThreadData data) int sz = get_int32(p); while (sz) { /* 0 == EOB */ p += 4; - driver_output2(desc->port, p, sz, NULL, 0); + if (sz - 1 > 0) { + driver_output2(desc->port, p, 1, p+1, sz-1); + } else { + driver_output2(desc->port, p, 1, NULL, 0); + } p += sz; sz = get_int32(p); } b1 = b1->next; EF_FREE(b2); } + d->c.read_dir.first_buf = NULL; d->c.read_dir.last_buf = NULL; } @@ -2113,9 +2153,9 @@ file_output(ErlDrvData e, char* buf, int count) case FILE_MKDIR: { - d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + strlen(name) + 1); + d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + FILENAME_BYTELEN(name) + FILENAME_CHARSIZE); - strcpy(d->b, name); + FILENAME_COPY(d->b, name); d->command = command; d->invoke = invoke_mkdir; d->free = free_data; @@ -2124,9 +2164,9 @@ file_output(ErlDrvData e, char* buf, int count) } case FILE_RMDIR: { - d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + strlen(name) + 1); + d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + FILENAME_BYTELEN(name) + FILENAME_CHARSIZE); - strcpy(d->b, name); + FILENAME_COPY(d->b, name); d->command = command; d->invoke = invoke_rmdir; d->free = free_data; @@ -2135,9 +2175,9 @@ file_output(ErlDrvData e, char* buf, int count) } case FILE_DELETE: { - d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + strlen(name) + 1); + d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + FILENAME_BYTELEN(name) + FILENAME_CHARSIZE); - strcpy(d->b, name); + FILENAME_COPY(d->b, name); d->command = command; d->invoke = invoke_delete_file; d->free = free_data; @@ -2147,14 +2187,14 @@ file_output(ErlDrvData e, char* buf, int count) case FILE_RENAME: { char* new_name; - - new_name = name+strlen(name)+1; + int namelen = FILENAME_BYTELEN(name)+FILENAME_CHARSIZE; + new_name = name+namelen; d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 - + strlen(name) + 1 - + strlen(new_name) + 1); + + namelen + + FILENAME_BYTELEN(new_name) + FILENAME_CHARSIZE); - strcpy(d->b, name); - strcpy(d->b + strlen(name) + 1, new_name); + FILENAME_COPY(d->b, name); + FILENAME_COPY(d->b + namelen, new_name); d->flags = desc->flags; d->fd = fd; d->command = command; @@ -2165,9 +2205,9 @@ file_output(ErlDrvData e, char* buf, int count) } case FILE_CHDIR: { - d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + strlen(name) + 1); + d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + FILENAME_BYTELEN(name) + FILENAME_CHARSIZE); - strcpy(d->b, name); + FILENAME_COPY(d->b, name); d->command = command; d->invoke = invoke_chdir; d->free = free_data; @@ -2190,9 +2230,10 @@ file_output(ErlDrvData e, char* buf, int count) #ifdef USE_THREADS if (sys_info.async_threads > 0) { - d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + strlen(name) + 1); + d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + FILENAME_BYTELEN(name) + + FILENAME_CHARSIZE); - strcpy(d->b, name); + FILENAME_COPY(d->b, name); d->dir_handle = NULL; d->command = command; d->invoke = invoke_readdir; @@ -2205,17 +2246,19 @@ file_output(ErlDrvData e, char* buf, int count) else #endif { + size_t resbufsize; char resbuf[RESBUFSIZE+1]; EFILE_DIR_HANDLE dir_handle; /* Handle to open directory. */ errInfo.posix_errno = 0; dir_handle = NULL; - resbuf[0] = FILE_RESP_OK; + resbuf[0] = FILE_RESP_FNAME; + resbufsize = RESBUFSIZE; while (efile_readdir(&errInfo, name, &dir_handle, - resbuf+1, RESBUFSIZE)) { - int length = 1 + strlen(resbuf+1); - driver_output2(desc->port, resbuf, length, NULL, 0); + resbuf+1, &resbufsize)) { + driver_output2(desc->port, resbuf, 1, resbuf+1, resbufsize); + resbufsize = RESBUFSIZE; } if (errInfo.posix_errno != 0) { reply_error(desc, &errInfo); @@ -2227,11 +2270,12 @@ file_output(ErlDrvData e, char* buf, int count) } case FILE_OPEN: { - d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + strlen(buf+4) + 1); + d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + FILENAME_BYTELEN(buf+4) + + FILENAME_CHARSIZE); d->flags = get_int32((uchar*)buf); name = buf+4; - strcpy(d->b, name); + FILENAME_COPY(d->b, name); d->command = command; d->invoke = invoke_open; d->free = free_data; @@ -2240,44 +2284,45 @@ file_output(ErlDrvData e, char* buf, int count) } case FILE_FDATASYNC: - { + { d = EF_SAFE_ALLOC(sizeof(struct t_data)); - + d->fd = fd; d->command = command; d->invoke = invoke_fdatasync; d->free = free_data; d->level = 2; goto done; - } + } case FILE_FSYNC: - { - d = EF_SAFE_ALLOC(sizeof(struct t_data)); - - d->fd = fd; - d->command = command; - d->invoke = invoke_fsync; - d->free = free_data; - d->level = 2; - goto done; - } + { + d = EF_SAFE_ALLOC(sizeof(struct t_data)); + + d->fd = fd; + d->command = command; + d->invoke = invoke_fsync; + d->free = free_data; + d->level = 2; + goto done; + } case FILE_FSTAT: case FILE_LSTAT: - { - d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + strlen(name) + 1); + { + d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + FILENAME_BYTELEN(name) + + FILENAME_CHARSIZE); + + FILENAME_COPY(d->b, name); + d->fd = fd; + d->command = command; + d->invoke = invoke_flstat; + d->free = free_data; + d->level = 2; + goto done; + } - strcpy(d->b, name); - d->fd = fd; - d->command = command; - d->invoke = invoke_flstat; - d->free = free_data; - d->level = 2; - goto done; - } - case FILE_TRUNCATE: { d = EF_SAFE_ALLOC(sizeof(struct t_data)); @@ -2294,7 +2339,7 @@ file_output(ErlDrvData e, char* buf, int count) case FILE_WRITE_INFO: { d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 - + strlen(buf+21*4) + 1); + + FILENAME_BYTELEN(buf+21*4) + FILENAME_CHARSIZE); d->info.mode = get_int32(buf + 0 * 4); d->info.uid = get_int32(buf + 1 * 4); @@ -2302,7 +2347,7 @@ file_output(ErlDrvData e, char* buf, int count) GET_TIME(d->info.accessTime, buf + 3 * 4); GET_TIME(d->info.modifyTime, buf + 9 * 4); GET_TIME(d->info.cTime, buf + 15 * 4); - strcpy(d->b, buf+21*4); + FILENAME_COPY(d->b, buf+21*4); d->command = command; d->invoke = invoke_write_info; d->free = free_data; @@ -2314,7 +2359,7 @@ file_output(ErlDrvData e, char* buf, int count) { d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + RESBUFSIZE + 1); - strcpy(d->b, name); + FILENAME_COPY(d->b, name); d->command = command; d->invoke = invoke_readlink; d->free = free_data; @@ -2323,28 +2368,29 @@ file_output(ErlDrvData e, char* buf, int count) } case FILE_ALTNAME: - { - d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + RESBUFSIZE + 1); - strcpy(d->b, name); - d->command = command; - d->invoke = invoke_altname; - d->free = free_data; - d->level = 2; - goto done; - } + { + d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 + RESBUFSIZE + 1); + FILENAME_COPY(d->b, name); + d->command = command; + d->invoke = invoke_altname; + d->free = free_data; + d->level = 2; + goto done; + } case FILE_LINK: { char* new_name; + int namelen = FILENAME_BYTELEN(name) + FILENAME_CHARSIZE; - new_name = name+strlen(name)+1; + new_name = name+namelen; d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 - + strlen(name) + 1 - + strlen(new_name) + 1); + + namelen + + FILENAME_BYTELEN(new_name) + FILENAME_CHARSIZE); - strcpy(d->b, name); - strcpy(d->b + strlen(name) + 1, new_name); + FILENAME_COPY(d->b, name); + FILENAME_COPY(d->b + namelen, new_name); d->flags = desc->flags; d->fd = fd; d->command = command; @@ -2357,14 +2403,15 @@ file_output(ErlDrvData e, char* buf, int count) case FILE_SYMLINK: { char* new_name; + int namelen = FILENAME_BYTELEN(name) + FILENAME_CHARSIZE; - new_name = name+strlen(name)+1; + new_name = name+namelen; d = EF_SAFE_ALLOC(sizeof(struct t_data) - 1 - + strlen(name) + 1 - + strlen(new_name) + 1); + + namelen + + FILENAME_BYTELEN(new_name) + FILENAME_CHARSIZE); - strcpy(d->b, name); - strcpy(d->b + strlen(name) + 1, new_name); + FILENAME_COPY(d->b, name); + FILENAME_COPY(d->b + namelen, new_name); d->flags = desc->flags; d->fd = fd; d->command = command; @@ -3004,6 +3051,7 @@ file_outputv(ErlDrvData e, ErlIOVec *ev) { case FILE_READ_FILE: { struct t_data *d; + char *filename; if (ev->size < 1+1) { /* Buffer contains empty name */ reply_posix_error(desc, ENOENT); @@ -3014,7 +3062,8 @@ file_outputv(ErlDrvData e, ErlIOVec *ev) { reply_posix_error(desc, EINVAL); goto done; } - d = EF_ALLOC(sizeof(struct t_data) + ev->size); + filename = EV_CHAR_P(ev, p, q); + d = EF_ALLOC(sizeof(struct t_data) -1 + FILENAME_BYTELEN(filename) + FILENAME_CHARSIZE); if (! d) { reply_posix_error(desc, ENOMEM); goto done; @@ -3022,8 +3071,7 @@ file_outputv(ErlDrvData e, ErlIOVec *ev) { d->command = command; d->reply = !0; /* Copy name */ - memcpy(d->c.read_file.name, EV_CHAR_P(ev, p, q), ev->size-1); - d->c.read_file.name[ev->size-1] = '\0'; + FILENAME_COPY(d->c.read_file.name, filename); d->c.read_file.binp = NULL; d->invoke = invoke_read_file; d->free = free_read_file; diff --git a/erts/emulator/drivers/common/erl_efile.h b/erts/emulator/drivers/common/erl_efile.h index ac95c1f949..3097ded3f1 100644 --- a/erts/emulator/drivers/common/erl_efile.h +++ b/erts/emulator/drivers/common/erl_efile.h @@ -59,6 +59,14 @@ #define FA_WRITE 1 #define FA_READ 2 +/* Some OS'es (i.e. Windows) has filenames in wide charaqcters. That requires special handling */ +/* Note that we do *not* honor alignment in the communication to the OS specific driver, */ +/* which is not a problem on x86, but might be on other platforms. The OS specific efile */ +/* implementation is expected to align if needed */ +#ifdef __WIN32__ +#define FILENAMES_16BIT 1 +#endif + /* * An handle to an open directory. To be cast to the correct type * in the system-dependent directory functions. @@ -123,7 +131,7 @@ int efile_getdcwd(Efile_error* errInfo, int drive, char* buffer, size_t size); int efile_readdir(Efile_error* errInfo, char* name, EFILE_DIR_HANDLE* dir_handle, - char* buffer, size_t size); + char* buffer, size_t *size); int efile_openfile(Efile_error* errInfo, char* name, int flags, int* pfd, Sint64* pSize); void efile_closefile(int fd); diff --git a/erts/emulator/drivers/common/gzio.c b/erts/emulator/drivers/common/gzio.c index 801bc61d4d..5531a275ea 100644 --- a/erts/emulator/drivers/common/gzio.c +++ b/erts/emulator/drivers/common/gzio.c @@ -28,6 +28,7 @@ #ifdef __WIN32__ #define HAVE_CONFLICTING_FREAD_DECLARATION +#define FILENAMES_16BIT 1 #endif #ifdef STDC @@ -102,6 +103,40 @@ local uLong getLong OF((gz_stream *s)); # define ERTS_GZREAD(File, Buf, Count) fread((Buf), 1, (Count), (File)) #endif +/* + * Ripped from efile_drv.c + */ + +#ifdef FILENAMES_16BIT +# define FILENAME_BYTELEN(Str) filename_len_16bit(Str) +# define FILENAME_COPY(To,From) filename_cpy_16bit((To),(From)) +# define FILENAME_CHARSIZE 2 + + static int filename_len_16bit(const char *str) + { + const char *p = str; + while(*p != '\0' || p[1] != '\0') { + p += 2; + } + return (p - str); + } + + static void filename_cpy_16bit(char *to, const char *from) + { + while(*from != '\0' || from[1] != '\0') { + *to++ = *from++; + *to++ = *from++; + } + *to++ = *from++; + *to++ = *from++; + } + +#else +# define FILENAME_BYTELEN(Str) strlen(Str) +# define FILENAME_COPY(To,From) strcpy(To,From) +# define FILENAME_CHARSIZE 1 +#endif + /* =========================================================================== Opens a gzip (.gz) file for reading or writing. The mode parameter is as in fopen ("rb" or "wb"). The file is given either by file descriptor @@ -144,11 +179,11 @@ local gzFile gz_open (path, mode) s->position = 0; s->destroy = destroy; - s->path = (char*)ALLOC(strlen(path)+1); + s->path = (char*)ALLOC(FILENAME_BYTELEN(path)+FILENAME_CHARSIZE); if (s->path == NULL) { return s->destroy(s), (gzFile)Z_NULL; } - strcpy(s->path, path); /* do this early for debugging */ + FILENAME_COPY(s->path, path); /* do this early for debugging */ s->mode = '\0'; do { @@ -194,7 +229,22 @@ local gzFile gz_open (path, mode) s->stream.avail_out = Z_BUFSIZE; errno = 0; -#ifdef UNIX +#if defined(FILENAMES_16BIT) + { + char wfmode[160]; + int i=0,j; + for(j=0;fmode[j] != '\0';++j) { + wfmode[i++]=fmode[j]; + wfmode[i++]='\0'; + } + wfmode[i++] = '\0'; + wfmode[i++] = '\0'; + s->file = F_OPEN(path, wfmode); + if (s->file == NULL) { + return s->destroy(s), (gzFile)Z_NULL; + } + } +#elif defined(UNIX) if (s->mode == 'r') { s->file = open(path, O_RDONLY); } else { diff --git a/erts/emulator/drivers/common/inet_drv.c b/erts/emulator/drivers/common/inet_drv.c index 18f7cdd15a..1382d1dfe4 100644 --- a/erts/emulator/drivers/common/inet_drv.c +++ b/erts/emulator/drivers/common/inet_drv.c @@ -88,9 +88,21 @@ #include <winsock2.h> #endif #include <windows.h> +#include <Ws2tcpip.h> /* NEED VC 6.0 or higher */ + +/* Visual studio 2008+: NTDDI_VERSION needs to be set for iphlpapi.h + to define the right structures. It needs to be set to WINXP (or LONGHORN) + for IPV6 to work and it's set lower by default, so we need to change it. */ +#ifdef HAVE_SDKDDKVER_H +# include <sdkddkver.h> +# ifdef NTDDI_VERSION +# undef NTDDI_VERSION +# endif +# define NTDDI_VERSION NTDDI_WINXP +#endif + #include <iphlpapi.h> -#include <Ws2tcpip.h> /* NEED VC 6.0 !!! */ #undef WANT_NONBLOCKING #include "sys.h" @@ -5811,9 +5823,12 @@ static int sctp_set_opts(inet_descriptor* desc, char* ptr, int len) char *after; # ifdef HAVE_STRUCT_SCTP_PADDRPARAMS_SPP_FLAGS int eflags, cflags, hb_enable, hb_disable, - pmtud_enable, pmtud_disable, + pmtud_enable, pmtud_disable; +# ifdef HAVE_STRUCT_SCTP_PADDRPARAMS_SPP_SACKDELAY + int sackdelay_enable, sackdelay_disable; # endif +# endif CHKLEN(curr, ASSOC_ID_LEN); arg.pap.spp_assoc_id = GET_ASSOC_ID(curr); curr += ASSOC_ID_LEN; diff --git a/erts/emulator/drivers/unix/unix_efile.c b/erts/emulator/drivers/unix/unix_efile.c index b19f632f52..6297ccb8bc 100644 --- a/erts/emulator/drivers/unix/unix_efile.c +++ b/erts/emulator/drivers/unix/unix_efile.c @@ -587,7 +587,8 @@ efile_readdir(Efile_error* errInfo, /* Where to return error codes. */ open directory.*/ char* buffer, /* Pointer to buffer for one filename. */ - size_t size) /* Size of buffer. */ + size_t *size) /* in-out Size of buffer, length + of name. */ { DIR *dp; /* Pointer to directory structure. */ struct dirent* dirp; /* Pointer to directory entry. */ @@ -620,6 +621,7 @@ efile_readdir(Efile_error* errInfo, /* Where to return error codes. */ continue; buffer[0] = '\0'; strncat(buffer, dirp->d_name, size-1); + *size = strlen(dirp->d_name); return 1; } } diff --git a/erts/emulator/drivers/win32/win_efile.c b/erts/emulator/drivers/win32/win_efile.c index 6de08e2fa6..4ec9579529 100755 --- a/erts/emulator/drivers/win32/win_efile.c +++ b/erts/emulator/drivers/win32/win_efile.c @@ -23,20 +23,20 @@ #include <windows.h> #include "sys.h" #include <ctype.h> - +#include <wchar.h> #include "erl_efile.h" /* * Microsoft-specific function to map a WIN32 error code to a Posix errno. */ -#define ISSLASH(a) ((a) == '\\' || (a) == '/') +#define ISSLASH(a) ((a) == L'\\' || (a) == L'/') #define ISDIR(st) (((st).st_mode&S_IFMT) == S_IFDIR) #define ISREG(st) (((st).st_mode&S_IFMT) == S_IFREG) #define IS_DOT_OR_DOTDOT(s) \ - (s[0] == '.' && (s[1] == '\0' || (s[1] == '.' && s[2] == '\0'))) + ((s)[0] == L'.' && ((s)[1] == L'\0' || ((s)[1] == L'.' && (s)[2] == L'\0'))) #ifndef INVALID_FILE_ATTRIBUTES #define INVALID_FILE_ATTRIBUTES ((DWORD) 0xFFFFFFFF) @@ -44,9 +44,9 @@ static int check_error(int result, Efile_error* errInfo); static int set_error(Efile_error* errInfo); -static int IsRootUNCName(const char* path); -static int extract_root(char* name); -static unsigned short dos_to_posix_mode(int attr, const char *name); +static int is_root_unc_name(const WCHAR *path); +static int extract_root(WCHAR *name); +static unsigned short dos_to_posix_mode(int attr, const WCHAR *name); static int errno_map(DWORD last_error) { @@ -196,27 +196,26 @@ win_writev(Efile_error* errInfo, int -efile_mkdir(errInfo, name) -Efile_error* errInfo; /* Where to return error codes. */ -char* name; /* Name of directory to create. */ +efile_mkdir(Efile_error* errInfo, /* Where to return error codes. */ + char* name) /* Name of directory to create. */ { - return check_error(mkdir(name), errInfo); + return check_error(_wmkdir((WCHAR *) name), errInfo); } int -efile_rmdir(errInfo, name) -Efile_error* errInfo; /* Where to return error codes. */ -char* name; /* Name of directory to delete. */ +efile_rmdir(Efile_error* errInfo, /* Where to return error codes. */ + char* name) /* Name of directory to delete. */ { OSVERSIONINFO os; DWORD attr; + WCHAR *wname = (WCHAR *) name; - if (RemoveDirectory(name) != FALSE) { + if (RemoveDirectoryW(wname) != FALSE) { return 1; } errno = errno_map(GetLastError()); if (errno == EACCES) { - attr = GetFileAttributes(name); + attr = GetFileAttributesW(wname); if (attr != (DWORD) -1) { if ((attr & FILE_ATTRIBUTE_DIRECTORY) == 0) { /* @@ -238,21 +237,21 @@ char* name; /* Name of directory to delete. */ GetVersionEx(&os); if (os.dwPlatformId == VER_PLATFORM_WIN32_WINDOWS) { HANDLE handle; - WIN32_FIND_DATA data; - char buffer[2*MAX_PATH]; + WIN32_FIND_DATAW data; + WCHAR buffer[2*MAX_PATH]; int len; - len = strlen(name); - strcpy(buffer, name); - if (buffer[0] && buffer[len-1] != '\\' && buffer[len-1] != '/') { - strcat(buffer, "\\"); + len = wcslen(wname); + wcscpy(buffer, wname); + if (buffer[0] && buffer[len-1] != L'\\' && buffer[len-1] != L'/') { + wcscat(buffer, L"\\"); } - strcat(buffer, "*.*"); - handle = FindFirstFile(buffer, &data); + wcscat(buffer, L"*.*"); + handle = FindFirstFileW(buffer, &data); if (handle != INVALID_HANDLE_VALUE) { while (1) { - if ((strcmp(data.cFileName, ".") != 0) - && (strcmp(data.cFileName, "..") != 0)) { + if ((wcscmp(data.cFileName, L".") != 0) + && (wcscmp(data.cFileName, L"..") != 0)) { /* * Found something in this directory. */ @@ -260,7 +259,7 @@ char* name; /* Name of directory to delete. */ errno = EEXIST; break; } - if (FindNextFile(handle, &data) == FALSE) { + if (FindNextFileW(handle, &data) == FALSE) { break; } } @@ -284,19 +283,19 @@ char* name; /* Name of directory to delete. */ } int -efile_delete_file(errInfo, name) -Efile_error* errInfo; /* Where to return error codes. */ -char* name; /* Name of file to delete. */ +efile_delete_file(Efile_error* errInfo, /* Where to return error codes. */ + char* name) /* Name of file to delete. */ { DWORD attr; + WCHAR *wname = (WCHAR *) name; - if (DeleteFile(name) != FALSE) { + if (DeleteFileW(wname) != FALSE) { return 1; } errno = errno_map(GetLastError()); if (errno == EACCES) { - attr = GetFileAttributes(name); + attr = GetFileAttributesW(wname); if (attr != (DWORD) -1) { if (attr & FILE_ATTRIBUTE_DIRECTORY) { /* @@ -308,7 +307,7 @@ char* name; /* Name of file to delete. */ } } } else if (errno == ENOENT) { - attr = GetFileAttributes(name); + attr = GetFileAttributesW(wname); if (attr != (DWORD) -1) { if (attr & FILE_ATTRIBUTE_DIRECTORY) { /* @@ -362,20 +361,21 @@ char* name; /* Name of file to delete. */ */ int -efile_rename(errInfo, src, dst) -Efile_error* errInfo; /* Where to return error codes. */ -char* src; /* Original name. */ -char* dst; /* New name. */ +efile_rename(Efile_error* errInfo, /* Where to return error codes. */ + char* src, /* Original name. */ + char* dst) /* New name. */ { DWORD srcAttr, dstAttr; + WCHAR *wsrc = (WCHAR *) src; + WCHAR *wdst = (WCHAR *) dst; - if (MoveFile(src, dst) != FALSE) { + if (MoveFileW(wsrc, wdst) != FALSE) { return 1; } errno = errno_map(GetLastError()); - srcAttr = GetFileAttributes(src); - dstAttr = GetFileAttributes(dst); + srcAttr = GetFileAttributesW(wsrc); + dstAttr = GetFileAttributesW(wdst); if (srcAttr == (DWORD) -1) { srcAttr = 0; } @@ -390,22 +390,22 @@ char* dst; /* New name. */ if (errno == EACCES) { decode: if (srcAttr & FILE_ATTRIBUTE_DIRECTORY) { - char srcPath[MAX_PATH], dstPath[MAX_PATH]; - char *srcRest, *dstRest; + WCHAR srcPath[MAX_PATH], dstPath[MAX_PATH]; + WCHAR *srcRest, *dstRest; int size; - size = GetFullPathName(src, sizeof(srcPath), srcPath, &srcRest); - if ((size == 0) || (size > sizeof(srcPath))) { + size = GetFullPathNameW(wsrc, MAX_PATH, srcPath, &srcRest); + if ((size == 0) || (size > MAX_PATH)) { return check_error(-1, errInfo); } - size = GetFullPathName(dst, sizeof(dstPath), dstPath, &dstRest); - if ((size == 0) || (size > sizeof(dstPath))) { + size = GetFullPathNameW(wdst, MAX_PATH, dstPath, &dstRest); + if ((size == 0) || (size > MAX_PATH)) { return check_error(-1, errInfo); } if (srcRest == NULL) { - srcRest = srcPath + strlen(srcPath); + srcRest = srcPath + wcslen(srcPath); } - if (strnicmp(srcPath, dstPath, srcRest - srcPath) == 0) { + if (_wcsnicmp(srcPath, dstPath, srcRest - srcPath) == 0) { /* * Trying to move a directory into itself. */ @@ -420,14 +420,14 @@ char* dst; /* New name. */ } (void) extract_root(dstPath); - if (dstPath[0] == '\0') { + if (dstPath[0] == L'\0') { /* * The filename was invalid. (Don't know why, * but play it safe.) */ errno = EINVAL; } - if (stricmp(srcPath, dstPath) != 0) { + if (_wcsicmp(srcPath, dstPath) != 0) { /* * If src is a directory and dst filesystem != src * filesystem, errno should be EXDEV. It is very @@ -463,14 +463,14 @@ char* dst; /* New name. */ * fails, it's because it wasn't empty. */ - if (RemoveDirectory(dst)) { + if (RemoveDirectoryW(wdst)) { /* * Now that that empty directory is gone, we can try * renaming again. If that fails, we'll put this empty * directory back, for completeness. */ - if (MoveFile(src, dst) != FALSE) { + if (MoveFileW(wsrc, wdst) != FALSE) { return 1; } @@ -480,8 +480,8 @@ char* dst; /* New name. */ */ errno = errno_map(GetLastError()); - CreateDirectory(dst, NULL); - SetFileAttributes(dst, dstAttr); + CreateDirectoryW(wdst, NULL); + SetFileAttributesW(wdst, dstAttr); if (errno == EACCES) { /* * Decode the EACCES to a more meaningful error. @@ -506,17 +506,17 @@ char* dst; /* New name. */ * put temp file back to old name. */ - char tempName[MAX_PATH]; + WCHAR tempName[MAX_PATH]; int result, size; - char *rest; + WCHAR *rest; - size = GetFullPathName(dst, sizeof(tempName), tempName, &rest); - if ((size == 0) || (size > sizeof(tempName)) || (rest == NULL)) { + size = GetFullPathNameW(wdst, MAX_PATH, tempName, &rest); + if ((size == 0) || (size > MAX_PATH) || (rest == NULL)) { return check_error(-1, errInfo); } - *rest = '\0'; + *rest = L'\0'; result = -1; - if (GetTempFileName(tempName, "erlr", 0, tempName) != 0) { + if (GetTempFileNameW(tempName, L"erlr", 0, tempName) != 0) { /* * Strictly speaking, need the following DeleteFile and * MoveFile to be joined as an atomic operation so no @@ -524,15 +524,15 @@ char* dst; /* New name. */ * same temp file. */ - DeleteFile(tempName); - if (MoveFile(dst, tempName) != FALSE) { - if (MoveFile(src, dst) != FALSE) { - SetFileAttributes(tempName, FILE_ATTRIBUTE_NORMAL); - DeleteFile(tempName); + DeleteFileW(tempName); + if (MoveFileW(wdst, tempName) != FALSE) { + if (MoveFileW(wsrc, wdst) != FALSE) { + SetFileAttributesW(tempName, FILE_ATTRIBUTE_NORMAL); + DeleteFileW(tempName); return 1; } else { - DeleteFile(dst); - MoveFile(tempName, dst); + DeleteFileW(wdst); + MoveFileW(tempName, wdst); } } @@ -558,11 +558,10 @@ char* dst; /* New name. */ } int -efile_chdir(errInfo, name) -Efile_error* errInfo; /* Where to return error codes. */ -char* name; /* Name of directory to make current. */ +efile_chdir(Efile_error* errInfo, /* Where to return error codes. */ + char* name) /* Name of directory to make current. */ { - int success = check_error(chdir(name), errInfo); + int success = check_error(_wchdir((WCHAR *) name), errInfo); if (!success && errInfo->posix_errno == EINVAL) /* POSIXification of errno */ errInfo->posix_errno = ENOENT; @@ -570,59 +569,65 @@ char* name; /* Name of directory to make current. */ } int -efile_getdcwd(errInfo, drive, buffer, size) -Efile_error* errInfo; /* Where to return error codes. */ -int drive; /* 0 - current, 1 - A, 2 - B etc. */ -char* buffer; /* Where to return the current directory. */ -size_t size; /* Size of buffer. */ +efile_getdcwd(Efile_error* errInfo, /* Where to return error codes. */ + int drive, /* 0 - current, 1 - A, 2 - B etc. */ + char* buffer, /* Where to return the current directory. */ + size_t size) /* Size of buffer. */ { - if (_getdcwd(drive, buffer, size) == NULL) + WCHAR *wbuffer = (WCHAR *) buffer; + size_t wbuffer_size = size / 2; + if (_wgetdcwd(drive, wbuffer, wbuffer_size) == NULL) return check_error(-1, errInfo); - for ( ; *buffer; buffer++) - if (*buffer == '\\') - *buffer = '/'; + for ( ; *wbuffer; wbuffer++) + if (*wbuffer == L'\\') + *wbuffer = L'/'; return 1; } int -efile_readdir(errInfo, name, dir_handle, buffer, size) -Efile_error* errInfo; /* Where to return error codes. */ -char* name; /* Name of directory to open. */ -EFILE_DIR_HANDLE* dir_handle; /* Directory handle of open directory. */ -char* buffer; /* Pointer to buffer for one filename. */ -size_t size; /* Size of buffer. */ +efile_readdir(Efile_error* errInfo, /* Where to return error codes. */ + char* name, /* Name of directory to list */ + EFILE_DIR_HANDLE* dir_handle, /* Handle of opened directory or NULL */ + char* buffer, /* Buffer to put one filename in */ + size_t *size) /* in-out size of buffer/size of filename excluding zero + termination in bytes*/ { HANDLE dir; /* Handle to directory. */ - char wildcard[MAX_PATH]; /* Wildcard to search for. */ - WIN32_FIND_DATA findData; /* Data found by FindFirstFile() or FindNext(). */ + WCHAR wildcard[MAX_PATH]; /* Wildcard to search for. */ + WIN32_FIND_DATAW findData; /* Data found by FindFirstFile() or FindNext(). */ + /* Alignment is not honored, this works on x86 because of alignment fixup by processor. + Not perfect, but faster than alinging by hand (really) */ + WCHAR *wname = (WCHAR *) name; + WCHAR *wbuffer = (WCHAR *) buffer; /* * First time we must setup everything. */ if (*dir_handle == NULL) { - int length = strlen(name); - char* s; + int length = wcslen(wname); + WCHAR* s; if (length+3 >= MAX_PATH) { errno = ENAMETOOLONG; return check_error(-1, errInfo); } - strcpy(wildcard, name); + wcscpy(wildcard, wname); s = wildcard+length-1; - if (*s != '/' && *s != '\\') - *++s = '\\'; - *++s = '*'; - *++s = '\0'; - DEBUGF(("Reading %s\n", wildcard)); - dir = FindFirstFile(wildcard, &findData); + if (*s != L'/' && *s != L'\\') + *++s = L'\\'; + *++s = L'*'; + *++s = L'\0'; + DEBUGF(("Reading %ws\n", wildcard)); + dir = FindFirstFileW(wildcard, &findData); if (dir == INVALID_HANDLE_VALUE) return set_error(errInfo); *dir_handle = (EFILE_DIR_HANDLE) dir; if (!IS_DOT_OR_DOTDOT(findData.cFileName)) { - strcpy(buffer, findData.cFileName); + wcscpy(wbuffer, findData.cFileName); + *size = wcslen(wbuffer)*2; return 1; } } @@ -635,10 +640,11 @@ size_t size; /* Size of buffer. */ dir = (HANDLE) *dir_handle; for (;;) { - if (FindNextFile(dir, &findData)) { + if (FindNextFileW(dir, &findData)) { if (IS_DOT_OR_DOTDOT(findData.cFileName)) continue; - strcpy(buffer, findData.cFileName); + wcscpy(wbuffer, findData.cFileName); + *size = wcslen(wbuffer)*2; return 1; } @@ -655,17 +661,17 @@ size_t size; /* Size of buffer. */ } int -efile_openfile(errInfo, name, flags, pfd, pSize) -Efile_error* errInfo; /* Where to return error codes. */ -char* name; /* Name of directory to open. */ -int flags; /* Flags to use for opening. */ -int* pfd; /* Where to store the file descriptor. */ -Sint64* pSize; /* Where to store the size of the file. */ +efile_openfile(Efile_error* errInfo, /* Where to return error codes. */ + char* name, /* Name of directory to open. */ + int flags, /* Flags to use for opening. */ + int* pfd, /* Where to store the file descriptor. */ + Sint64* pSize) /* Where to store the size of the file. */ { BY_HANDLE_FILE_INFORMATION fileInfo; /* File information from a handle. */ HANDLE fd; /* Handle to open file. */ DWORD access; /* Access mode: GENERIC_READ, GENERIC_WRITE. */ DWORD crFlags; + WCHAR *wname = (WCHAR *) name; switch (flags & (EFILE_MODE_READ|EFILE_MODE_WRITE)) { case EFILE_MODE_READ: @@ -692,7 +698,7 @@ Sint64* pSize; /* Where to store the size of the file. */ if (flags & EFILE_MODE_EXCL) { crFlags = CREATE_NEW; } - fd = CreateFile(name, access, + fd = CreateFileW(wname, access, FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, crFlags, FILE_ATTRIBUTE_NORMAL, NULL); @@ -711,7 +717,7 @@ Sint64* pSize; /* Where to store the size of the file. */ * to EISDIR. */ if (errInfo->posix_errno && - (attr = GetFileAttributes(name)) != INVALID_FILE_ATTRIBUTES && + (attr = GetFileAttributesW(wname)) != INVALID_FILE_ATTRIBUTES && (attr & FILE_ATTRIBUTE_DIRECTORY)) { errInfo->posix_errno = EISDIR; } @@ -735,9 +741,10 @@ Sint64* pSize; /* Where to store the size of the file. */ int efile_may_openfile(Efile_error* errInfo, char *name) { + WCHAR *wname = (WCHAR *) name; DWORD attr; - if ((attr = GetFileAttributes(name)) == INVALID_FILE_ATTRIBUTES) { + if ((attr = GetFileAttributesW(wname)) == INVALID_FILE_ATTRIBUTES) { return check_error(-1, errInfo); } @@ -746,18 +753,6 @@ efile_may_openfile(Efile_error* errInfo, char *name) { return check_error(-1, errInfo); } return 1; -#if 0 - struct stat statbuf; - - if (stat(name, &statbuf)) { - return check_error(-1, errInfo); - } - if (ISDIR(statbuf)) { - errno = EISDIR; - return check_error(-1, errInfo); - } - return 1; -#endif } void @@ -792,16 +787,17 @@ efile_fileinfo(Efile_error* errInfo, Efile_info* pInfo, char* orig_name, int info_for_link) { HANDLE findhandle; /* Handle returned by FindFirstFile(). */ - WIN32_FIND_DATA findbuf; /* Data return by FindFirstFile(). */ - char name[_MAX_PATH]; + WIN32_FIND_DATAW findbuf; /* Data return by FindFirstFile(). */ + WCHAR name[_MAX_PATH]; int name_len; - char* path; - char pathbuf[_MAX_PATH]; + WCHAR *path; + WCHAR pathbuf[_MAX_PATH]; int drive; /* Drive for filename (1 = A:, 2 = B: etc). */ + WCHAR *worig_name = (WCHAR *) orig_name; /* Don't allow wildcards to be interpreted by system */ - if (strpbrk(orig_name, "?*")) { + if (wcspbrk(worig_name, L"?*")) { enoent: errInfo->posix_errno = ENOENT; errInfo->os_errno = ERROR_FILE_NOT_FOUND; @@ -813,25 +809,25 @@ efile_fileinfo(Efile_error* errInfo, Efile_info* pInfo, * slash, because it causes FindFirstFile() to fail on Win95. */ - if ((name_len = strlen(orig_name)) >= _MAX_PATH) { + if ((name_len = wcslen(worig_name)) >= _MAX_PATH) { goto enoent; } else { - strcpy(name, orig_name); + wcscpy(name, worig_name); if (name_len > 2 && ISSLASH(name[name_len-1]) && - name[name_len-2] != ':') { - name[name_len-1] = '\0'; + name[name_len-2] != L':') { + name[name_len-1] = L'\0'; } } /* Try to get disk from name. If none, get current disk. */ - if (name[1] != ':') { + if (name[1] != L':') { drive = 0; - if (GetCurrentDirectory(sizeof(pathbuf), pathbuf) && - pathbuf[1] == ':') { - drive = tolower(pathbuf[0]) - 'a' + 1; + if (GetCurrentDirectoryW(_MAX_PATH, pathbuf) && + pathbuf[1] == L':') { + drive = towlower(pathbuf[0]) - L'a' + 1; } - } else if (*name && name[2] == '\0') { + } else if (*name && name[2] == L'\0') { /* * X: and nothing more is an error. */ @@ -839,15 +835,15 @@ efile_fileinfo(Efile_error* errInfo, Efile_info* pInfo, errInfo->os_errno = ERROR_FILE_NOT_FOUND; return 0; } else - drive = tolower(*name) - 'a' + 1; + drive = towlower(*name) - L'a' + 1; - findhandle = FindFirstFile(name, &findbuf); + findhandle = FindFirstFileW(name, &findbuf); if (findhandle == INVALID_HANDLE_VALUE) { - if (!(strpbrk(name, "./\\") && - (path = _fullpath(pathbuf, name, _MAX_PATH)) && + if (!(wcspbrk(name, L"./\\") && + (path = _wfullpath(pathbuf, name, _MAX_PATH)) && /* root dir. ('C:\') or UNC root dir. ('\\server\share\') */ - ((strlen(path) == 3) || IsRootUNCName(path)) && - (GetDriveType(path) > 1) ) ) { + ((wcslen(path) == 3) || is_root_unc_name(path)) && + (GetDriveTypeW(path) > 1) ) ) { errInfo->posix_errno = ENOENT; errInfo->os_errno = ERROR_FILE_NOT_FOUND; return 0; @@ -860,7 +856,7 @@ efile_fileinfo(Efile_error* errInfo, Efile_info* pInfo, findbuf.dwFileAttributes = FILE_ATTRIBUTE_DIRECTORY; findbuf.nFileSizeHigh = 0; findbuf.nFileSizeLow = 0; - findbuf.cFileName[0] = '\0'; + findbuf.cFileName[0] = L'\0'; pInfo->links = 1; pInfo->modifyTime.year = 1980; @@ -881,26 +877,28 @@ efile_fileinfo(Efile_error* errInfo, Efile_info* pInfo, /* * given that we know this is a symlink, we should be able to find its target */ - char target_name[256]; - if (efile_readlink(errInfo, name, target_name,256) == 1) { + WCHAR target_name[_MAX_PATH]; + if (efile_readlink(errInfo, (char *) name, + (char *) target_name,256) == 1) { + FindClose(findhandle); return efile_fileinfo(errInfo, pInfo, - target_name, info_for_link); + (char *) target_name, info_for_link); } } -#if 0 /* number of links: */ { HANDLE handle; /* Handle returned by CreateFile() */ BY_HANDLE_FILE_INFORMATION fileInfo; /* from CreateFile() */ - if (handle = CreateFile(name, GENERIC_READ, 0,NULL, + if (handle = CreateFileW(name, GENERIC_READ, 0,NULL, OPEN_EXISTING, 0, NULL)) { GetFileInformationByHandle(handle, &fileInfo); pInfo->links = fileInfo.nNumberOfLinks; CloseHandle(handle); - } + } else { + pInfo->links = 1; + } } -#endif #define GET_TIME(dst, src) \ if (!FileTimeToLocalFileTime(&findbuf.src, &LocalFTime) || \ @@ -960,10 +958,9 @@ if (!FileTimeToLocalFileTime(&findbuf.src, &LocalFTime) || \ } int -efile_write_info(errInfo, pInfo, name) -Efile_error* errInfo; -Efile_info* pInfo; -char* name; +efile_write_info(Efile_error* errInfo, + Efile_info* pInfo, + char* name) { SYSTEMTIME timebuf; FILETIME LocalFileTime; @@ -977,12 +974,13 @@ char* name; DWORD attr; DWORD tempAttr; BOOL modifyTime = FALSE; + WCHAR *wname = (WCHAR *) name; /* * Get the attributes for the file. */ - tempAttr = attr = GetFileAttributes((LPTSTR)name); + tempAttr = attr = GetFileAttributesW(wname); if (attr == 0xffffffff) { return set_error(errInfo); } @@ -1036,12 +1034,12 @@ char* name; if (tempAttr & FILE_ATTRIBUTE_READONLY) { tempAttr &= ~FILE_ATTRIBUTE_READONLY; - if (!SetFileAttributes((LPTSTR) name, tempAttr)) { + if (!SetFileAttributesW(wname, tempAttr)) { return set_error(errInfo); } } - fd = CreateFile(name, GENERIC_READ|GENERIC_WRITE, + fd = CreateFileW(wname, GENERIC_READ|GENERIC_WRITE, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL); if (fd != INVALID_HANDLE_VALUE) { @@ -1059,7 +1057,7 @@ char* name; */ if (tempAttr != attr) { - if (!SetFileAttributes((LPTSTR) name, attr)) { + if (!SetFileAttributesW(wname, attr)) { return set_error(errInfo); } } @@ -1235,7 +1233,7 @@ int flags; /* - * IsRootUNCName - returns TRUE if the argument is a UNC name specifying + * is_root_unc_name - returns TRUE if the argument is a UNC name specifying * a root share. That is, if it is of the form \\server\share\. * This routine will also return true if the argument is of the * form \\server\share (no trailing slash) but Win32 currently @@ -1245,16 +1243,16 @@ int flags; */ static int -IsRootUNCName(const char* path) +is_root_unc_name(const WCHAR *path) { /* * If a root UNC name, path will start with 2 (but not 3) slashes */ - if ((strlen(path) >= 5) /* minimum string is "//x/y" */ + if ((wcslen(path) >= 5) /* minimum string is "//x/y" */ && ISSLASH(path[0]) && ISSLASH(path[1])) { - const char * p = path + 2 ; + const WCHAR *p = path + 2; /* * find the slash between the server name and share name @@ -1297,19 +1295,19 @@ IsRootUNCName(const char* path) */ static int -extract_root(char* name) +extract_root(WCHAR* name) { - int len = strlen(name); + int len = wcslen(name); - if (isalpha(name[0]) && name[1] == ':' && ISSLASH(name[2])) { - int c = name[3]; - name[3] = '\0'; - return c == '\0'; + if (iswalpha(name[0]) && name[1] == L':' && ISSLASH(name[2])) { + WCHAR c = name[3]; + name[3] = L'\0'; + return c == L'\0'; } else if (len < 5 || !ISSLASH(name[0]) || !ISSLASH(name[1])) { goto error; } else { /* Try to find the end of the UNC name. */ - char* p; - int c; + WCHAR* p; + WCHAR c; /* * Find the slash between the server name and share name. @@ -1318,7 +1316,7 @@ extract_root(char* name) for (p = name + 2; *p; p++) if (ISSLASH(*p)) break; - if (*p == '\0') + if (*p == L'\0') goto error; /* @@ -1329,24 +1327,24 @@ extract_root(char* name) if (ISSLASH(*p)) break; c = *p; - *p = '\0'; - return c == '\0' || p[1] == '\0'; + *p = L'\0'; + return c == L'\0' || p[1] == L'\0'; } error: - *name = '\0'; + *name = L'\0'; return 1; } static unsigned short -dos_to_posix_mode(int attr, const char *name) +dos_to_posix_mode(int attr, const WCHAR *name) { register unsigned short uxmode; unsigned dosmode; - register const char *p; + register const WCHAR *p; dosmode = attr & 0xff; - if ((p = name)[1] == ':') + if ((p = name)[1] == L':') p += 2; /* check to see if this is a directory - note we must make a special @@ -1355,7 +1353,7 @@ dos_to_posix_mode(int attr, const char *name) uxmode = (unsigned short) (((ISSLASH(*p) && !p[1]) || (dosmode & FILE_ATTRIBUTE_DIRECTORY) || - *p == '\0') ? _S_IFDIR|_S_IEXEC : _S_IFREG); + *p == L'\0') ? _S_IFDIR|_S_IEXEC : _S_IFREG); /* If attribute byte does not have read-only bit, it is read-write */ @@ -1364,11 +1362,11 @@ dos_to_posix_mode(int attr, const char *name) /* see if file appears to be executable - check extension of name */ - if (p = strrchr(name, '.')) { - if (!stricmp(p, ".exe") || - !stricmp(p, ".cmd") || - !stricmp(p, ".bat") || - !stricmp(p, ".com")) + if (p = wcsrchr(name, L'.')) { + if (!_wcsicmp(p, L".exe") || + !_wcsicmp(p, L".cmd") || + !_wcsicmp(p, L".bat") || + !_wcsicmp(p, L".com")) uxmode |= _S_IEXEC; } @@ -1388,28 +1386,40 @@ efile_readlink(Efile_error* errInfo, char* name, char* buffer, size_t size) * (Vista only) */ HINSTANCE hModule = NULL; + WCHAR *wname = (WCHAR *) name; + WCHAR *wbuffer = (WCHAR *) buffer; if ((hModule = LoadLibrary("kernel32.dll")) != NULL) { typedef DWORD (WINAPI * GETFINALPATHNAMEBYHANDLEPTR)( HANDLE hFile, - LPCSTR lpFilePath, + LPCWSTR lpFilePath, DWORD cchFilePath, DWORD dwFlags); GETFINALPATHNAMEBYHANDLEPTR pGetFinalPathNameByHandle = - (GETFINALPATHNAMEBYHANDLEPTR)GetProcAddress(hModule, "GetFinalPathNameByHandleA"); + (GETFINALPATHNAMEBYHANDLEPTR)GetProcAddress(hModule, "GetFinalPathNameByHandleW"); if (pGetFinalPathNameByHandle == NULL) { FreeLibrary(hModule); } else { /* first check if file is a symlink; {error, einval} otherwise */ - DWORD fileAttributes = GetFileAttributes(name); + DWORD fileAttributes = GetFileAttributesW(wname); if ((fileAttributes & FILE_ATTRIBUTE_REPARSE_POINT)) { BOOLEAN success = 0; - HANDLE h = CreateFile(name, GENERIC_READ, 0,NULL, OPEN_EXISTING, 0, NULL); + HANDLE h = CreateFileW(wname, GENERIC_READ, 0,NULL, OPEN_EXISTING, 0, NULL); + int len; if(h != INVALID_HANDLE_VALUE) { - success = pGetFinalPathNameByHandle(h, buffer, size,0); + success = pGetFinalPathNameByHandle(h, wbuffer, size,0); /* GetFinalPathNameByHandle prepends path with "\\?\": */ - sprintf(buffer, buffer+4); + len = wcslen(wbuffer); + wmemmove(wbuffer,wbuffer+4,len-3); + if (len - 4 >= 2 && wbuffer[1] == L':' && wbuffer[0] >= L'A' && + wbuffer[0] <= L'Z') { + wbuffer[0] = wbuffer[0] + L'a' - L'A'; + } + + for ( ; *wbuffer; wbuffer++) + if (*wbuffer == L'\\') + *wbuffer = L'/'; CloseHandle(h); } FreeLibrary(hModule); @@ -1433,17 +1443,20 @@ efile_readlink(Efile_error* errInfo, char* name, char* buffer, size_t size) int efile_altname(Efile_error* errInfo, char* orig_name, char* buffer, size_t size) { - WIN32_FIND_DATA wfd; + WIN32_FIND_DATAW wfd; HANDLE fh; - char name[_MAX_PATH]; + WCHAR name[_MAX_PATH+1]; int name_len; - char* path; - char pathbuf[_MAX_PATH]; + WCHAR* path; + WCHAR pathbuf[_MAX_PATH+1]; /* Unclear weather GetCurrentDirectory will access one char after + _MAX_PATH */ + WCHAR *worig_name = (WCHAR *) orig_name; + WCHAR *wbuffer = (WCHAR *) buffer; int drive; /* Drive for filename (1 = A:, 2 = B: etc). */ /* Don't allow wildcards to be interpreted by system */ - if (strpbrk(orig_name, "?*")) { + if (wcspbrk(worig_name, L"?*")) { enoent: errInfo->posix_errno = ENOENT; errInfo->os_errno = ERROR_FILE_NOT_FOUND; @@ -1455,61 +1468,64 @@ efile_altname(Efile_error* errInfo, char* orig_name, char* buffer, size_t size) * slash, because it causes FindFirstFile() to fail on Win95. */ - if ((name_len = strlen(orig_name)) >= _MAX_PATH) { + if ((name_len = wcslen(worig_name)) >= _MAX_PATH) { goto enoent; } else { - strcpy(name, orig_name); + wcscpy(name, worig_name); if (name_len > 2 && ISSLASH(name[name_len-1]) && - name[name_len-2] != ':') { - name[name_len-1] = '\0'; + name[name_len-2] != L':') { + name[name_len-1] = L'\0'; } } /* Try to get disk from name. If none, get current disk. */ - if (name[1] != ':') { + if (name[1] != L':') { drive = 0; - if (GetCurrentDirectory(sizeof(pathbuf), pathbuf) && - pathbuf[1] == ':') { - drive = tolower(pathbuf[0]) - 'a' + 1; + if (GetCurrentDirectoryW(_MAX_PATH, pathbuf) && + pathbuf[1] == L':') { + drive = towlower(pathbuf[0]) - L'a' + 1; } - } else if (*name && name[2] == '\0') { + } else if (*name && name[2] == L'\0') { /* * X: and nothing more is an error. */ goto enoent; } else { - drive = tolower(*name) - 'a' + 1; + drive = towlower(*name) - L'a' + 1; } - fh = FindFirstFile(name,&wfd); + fh = FindFirstFileW(name,&wfd); if (fh == INVALID_HANDLE_VALUE) { - if (!(strpbrk(name, "./\\") && - (path = _fullpath(pathbuf, name, _MAX_PATH)) && + if (!(wcspbrk(name, L"./\\") && + (path = _wfullpath(pathbuf, name, _MAX_PATH)) && /* root dir. ('C:\') or UNC root dir. ('\\server\share\') */ - ((strlen(path) == 3) || IsRootUNCName(path)) && - (GetDriveType(path) > 1) ) ) { + ((wcslen(path) == 3) || is_root_unc_name(path)) && + (GetDriveTypeW(path) > 1) ) ) { errno = errno_map(GetLastError()); return check_error(-1, errInfo); } /* * Root directories (such as C:\ or \\server\share\ are fabricated. */ - strcpy(buffer,name); + wcscpy(wbuffer,name); return 1; } - strcpy(buffer,wfd.cAlternateFileName); - if (!*buffer) { - strcpy(buffer,wfd.cFileName); + wcscpy(wbuffer,wfd.cAlternateFileName); + if (!*wbuffer) { + wcscpy(wbuffer,wfd.cFileName); } - + FindClose(fh); return 1; } + int efile_link(Efile_error* errInfo, char* old, char* new) { - if(!CreateHardLink(new, old, NULL)) { + WCHAR *wold = (WCHAR *) old; + WCHAR *wnew = (WCHAR *) new; + if(!CreateHardLinkW(wnew, wold, NULL)) { return set_error(errInfo); } return 1; @@ -1523,22 +1539,24 @@ efile_symlink(Efile_error* errInfo, char* old, char* new) * (Vista only) */ HINSTANCE hModule = NULL; + WCHAR *wold = (WCHAR *) old; + WCHAR *wnew = (WCHAR *) new; if ((hModule = LoadLibrary("kernel32.dll")) != NULL) { typedef BOOLEAN (WINAPI * CREATESYMBOLICLINKFUNCPTR) ( - LPCSTR lpSymlinkFileName, - LPCSTR lpTargetFileName, + LPCWSTR lpSymlinkFileName, + LPCWSTR lpTargetFileName, DWORD dwFlags); CREATESYMBOLICLINKFUNCPTR pCreateSymbolicLink = (CREATESYMBOLICLINKFUNCPTR) GetProcAddress(hModule, - "CreateSymbolicLinkA"); - /* A for MBCS, W for UNICODE... char* above implies 'A'! */ + "CreateSymbolicLinkW"); + /* A for MBCS, W for UNICODE... char* above implies 'W'! */ if (pCreateSymbolicLink != NULL) { - DWORD attr = GetFileAttributes(old); + DWORD attr = GetFileAttributesW(wold); int flag = (attr != INVALID_FILE_ATTRIBUTES && attr & FILE_ATTRIBUTE_DIRECTORY) ? 1 : 0; /* SYMBOLIC_LINK_FLAG_DIRECTORY = 1 */ - BOOLEAN success = pCreateSymbolicLink(new, old, flag); + BOOLEAN success = pCreateSymbolicLink(wnew, wold, flag); FreeLibrary(hModule); if (success) { diff --git a/erts/emulator/internal_doc/dec.dat b/erts/emulator/internal_doc/dec.dat new file mode 100644 index 0000000000..771ef51baa --- /dev/null +++ b/erts/emulator/internal_doc/dec.dat @@ -0,0 +1,942 @@ +{[59],894}. +{[96],8175}. +{[180],8189}. +{[183],903}. +{[198],1236}. +{[230],1237}. +{[399],1240}. +{[415],1256}. +{[439],1248}. +{[601],1241}. +{[629],1257}. +{[658],1249}. +{[697],884}. +{[768],832}. +{[768,65],192}. +{[768,69],200}. +{[768,73],204}. +{[768,79],210}. +{[768,85],217}. +{[768,87],7808}. +{[768,89],7922}. +{[768,97],224}. +{[768,101],232}. +{[768,105],236}. +{[768,111],242}. +{[768,117],249}. +{[768,119],7809}. +{[768,121],7923}. +{[768,168],8173}. +{[768,770,65],7846}. +{[768,770,69],7872}. +{[768,770,79],7890}. +{[768,770,97],7847}. +{[768,770,101],7873}. +{[768,770,111],7891}. +{[768,772,69],7700}. +{[768,772,79],7760}. +{[768,772,101],7701}. +{[768,772,111],7761}. +{[768,774,65],7856}. +{[768,774,97],7857}. +{[768,776,85],475}. +{[768,776,117],476}. +{[768,776,953],8146}. +{[768,776,965],8162}. +{[768,787,837,913],8074}. +{[768,787,837,919],8090}. +{[768,787,837,937],8106}. +{[768,787,837,945],8066}. +{[768,787,837,951],8082}. +{[768,787,837,969],8098}. +{[768,787,913],7946}. +{[768,787,917],7962}. +{[768,787,919],7978}. +{[768,787,921],7994}. +{[768,787,927],8010}. +{[768,787,937],8042}. +{[768,787,945],7938}. +{[768,787,949],7954}. +{[768,787,951],7970}. +{[768,787,953],7986}. +{[768,787,959],8002}. +{[768,787,965],8018}. +{[768,787,969],8034}. +{[768,788,837,913],8075}. +{[768,788,837,919],8091}. +{[768,788,837,937],8107}. +{[768,788,837,945],8067}. +{[768,788,837,951],8083}. +{[768,788,837,969],8099}. +{[768,788,913],7947}. +{[768,788,917],7963}. +{[768,788,919],7979}. +{[768,788,921],7995}. +{[768,788,927],8011}. +{[768,788,933],8027}. +{[768,788,937],8043}. +{[768,788,945],7939}. +{[768,788,949],7955}. +{[768,788,951],7971}. +{[768,788,953],7987}. +{[768,788,959],8003}. +{[768,788,965],8019}. +{[768,788,969],8035}. +{[768,795,79],7900}. +{[768,795,85],7914}. +{[768,795,111],7901}. +{[768,795,117],7915}. +{[768,837,945],8114}. +{[768,837,951],8130}. +{[768,837,969],8178}. +{[768,913],8122}. +{[768,917],8136}. +{[768,919],8138}. +{[768,921],8154}. +{[768,927],8184}. +{[768,933],8170}. +{[768,937],8186}. +{[768,945],8048}. +{[768,949],8050}. +{[768,951],8052}. +{[768,953],8054}. +{[768,959],8056}. +{[768,965],8058}. +{[768,969],8060}. +{[768,8127],8141}. +{[768,8190],8157}. +{[769],833}. +{[769,65],193}. +{[769,67],262}. +{[769,69],201}. +{[769,71],500}. +{[769,73],205}. +{[769,75],7728}. +{[769,76],313}. +{[769,77],7742}. +{[769,78],323}. +{[769,79],211}. +{[769,80],7764}. +{[769,82],340}. +{[769,83],346}. +{[769,85],218}. +{[769,87],7810}. +{[769,89],221}. +{[769,90],377}. +{[769,97],225}. +{[769,99],263}. +{[769,101],233}. +{[769,103],501}. +{[769,105],237}. +{[769,107],7729}. +{[769,108],314}. +{[769,109],7743}. +{[769,110],324}. +{[769,111],243}. +{[769,112],7765}. +{[769,114],341}. +{[769,115],347}. +{[769,117],250}. +{[769,119],7811}. +{[769,121],253}. +{[769,122],378}. +{[769,168],8174}. +{[769,198],508}. +{[769,216],510}. +{[769,230],509}. +{[769,248],511}. +{[769,770,65],7844}. +{[769,770,69],7870}. +{[769,770,79],7888}. +{[769,770,97],7845}. +{[769,770,101],7871}. +{[769,770,111],7889}. +{[769,771,79],7756}. +{[769,771,85],7800}. +{[769,771,111],7757}. +{[769,771,117],7801}. +{[769,772,69],7702}. +{[769,772,79],7762}. +{[769,772,101],7703}. +{[769,772,111],7763}. +{[769,774,65],7854}. +{[769,774,97],7855}. +{[769,776,73],7726}. +{[769,776,85],471}. +{[769,776,105],7727}. +{[769,776,117],472}. +{[769,776,953],8147}. +{[769,776,965],8163}. +{[769,778,65],506}. +{[769,778,97],507}. +{[769,787,837,913],8076}. +{[769,787,837,919],8092}. +{[769,787,837,937],8108}. +{[769,787,837,945],8068}. +{[769,787,837,951],8084}. +{[769,787,837,969],8100}. +{[769,787,913],7948}. +{[769,787,917],7964}. +{[769,787,919],7980}. +{[769,787,921],7996}. +{[769,787,927],8012}. +{[769,787,937],8044}. +{[769,787,945],7940}. +{[769,787,949],7956}. +{[769,787,951],7972}. +{[769,787,953],7988}. +{[769,787,959],8004}. +{[769,787,965],8020}. +{[769,787,969],8036}. +{[769,788,837,913],8077}. +{[769,788,837,919],8093}. +{[769,788,837,937],8109}. +{[769,788,837,945],8069}. +{[769,788,837,951],8085}. +{[769,788,837,969],8101}. +{[769,788,913],7949}. +{[769,788,917],7965}. +{[769,788,919],7981}. +{[769,788,921],7997}. +{[769,788,927],8013}. +{[769,788,933],8029}. +{[769,788,937],8045}. +{[769,788,945],7941}. +{[769,788,949],7957}. +{[769,788,951],7973}. +{[769,788,953],7989}. +{[769,788,959],8005}. +{[769,788,965],8021}. +{[769,788,969],8037}. +{[769,795,79],7898}. +{[769,795,85],7912}. +{[769,795,111],7899}. +{[769,795,117],7913}. +{[769,807,67],7688}. +{[769,807,99],7689}. +{[769,837,945],8116}. +{[769,837,951],8132}. +{[769,837,959],8180}. +{[769,913],8123}. +{[769,917],8137}. +{[769,919],8139}. +{[769,921],8155}. +{[769,927],8185}. +{[769,933],8171}. +{[769,937],8187}. +{[769,945],8049}. +{[769,949],8051}. +{[769,951],8053}. +{[769,953],8055}. +{[769,959],8057}. +{[769,965],8059}. +{[769,969],8061}. +{[769,1043],1027}. +{[769,1050],1036}. +{[769,1075],1107}. +{[769,1082],1116}. +{[769,8127],8142}. +{[769,8190],8158}. +{[770,65],194}. +{[770,67],264}. +{[770,69],202}. +{[770,71],284}. +{[770,72],292}. +{[770,73],206}. +{[770,74],308}. +{[770,79],212}. +{[770,83],348}. +{[770,85],219}. +{[770,87],372}. +{[770,89],374}. +{[770,90],7824}. +{[770,97],226}. +{[770,99],265}. +{[770,101],234}. +{[770,103],285}. +{[770,104],293}. +{[770,105],238}. +{[770,106],309}. +{[770,111],244}. +{[770,115],349}. +{[770,117],251}. +{[770,119],373}. +{[770,121],375}. +{[770,122],7825}. +{[770,803,65],7852}. +{[770,803,69],7878}. +{[770,803,79],7896}. +{[770,803,97],7853}. +{[770,803,101],7879}. +{[770,803,111],7897}. +{[771,65],195}. +{[771,69],7868}. +{[771,73],296}. +{[771,78],209}. +{[771,79],213}. +{[771,85],360}. +{[771,86],7804}. +{[771,89],7928}. +{[771,97],227}. +{[771,101],7869}. +{[771,105],297}. +{[771,110],241}. +{[771,111],245}. +{[771,117],361}. +{[771,118],7805}. +{[771,121],7929}. +{[771,770,65],7850}. +{[771,770,69],7876}. +{[771,770,79],7894}. +{[771,770,97],7851}. +{[771,770,101],7877}. +{[771,770,111],7895}. +{[771,774,65],7860}. +{[771,774,97],7861}. +{[771,795,79],7904}. +{[771,795,85],7918}. +{[771,795,111],7905}. +{[771,795,117],7919}. +{[772,65],256}. +{[772,69],274}. +{[772,71],7712}. +{[772,73],298}. +{[772,79],332}. +{[772,85],362}. +{[772,97],257}. +{[772,101],275}. +{[772,103],7713}. +{[772,105],299}. +{[772,111],333}. +{[772,117],363}. +{[772,198],482}. +{[772,230],483}. +{[772,775,65],480}. +{[772,775,97],481}. +{[772,776,65],478}. +{[772,776,85],469}. +{[772,776,97],479}. +{[772,776,117],470}. +{[772,803,76],7736}. +{[772,803,82],7772}. +{[772,803,108],7737}. +{[772,803,114],7773}. +{[772,808,79],492}. +{[772,808,111],493}. +{[772,913],8121}. +{[772,921],8153}. +{[772,933],8169}. +{[772,945],8113}. +{[772,953],8145}. +{[772,965],8161}. +{[772,1048],1250}. +{[772,1059],1262}. +{[772,1080],1251}. +{[772,1091],1263}. +{[774,65],258}. +{[774,69],276}. +{[774,71],286}. +{[774,73],300}. +{[774,79],334}. +{[774,85],364}. +{[774,97],259}. +{[774,101],277}. +{[774,103],287}. +{[774,105],301}. +{[774,111],335}. +{[774,117],365}. +{[774,803,65],7862}. +{[774,803,97],7863}. +{[774,807,69],7708}. +{[774,807,101],7709}. +{[774,913],8120}. +{[774,921],8152}. +{[774,933],8168}. +{[774,945],8112}. +{[774,953],8144}. +{[774,965],8160}. +{[774,1040],1232}. +{[774,1045],1238}. +{[774,1046],1217}. +{[774,1048],1049}. +{[774,1059],1038}. +{[774,1072],1233}. +{[774,1077],1239}. +{[774,1078],1218}. +{[774,1080],1081}. +{[774,1091],1118}. +{[775,66],7682}. +{[775,67],266}. +{[775,68],7690}. +{[775,69],278}. +{[775,70],7710}. +{[775,71],288}. +{[775,72],7714}. +{[775,73],304}. +{[775,77],7744}. +{[775,78],7748}. +{[775,80],7766}. +{[775,82],7768}. +{[775,83],7776}. +{[775,84],7786}. +{[775,87],7814}. +{[775,88],7818}. +{[775,89],7822}. +{[775,90],379}. +{[775,98],7683}. +{[775,99],267}. +{[775,100],7691}. +{[775,101],279}. +{[775,102],7711}. +{[775,103],289}. +{[775,104],7715}. +{[775,109],7745}. +{[775,110],7749}. +{[775,112],7767}. +{[775,114],7769}. +{[775,115],7777}. +{[775,116],7787}. +{[775,119],7815}. +{[775,120],7819}. +{[775,121],7823}. +{[775,122],380}. +{[775,383],7835}. +{[775,769,83],7780}. +{[775,769,115],7781}. +{[775,774],784}. +{[775,780,83],7782}. +{[775,780,115],7783}. +{[775,803,83],7784}. +{[775,803,115],7785}. +{[776,65],196}. +{[776,69],203}. +{[776,72],7718}. +{[776,73],207}. +{[776,79],214}. +{[776,85],220}. +{[776,87],7812}. +{[776,88],7820}. +{[776,89],376}. +{[776,97],228}. +{[776,101],235}. +{[776,104],7719}. +{[776,105],239}. +{[776,111],246}. +{[776,116],7831}. +{[776,117],252}. +{[776,119],7813}. +{[776,120],7821}. +{[776,121],255}. +{[776,399],1242}. +{[776,415],1258}. +{[776,601],1243}. +{[776,629],1259}. +{[776,771,79],7758}. +{[776,771,111],7759}. +{[776,772,85],7802}. +{[776,772,117],7803}. +{[776,921],938}. +{[776,933],939}. +{[776,953],970}. +{[776,965],971}. +{[776,978],980}. +{[776,1030],1031}. +{[776,1040],1234}. +{[776,1045],1025}. +{[776,1046],1244}. +{[776,1047],1246}. +{[776,1048],1252}. +{[776,1054],1254}. +{[776,1059],1264}. +{[776,1063],1268}. +{[776,1067],1272}. +{[776,1072],1235}. +{[776,1077],1105}. +{[776,1078],1245}. +{[776,1079],1247}. +{[776,1080],1253}. +{[776,1086],1255}. +{[776,1091],1265}. +{[776,1095],1269}. +{[776,1099],1273}. +{[776,1110],1111}. +{[777,65],7842}. +{[777,69],7866}. +{[777,73],7880}. +{[777,79],7886}. +{[777,85],7910}. +{[777,89],7926}. +{[777,97],7843}. +{[777,101],7867}. +{[777,105],7881}. +{[777,111],7887}. +{[777,117],7911}. +{[777,121],7927}. +{[777,770,65],7848}. +{[777,770,69],7874}. +{[777,770,79],7892}. +{[777,770,97],7849}. +{[777,770,101],7875}. +{[777,770,111],7893}. +{[777,774,65],7858}. +{[777,774,97],7859}. +{[777,795,79],7902}. +{[777,795,85],7916}. +{[777,795,111],7903}. +{[777,795,117],7917}. +{[778,65],197}. +{[778,85],366}. +{[778,97],229}. +{[778,117],367}. +{[778,119],7832}. +{[778,121],7833}. +{[779,79],336}. +{[779,85],368}. +{[779,111],337}. +{[779,117],369}. +{[779,1059],1266}. +{[779,1091],1267}. +{[780,65],461}. +{[780,67],268}. +{[780,68],270}. +{[780,69],282}. +{[780,71],486}. +{[780,73],463}. +{[780,75],488}. +{[780,76],317}. +{[780,78],327}. +{[780,79],465}. +{[780,82],344}. +{[780,83],352}. +{[780,84],356}. +{[780,85],467}. +{[780,90],381}. +{[780,97],462}. +{[780,99],269}. +{[780,100],271}. +{[780,101],283}. +{[780,103],487}. +{[780,105],464}. +{[780,106],496}. +{[780,107],489}. +{[780,108],318}. +{[780,110],328}. +{[780,111],466}. +{[780,114],345}. +{[780,115],353}. +{[780,116],357}. +{[780,117],468}. +{[780,122],382}. +{[780,439],494}. +{[780,658],495}. +{[780,776,85],473}. +{[780,776,117],474}. +{[781,168],901}. +{[781,776],836}. +{[781,776,953],912}. +{[781,776,965],944}. +{[781,913],902}. +{[781,917],904}. +{[781,919],905}. +{[781,921],906}. +{[781,927],908}. +{[781,933],910}. +{[781,937],911}. +{[781,945],940}. +{[781,949],941}. +{[781,951],942}. +{[781,953],943}. +{[781,959],972}. +{[781,965],973}. +{[781,969],974}. +{[781,978],979}. +{[783,65],512}. +{[783,69],516}. +{[783,73],520}. +{[783,79],524}. +{[783,82],528}. +{[783,85],532}. +{[783,97],513}. +{[783,101],517}. +{[783,105],521}. +{[783,111],525}. +{[783,114],529}. +{[783,117],533}. +{[783,1140],1142}. +{[783,1141],1143}. +{[785,65],514}. +{[785,69],518}. +{[785,73],522}. +{[785,79],526}. +{[785,82],530}. +{[785,85],534}. +{[785,97],515}. +{[785,101],519}. +{[785,105],523}. +{[785,111],527}. +{[785,114],531}. +{[785,117],535}. +{[787],835}. +{[787,837,913],8072}. +{[787,837,919],8088}. +{[787,837,937],8104}. +{[787,837,945],8064}. +{[787,837,951],8080}. +{[787,837,969],8096}. +{[787,913],7944}. +{[787,917],7960}. +{[787,919],7976}. +{[787,921],7992}. +{[787,927],8008}. +{[787,937],8040}. +{[787,945],7936}. +{[787,949],7952}. +{[787,951],7968}. +{[787,953],7984}. +{[787,959],8000}. +{[787,961],8164}. +{[787,965],8016}. +{[787,969],8032}. +{[788,837,913],8073}. +{[788,837,919],8089}. +{[788,837,937],8105}. +{[788,837,945],8065}. +{[788,837,951],8081}. +{[788,837,969],8097}. +{[788,913],7945}. +{[788,917],7961}. +{[788,919],7977}. +{[788,921],7993}. +{[788,927],8009}. +{[788,929],8172}. +{[788,933],8025}. +{[788,937],8041}. +{[788,945],7937}. +{[788,949],7953}. +{[788,951],7969}. +{[788,953],7985}. +{[788,959],8001}. +{[788,961],8165}. +{[788,965],8017}. +{[788,969],8033}. +{[795,79],416}. +{[795,85],431}. +{[795,111],417}. +{[795,117],432}. +{[803,65],7840}. +{[803,66],7684}. +{[803,68],7692}. +{[803,69],7864}. +{[803,72],7716}. +{[803,73],7882}. +{[803,75],7730}. +{[803,76],7734}. +{[803,77],7746}. +{[803,78],7750}. +{[803,79],7884}. +{[803,82],7770}. +{[803,83],7778}. +{[803,84],7788}. +{[803,85],7908}. +{[803,86],7806}. +{[803,87],7816}. +{[803,89],7924}. +{[803,90],7826}. +{[803,97],7841}. +{[803,98],7685}. +{[803,100],7693}. +{[803,101],7865}. +{[803,104],7717}. +{[803,105],7883}. +{[803,107],7731}. +{[803,108],7735}. +{[803,109],7747}. +{[803,110],7751}. +{[803,111],7885}. +{[803,114],7771}. +{[803,115],7779}. +{[803,116],7789}. +{[803,117],7909}. +{[803,118],7807}. +{[803,119],7817}. +{[803,121],7925}. +{[803,122],7827}. +{[803,795,79],7906}. +{[803,795,85],7920}. +{[803,795,111],7907}. +{[803,795,117],7921}. +{[804,85],7794}. +{[804,117],7795}. +{[805,65],7680}. +{[805,97],7681}. +{[807,67],199}. +{[807,68],7696}. +{[807,71],290}. +{[807,72],7720}. +{[807,75],310}. +{[807,76],315}. +{[807,78],325}. +{[807,82],342}. +{[807,83],350}. +{[807,84],354}. +{[807,99],231}. +{[807,100],7697}. +{[807,103],291}. +{[807,104],7721}. +{[807,107],311}. +{[807,108],316}. +{[807,110],326}. +{[807,114],343}. +{[807,115],351}. +{[807,116],355}. +{[808,65],260}. +{[808,69],280}. +{[808,73],302}. +{[808,79],490}. +{[808,85],370}. +{[808,97],261}. +{[808,101],281}. +{[808,105],303}. +{[808,111],491}. +{[808,117],371}. +{[813,68],7698}. +{[813,69],7704}. +{[813,76],7740}. +{[813,78],7754}. +{[813,84],7792}. +{[813,85],7798}. +{[813,100],7699}. +{[813,101],7705}. +{[813,108],7741}. +{[813,110],7755}. +{[813,116],7793}. +{[813,117],7799}. +{[814,72],7722}. +{[814,104],7723}. +{[816,69],7706}. +{[816,73],7724}. +{[816,85],7796}. +{[816,101],7707}. +{[816,105],7725}. +{[816,117],7797}. +{[817,66],7686}. +{[817,68],7694}. +{[817,75],7732}. +{[817,76],7738}. +{[817,78],7752}. +{[817,82],7774}. +{[817,84],7790}. +{[817,90],7828}. +{[817,98],7687}. +{[817,100],7695}. +{[817,104],7830}. +{[817,107],7733}. +{[817,108],7739}. +{[817,110],7753}. +{[817,114],7775}. +{[817,116],7791}. +{[817,122],7829}. +{[834,168],8129}. +{[834,776,953],8151}. +{[834,776,965],8167}. +{[834,787,837,913],8078}. +{[834,787,837,919],8094}. +{[834,787,837,937],8110}. +{[834,787,837,945],8070}. +{[834,787,837,951],8086}. +{[834,787,837,969],8102}. +{[834,787,913],7950}. +{[834,787,919],7982}. +{[834,787,921],7998}. +{[834,787,937],8046}. +{[834,787,945],7942}. +{[834,787,951],7974}. +{[834,787,953],7990}. +{[834,787,965],8022}. +{[834,787,969],8038}. +{[834,788,837,913],8079}. +{[834,788,837,919],8095}. +{[834,788,837,937],8111}. +{[834,788,837,945],8071}. +{[834,788,837,951],8087}. +{[834,788,837,969],8103}. +{[834,788,913],7951}. +{[834,788,919],7983}. +{[834,788,921],7999}. +{[834,788,933],8031}. +{[834,788,937],8047}. +{[834,788,945],7943}. +{[834,788,951],7975}. +{[834,788,953],7991}. +{[834,788,965],8023}. +{[834,788,969],8039}. +{[834,837,945],8119}. +{[834,837,951],8135}. +{[834,837,969],8183}. +{[834,945],8118}. +{[834,951],8134}. +{[834,953],8150}. +{[834,965],8166}. +{[834,969],8182}. +{[834,8127],8143}. +{[834,8190],8159}. +{[837,913],8124}. +{[837,919],8140}. +{[837,937],8188}. +{[837,945],8115}. +{[837,951],8131}. +{[837,969],8179}. +{[953],8126}. +{[1463,1488],64302}. +{[1463,1522],64287}. +{[1464,1488],64303}. +{[1465,1493],64331}. +{[1468,1488],64304}. +{[1468,1489],64305}. +{[1468,1490],64306}. +{[1468,1491],64307}. +{[1468,1492],64308}. +{[1468,1493],64309}. +{[1468,1494],64310}. +{[1468,1496],64312}. +{[1468,1497],64313}. +{[1468,1498],64314}. +{[1468,1499],64315}. +{[1468,1500],64316}. +{[1468,1502],64318}. +{[1468,1504],64320}. +{[1468,1505],64321}. +{[1468,1507],64323}. +{[1468,1508],64324}. +{[1468,1510],64326}. +{[1468,1511],64327}. +{[1468,1512],64328}. +{[1468,1513],64329}. +{[1468,1514],64330}. +{[1471,1489],64332}. +{[1471,1499],64333}. +{[1471,1508],64334}. +{[1473,1468,1513],64300}. +{[1473,1513],64298}. +{[1474,1468,1513],64301}. +{[1474,1513],64299}. +{[2364,2325],2392}. +{[2364,2326],2393}. +{[2364,2327],2394}. +{[2364,2332],2395}. +{[2364,2337],2396}. +{[2364,2338],2397}. +{[2364,2344],2345}. +{[2364,2347],2398}. +{[2364,2351],2399}. +{[2364,2352],2353}. +{[2364,2355],2356}. +{[2492,2465],2524}. +{[2492,2466],2525}. +{[2492,2476],2480}. +{[2492,2479],2527}. +{[2494,2503],2507}. +{[2519,2503],2508}. +{[2620,2582],2649}. +{[2620,2583],2650}. +{[2620,2588],2651}. +{[2620,2593],2652}. +{[2620,2603],2654}. +{[2876,2849],2908}. +{[2876,2850],2909}. +{[2876,2863],2911}. +{[2878,2887],2891}. +{[2902,2887],2888}. +{[2903,2887],2892}. +{[3006,3014],3018}. +{[3006,3015],3019}. +{[3031,2962],2964}. +{[3031,3014],3020}. +{[3158,3142],3144}. +{[3266,3270],3274}. +{[3285,3263],3264}. +{[3285,3266,3270],3275}. +{[3285,3270],3271}. +{[3286,3270],3272}. +{[3390,3398],3402}. +{[3390,3399],3403}. +{[3415,3398],3404}. +{[3634,3661],3635}. +{[3762,3789],3763}. +{[3953,3954],3955}. +{[3953,3956],3957}. +{[3953,3968],3969}. +{[3953,3968,4018],3959}. +{[3953,3968,4019],3961}. +{[3968,4018],3958}. +{[3968,4019],3960}. +{[4021,3904],3945}. +{[4021,3984],4025}. +{[4023,3906],3907}. +{[4023,3916],3917}. +{[4023,3921],3922}. +{[4023,3926],3927}. +{[4023,3931],3932}. +{[4023,3986],3987}. +{[4023,3996],3997}. +{[4023,4001],4002}. +{[4023,4006],4007}. +{[4023,4011],4012}. +{[12441,12358],12436}. +{[12441,12363],12364}. +{[12441,12365],12366}. +{[12441,12367],12368}. +{[12441,12369],12370}. +{[12441,12371],12372}. +{[12441,12373],12374}. +{[12441,12375],12376}. +{[12441,12377],12378}. +{[12441,12379],12380}. +{[12441,12381],12382}. +{[12441,12383],12384}. +{[12441,12385],12386}. +{[12441,12388],12389}. +{[12441,12390],12391}. +{[12441,12392],12393}. +{[12441,12399],12400}. +{[12441,12402],12403}. +{[12441,12405],12406}. +{[12441,12408],12409}. +{[12441,12411],12412}. +{[12441,12445],12446}. +{[12441,12454],12532}. +{[12441,12459],12460}. +{[12441,12461],12462}. +{[12441,12463],12464}. +{[12441,12465],12466}. +{[12441,12467],12468}. +{[12441,12469],12470}. +{[12441,12471],12472}. +{[12441,12473],12474}. +{[12441,12475],12476}. +{[12441,12477],12478}. +{[12441,12479],12480}. +{[12441,12481],12482}. +{[12441,12484],12485}. +{[12441,12486],12487}. +{[12441,12488],12489}. +{[12441,12495],12496}. +{[12441,12498],12499}. +{[12441,12501],12502}. +{[12441,12504],12505}. +{[12441,12507],12508}. +{[12441,12527],12535}. +{[12441,12528],12536}. +{[12441,12529],12537}. +{[12441,12530],12538}. +{[12441,12541],12542}. +{[12442,12399],12401}. +{[12442,12402],12404}. +{[12442,12405],12407}. +{[12442,12408],12410}. +{[12442,12411],12413}. +{[12442,12495],12497}. +{[12442,12498],12500}. +{[12442,12501],12503}. +{[12442,12504],12506}. +{[12442,12507],12509}. diff --git a/erts/emulator/internal_doc/dec.erl b/erts/emulator/internal_doc/dec.erl new file mode 100644 index 0000000000..0315f2a52d --- /dev/null +++ b/erts/emulator/internal_doc/dec.erl @@ -0,0 +1,237 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2000-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%% This program is used to generate a header file with data for +%% normalizing denormalized unicode. + +%% The C header is generated from a text file containing tuples in the +%% following format: +%% {RevList,Translation} +%% Where 'RevList' is a reversed list of the denormalized repressentation of +%% the character 'Translation'. An example would be the swedish character +%% '�', which would be represented in the file as: +%% {[776,111],246}, as the denormalized representation of codepoint 246 +%% is [111,776] (i.e an 'o' followed by the "double dot accent character 776), +%% while '�' instead is represented as {[776,97],228}, as the denormalized +%% form would be [97,776] (same accent but an 'a' instead). +%% The datafile is generated from the table on Apple's developer connection +%% http://developer.apple.com/library/mac/#technotes/tn/tn1150table.html +%% The generating is done whenever new data is present (i.e. dec.dat has +%% to be changed) and not for every build. The product (the C header) is copied +%% to $ERL_TOP/erts/beam after generation and checked in. +%% The program and the data file is included for reference. + +-module(dec). + +-compile(export_all). + +-define(HASH_SIZE_FACTOR,2). +-define(BIG_PREFIX_SIZE,392). + +-define(INPUT_FILE_NAME,"dec.dat"). +-define(OUTPUT_FILE_NAME,"erl_unicode_normalize.h"). + +read(FName) -> + {ok,L} = file:consult(FName), + [{A,B} || {A,B} <- L, + length(A) > 1% , hd(A) < 769 + ]. + +dec() -> + L = read(?INPUT_FILE_NAME), + G = group(L), + {ok,Out} = file:open(?OUTPUT_FILE_NAME,[write]), + io:format + (Out, + "/*~n" + "* %CopyrightBegin%~n" + "*~n" + "* Copyright Ericsson AB 1999-2010. All Rights Reserved.~n" + "*~n" + "* The contents of this file are subject to the Erlang Public License,~n" + "* Version 1.1, (the \"License\"); you may not use this file except in~n" + "* compliance with the License. You should have received a copy of the~n" + "* Erlang Public License along with this software. If not, it can be~n" + "* retrieved online at http://www.erlang.org/.~n" + "*~n" + "* Software distributed under the License is distributed on an " + "\"AS IS\"~n" + "* basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See~n" + "* the License for the specific language governing rights and " + "limitations~n" + "* under the License.~n" + "*~n" + "* %CopyrightEnd%~n" + "*/~n" + "/*~n" + "* This file is automatically generated by ~p.erl, " + "do not edit manually~n" + "*/~n", + [?MODULE]), + + io:format(Out, + "#define HASH_SIZE_FACTOR ~w~n" + "typedef struct _compose_entry {~n" + " Uint16 c;~n" + " Uint16 res;~n" + " Uint16 num_subs;~n" + " struct _compose_entry *subs;~n" + " int *hash;~n" + "} CompEntry;~n~n" + "static int compose_tab_size = ~p;~n", + [?HASH_SIZE_FACTOR,length(G)]), + d(Out,G,[],0), + PreTab = tuple_to_list(make_prefix_table(G,erlang:make_tuple(102,0))), + dump_prefixes(Out,PreTab), +%% Using this cuts down on the searching in the +%% actual implementation, but wastes memory with little real gain.. +%% LL = lists:flatten([PartList || {PartList,_} <- L]), +%% BigPreTab = tuple_to_list( +%% make_big_prefixes(LL, +%% erlang:make_tuple(?BIG_PREFIX_SIZE,0))), +%% dump_big_prefixes(Out,BigPreTab), + file:close(Out), + ok. + + + +d(Out,List,D,C) -> + d_sub(Out,List,D,C), + d_top_hash(Out,List,D,C), + d_top(Out,List,D,C). +d_sub(_Out,[],_D,_C) -> + ok; +d_sub(Out,[{_CP,[],_Res}|T],D,C) -> + d_sub(Out,T,D,C+1); +d_sub(Out,[{_CP,Subs,_Res0}|T],D,C) -> + d(Out,Subs,[C|D],0), + d_sub(Out,T,D,C+1). +d_top(Out,L,D,C) -> + io:format(Out,"static CompEntry ~s[] = {~n",[format_depth(D)]), + d_top_1(Out,L,D,C), + io:format(Out,"}; /* ~s */ ~n",[format_depth(D)]). + +d_top_1(_Out,[],_D,_C) -> + ok; +d_top_1(Out,[{CP,[],Res}|T],D,C) -> + io:format(Out, + "{~w, ~w, 0, NULL, NULL}",[CP,Res]), + if + T =:= [] -> + io:format(Out,"~n",[]); + true -> + io:format(Out,",~n",[]) + end, + d_top_1(Out,T,D,C+1); +d_top_1(Out,[{CP,Subs,_Res}|T],D,C) -> + io:format(Out, + "{~w, 0, ~w, ~s, ~s}",[CP,length(Subs), + format_depth([C|D]), + "hash_"++format_depth([C|D])]), + if + T =:= [] -> + io:format(Out,"~n",[]); + true -> + io:format(Out,",~n",[]) + end, + d_top_1(Out,T,D,C+1). + + +d_top_hash(Out,List,D,_C) -> + HSize = length(List)*?HASH_SIZE_FACTOR, + io:format(Out,"static int ~s[~p] = ~n",["hash_"++format_depth(D),HSize]), + Tup = d_top_hash_1(List,0,erlang:make_tuple(HSize,-1),HSize), + io:format(Out,"~p; /* ~s */ ~n",[Tup,"hash_"++format_depth(D)]). + +d_top_hash_1([],_,Hash,_HSize) -> + Hash; +d_top_hash_1([{CP,_,_}|T],Index,Hash,HSize) -> + Bucket = hash_search(Hash,HSize,CP rem HSize), + d_top_hash_1(T,Index+1,erlang:setelement(Bucket+1,Hash,Index),HSize). + +hash_search(Hash,_HSize,Bucket) when element(Bucket+1,Hash) =:= -1 -> + Bucket; +hash_search(Hash,HSize,Bucket) -> + hash_search(Hash,HSize,(Bucket + 1) rem HSize). + +format_depth(D) -> + lists:reverse(tl(lists:reverse(lists:flatten(["compose_tab_",[ integer_to_list(X) ++ "_" || X <- lists:reverse(D) ]])))). + + + + +make_prefix_table([],Table) -> + Table; +make_prefix_table([{C,_,_}|T],Table) when C =< 4023 -> + Index = (C div 32) + 1 - 24, + Pos = C rem 32, + X = element(Index,Table), + Y = X bor (1 bsl Pos), + NewTab = setelement(Index,Table,Y), + make_prefix_table(T,NewTab); +make_prefix_table([_|T],Tab) -> + make_prefix_table(T,Tab). + +dump_prefixes(Out,L) -> + io:format(Out,"#define COMP_CANDIDATE_MAP_OFFSET 24~n",[]), + io:format(Out,"static Uint32 comp_candidate_map[] = {~n",[]), + dump_prefixes_1(Out,L). +dump_prefixes_1(Out,[H]) -> + io:format(Out," 0x~8.16.0BU~n",[H]), + io:format(Out,"};~n",[]); +dump_prefixes_1(Out,[H|T]) -> + io:format(Out," 0x~8.16.0BU,~n",[H]), + dump_prefixes_1(Out,T). + +%% make_big_prefixes([],Table) -> +%% Table; +%% make_big_prefixes([C|T],Table) -> +%% Index = (C div 32) + 1, +%% Pos = C rem 32, +%% X = element(Index,Table), +%% Y = X bor (1 bsl Pos), +%% NewTab = setelement(Index,Table,Y), +%% make_big_prefixes(T,NewTab). + +%% dump_big_prefixes(Out,L) -> +%% io:format(Out,"#define BIG_COMP_CANDIDATE_SIZE ~w~n", [?BIG_PREFIX_SIZE]), +%% io:format(Out,"static Uint32 big_comp_candidate_map[] = {~n",[]), +%% dump_prefixes_1(Out,L). + +pick([],_,Acc) -> + {lists:reverse(Acc),[]}; +pick([{[H|TT],N}|T],H,Acc) -> + pick(T,H,[{TT,N}|Acc]); +pick([{[H|_],_}|_]=L,M,Acc) when H =/= M -> + {lists:reverse(Acc),L}. + + +group([]) -> + []; +group([{[H],N}|T]) -> + {Part,Rest} = pick(T,H,[]), + [{H,group(Part),N}| group(Rest)]; +group([{[H|_],_}|_]=L) -> + {Part,Rest} = pick(L,H,[]), + [{H,group(Part),0}| group(Rest)]. + + + + + diff --git a/erts/emulator/sys/common/erl_poll.c b/erts/emulator/sys/common/erl_poll.c index c17806d96c..4d0ca97889 100644 --- a/erts/emulator/sys/common/erl_poll.c +++ b/erts/emulator/sys/common/erl_poll.c @@ -124,9 +124,9 @@ erts_smp_mtx_unlock(&(PS)->mtx) #define ERTS_POLLSET_SET_POLLED_CHK(PS) \ - ((int) erts_smp_atomic_xchg(&(PS)->polled, (long) 1)) + ((int) erts_smp_atomic_xchg(&(PS)->polled, (erts_aint_t) 1)) #define ERTS_POLLSET_UNSET_POLLED(PS) \ - erts_smp_atomic_set(&(PS)->polled, (long) 0) + erts_smp_atomic_set(&(PS)->polled, (erts_aint_t) 0) #define ERTS_POLLSET_IS_POLLED(PS) \ ((int) erts_smp_atomic_read(&(PS)->polled)) @@ -134,11 +134,11 @@ #define ERTS_POLLSET_SET_POLLER_WOKEN(PS) \ do { \ ERTS_THR_MEMORY_BARRIER; \ - erts_smp_atomic_set(&(PS)->woken, (long) 1); \ + erts_smp_atomic_set(&(PS)->woken, (erts_aint_t) 1); \ } while (0) #define ERTS_POLLSET_UNSET_POLLER_WOKEN(PS) \ do { \ - erts_smp_atomic_set(&(PS)->woken, (long) 0); \ + erts_smp_atomic_set(&(PS)->woken, (erts_aint_t) 0); \ ERTS_THR_MEMORY_BARRIER; \ } while (0) #define ERTS_POLLSET_IS_POLLER_WOKEN(PS) \ @@ -179,9 +179,9 @@ do { \ #if ERTS_POLL_USE_UPDATE_REQUESTS_QUEUE #define ERTS_POLLSET_SET_HAVE_UPDATE_REQUESTS(PS) \ - erts_smp_atomic_set(&(PS)->have_update_requests, (long) 1) + erts_smp_atomic_set(&(PS)->have_update_requests, (erts_aint_t) 1) #define ERTS_POLLSET_UNSET_HAVE_UPDATE_REQUESTS(PS) \ - erts_smp_atomic_set(&(PS)->have_update_requests, (long) 0) + erts_smp_atomic_set(&(PS)->have_update_requests, (erts_aint_t) 0) #define ERTS_POLLSET_HAVE_UPDATE_REQUESTS(PS) \ ((int) erts_smp_atomic_read(&(PS)->have_update_requests)) #else @@ -202,13 +202,13 @@ do { \ #define ERTS_POLLSET_UNSET_INTERRUPTED_CHK(PS) unset_interrupted_chk((PS)) #define ERTS_POLLSET_UNSET_INTERRUPTED(PS) \ do { \ - erts_smp_atomic_set(&(PS)->interrupt, (long) 0); \ + erts_smp_atomic_set(&(PS)->interrupt, (erts_aint_t) 0); \ ERTS_THR_MEMORY_BARRIER; \ } while (0) #define ERTS_POLLSET_SET_INTERRUPTED(PS) \ do { \ ERTS_THR_MEMORY_BARRIER; \ - erts_smp_atomic_set(&(PS)->interrupt, (long) 1); \ + erts_smp_atomic_set(&(PS)->interrupt, (erts_aint_t) 1); \ } while (0) #define ERTS_POLLSET_IS_INTERRUPTED(PS) \ ((int) erts_smp_atomic_read(&(PS)->interrupt)) @@ -356,7 +356,7 @@ unset_interrupted_chk(ErtsPollSet ps) res = ps->interrupt; ps->interrupt = 0; #else - res = (int) erts_smp_atomic_xchg(&ps->interrupt, (long) 0); + res = (int) erts_smp_atomic_xchg(&ps->interrupt, (erts_aint_t) 0); ERTS_THR_MEMORY_BARRIER; #endif return res; @@ -369,7 +369,7 @@ static ERTS_INLINE int set_poller_woken_chk(ErtsPollSet ps) { ERTS_THR_MEMORY_BARRIER; - return (int) erts_smp_atomic_xchg(&ps->woken, (long) 1); + return (int) erts_smp_atomic_xchg(&ps->woken, (erts_aint_t) 1); } #endif @@ -1918,7 +1918,7 @@ check_fd_events(ErtsPollSet ps, SysTimeval *tv, int max_res, int *ps_locked) return 0; } else { - long timeout = tv->tv_sec*1000 + tv->tv_usec/1000; + erts_aint_t timeout = tv->tv_sec*1000 + tv->tv_usec/1000; ASSERT(timeout >= 0); erts_smp_atomic_set(&ps->timeout, timeout); #if ERTS_POLL_USE_FALLBACK @@ -2112,7 +2112,7 @@ ERTS_POLL_EXPORT(erts_poll_wait)(ErtsPollSet ps, #endif done: - erts_smp_atomic_set(&ps->timeout, LONG_MAX); + erts_smp_atomic_set(&ps->timeout, ERTS_AINT_T_MAX); #ifdef ERTS_POLL_DEBUG_PRINT erts_printf("Leaving %s = erts_poll_wait()\n", res == 0 ? "0" : erl_errno_id(res)); @@ -2150,10 +2150,12 @@ ERTS_POLL_EXPORT(erts_poll_interrupt)(ErtsPollSet ps, int set) * is not guaranteed that it will timeout before 'msec' milli seconds. */ void -ERTS_POLL_EXPORT(erts_poll_interrupt_timed)(ErtsPollSet ps, int set, long msec) +ERTS_POLL_EXPORT(erts_poll_interrupt_timed)(ErtsPollSet ps, + int set, + long msec) { if (set) { - if (erts_smp_atomic_read(&ps->timeout) > msec) { + if (erts_smp_atomic_read(&ps->timeout) > (erts_aint_t) msec) { ERTS_POLLSET_SET_INTERRUPTED(ps); #if ERTS_POLL_ASYNC_INTERRUPT_SUPPORT || defined(ERTS_SMP) wake_poller(ps); @@ -2315,7 +2317,7 @@ ERTS_POLL_EXPORT(erts_poll_create_pollset)(void) #else erts_smp_atomic_init(&ps->interrupt, 0); #endif - erts_smp_atomic_init(&ps->timeout, LONG_MAX); + erts_smp_atomic_init(&ps->timeout, ERTS_AINT_T_MAX); #ifdef ERTS_POLL_COUNT_AVOIDED_WAKEUPS erts_smp_atomic_init(&ps->no_avoided_wakeups, 0); erts_smp_atomic_init(&ps->no_avoided_interrupts, 0); diff --git a/erts/emulator/sys/common/erl_sys_common_misc.c b/erts/emulator/sys/common/erl_sys_common_misc.c new file mode 100644 index 0000000000..461e763f03 --- /dev/null +++ b/erts/emulator/sys/common/erl_sys_common_misc.c @@ -0,0 +1,107 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2006-2010. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + + + +/* + * Darwin needs conversion! + * http://developer.apple.com/library/mac/#qa/qa2001/qa1235.html + */ + +#ifdef HAVE_CONFIG_H +# include "config.h" +#endif + +#include "sys.h" +#include "global.h" + +#if defined(__APPLE__) && defined(__MACH__) && !defined(__DARWIN__) +#define __DARWIN__ 1 +#endif + +#if !defined(__WIN32__) +#include <locale.h> +#if !defined(HAVE_SETLOCALE) || !defined(HAVE_NL_LANGINFO) || !defined(HAVE_LANGINFO_H) +#define PRIMITIVE_UTF8_CHECK 1 +#else +#include <langinfo.h> +#endif +#endif + +/* Written once and only once */ + +static int filename_encoding = ERL_FILENAME_UNKNOWN; +#if defined(__WIN32__) || defined(__DARWIN__) +static int user_filename_encoding = ERL_FILENAME_UTF8; /* Default unicode on windows */ +#else +static int user_filename_encoding = ERL_FILENAME_LATIN1; +#endif +void erts_set_user_requested_filename_encoding(int encoding) +{ + user_filename_encoding = encoding; +} + +int erts_get_user_requested_filename_encoding(void) +{ + return user_filename_encoding; +} + +void erts_init_sys_common_misc(void) +{ +#if defined(__WIN32__) + /* win_efile will totally fail if this is not set. */ + filename_encoding = ERL_FILENAME_WIN_WCHAR; +#else + if (user_filename_encoding != ERL_FILENAME_UNKNOWN) { + filename_encoding = user_filename_encoding; + } else { + char *l; + filename_encoding = ERL_FILENAME_LATIN1; +# ifdef PRIMITIVE_UTF8_CHECK + setlocale(LC_CTYPE, ""); /* Set international environment, + ignore result */ + if (((l = getenv("LC_ALL")) && *l) || + ((l = getenv("LC_CTYPE")) && *l) || + ((l = getenv("LANG")) && *l)) { + if (strstr(l, "UTF-8")) { + filename_encoding = ERL_FILENAME_UTF8; + } + } + +# else + l = setlocale(LC_CTYPE, ""); /* Set international environment */ + if (l != NULL) { + if (strcmp(nl_langinfo(CODESET), "UTF-8") == 0) { + filename_encoding = ERL_FILENAME_UTF8; + } + } +# endif + } +# if defined(__DARWIN__) + if (filename_encoding == ERL_FILENAME_UTF8) { + filename_encoding = ERL_FILENAME_UTF8_MAC; + } +# endif +#endif +} + +int erts_get_native_filename_encoding(void) +{ + return filename_encoding; +} diff --git a/erts/emulator/sys/unix/sys.c b/erts/emulator/sys/unix/sys.c index 01ba773688..d021baa6bf 100644 --- a/erts/emulator/sys/unix/sys.c +++ b/erts/emulator/sys/unix/sys.c @@ -237,9 +237,9 @@ static int max_files = -1; #ifdef ERTS_SMP erts_smp_atomic_t erts_break_requested; #define ERTS_SET_BREAK_REQUESTED \ - erts_smp_atomic_set(&erts_break_requested, (long) 1) + erts_smp_atomic_set(&erts_break_requested, (erts_aint_t) 1) #define ERTS_UNSET_BREAK_REQUESTED \ - erts_smp_atomic_set(&erts_break_requested, (long) 0) + erts_smp_atomic_set(&erts_break_requested, (erts_aint_t) 0) #else volatile int erts_break_requested = 0; #define ERTS_SET_BREAK_REQUESTED (erts_break_requested = 1) diff --git a/erts/emulator/sys/win32/erl_poll.c b/erts/emulator/sys/win32/erl_poll.c index a766fe9575..d84ae2ede2 100644 --- a/erts/emulator/sys/win32/erl_poll.c +++ b/erts/emulator/sys/win32/erl_poll.c @@ -297,11 +297,11 @@ struct ErtsPollSet_ { #define ERTS_POLLSET_UNLOCK(PS) \ erts_smp_mtx_unlock(&(PS)->mtx) #define ERTS_POLLSET_SET_POLLED_CHK(PS) \ - ((int) erts_smp_atomic_xchg(&(PS)->polled, (long) 1)) + ((int) erts_smp_atomic_xchg(&(PS)->polled, (erts_aint_t) 1)) #define ERTS_POLLSET_SET_POLLED(PS) \ - erts_smp_atomic_set(&(PS)->polled, (long) 1) + erts_smp_atomic_set(&(PS)->polled, (erts_aint_t) 1) #define ERTS_POLLSET_UNSET_POLLED(PS) \ - erts_smp_atomic_set(&(PS)->polled, (long) 0) + erts_smp_atomic_set(&(PS)->polled, (erts_aint_t) 0) #define ERTS_POLLSET_IS_POLLED(PS) \ ((int) erts_smp_atomic_read(&(PS)->polled)) @@ -309,11 +309,11 @@ struct ErtsPollSet_ { #define ERTS_POLLSET_SET_POLLER_WOKEN(PS) \ do { \ ERTS_THR_MEMORY_BARRIER; \ - erts_smp_atomic_set(&(PS)->woken, (long) 1); \ + erts_smp_atomic_set(&(PS)->woken, (erts_aint_t) 1); \ } while (0) #define ERTS_POLLSET_UNSET_POLLER_WOKEN(PS) \ do { \ - erts_smp_atomic_set(&(PS)->woken, (long) 0); \ + erts_smp_atomic_set(&(PS)->woken, (erts_aint_t) 0); \ ERTS_THR_MEMORY_BARRIER; \ } while (0) #define ERTS_POLLSET_IS_POLLER_WOKEN(PS) \ @@ -322,13 +322,13 @@ do { \ #define ERTS_POLLSET_UNSET_INTERRUPTED_CHK(PS) unset_interrupted_chk((PS)) #define ERTS_POLLSET_UNSET_INTERRUPTED(PS) \ do { \ - erts_smp_atomic_set(&(PS)->interrupt, (long) 0); \ + erts_smp_atomic_set(&(PS)->interrupt, (erts_aint_t) 0); \ ERTS_THR_MEMORY_BARRIER; \ } while (0) #define ERTS_POLLSET_SET_INTERRUPTED(PS) \ do { \ ERTS_THR_MEMORY_BARRIER; \ - erts_smp_atomic_set(&(PS)->interrupt, (long) 1); \ + erts_smp_atomic_set(&(PS)->interrupt, (erts_aint_t) 1); \ } while (0) #define ERTS_POLLSET_IS_INTERRUPTED(PS) \ ((int) erts_smp_atomic_read(&(PS)->interrupt)) @@ -336,7 +336,7 @@ do { \ static ERTS_INLINE int unset_interrupted_chk(ErtsPollSet ps) { - int res = (int) erts_smp_atomic_xchg(&ps->interrupt, (long) 0); + int res = (int) erts_smp_atomic_xchg(&ps->interrupt, (erts_aint_t) 0); ERTS_THR_MEMORY_BARRIER; return res; @@ -346,7 +346,7 @@ static ERTS_INLINE int set_poller_woken_chk(ErtsPollSet ps) { ERTS_THR_MEMORY_BARRIER; - return (int) erts_smp_atomic_xchg(&ps->woken, (long) 1); + return (int) erts_smp_atomic_xchg(&ps->woken, (erts_aint_t) 1); } #else @@ -413,9 +413,9 @@ set_poller_woken_chk(ErtsPollSet ps) #ifdef ERTS_SMP extern erts_smp_atomic_t erts_break_requested; #define ERTS_SET_BREAK_REQUESTED \ - erts_smp_atomic_set(&erts_break_requested, (long) 1) + erts_smp_atomic_set(&erts_break_requested, (erts_aint_t) 1) #define ERTS_UNSET_BREAK_REQUESTED \ - erts_smp_atomic_set(&erts_break_requested, (long) 0) + erts_smp_atomic_set(&erts_break_requested, (erts_aint_t) 0) #else extern volatile int erts_break_requested; #define ERTS_SET_BREAK_REQUESTED (erts_break_requested = 1) @@ -986,7 +986,7 @@ void erts_poll_interrupt_timed(ErtsPollSet ps, HARDTRACEF(("In erts_poll_interrupt_timed(%d,%ld)",set,msec)); #ifdef ERTS_SMP if (set) { - if (erts_smp_atomic_read(&ps->timeout) > msec) { + if (erts_smp_atomic_read(&ps->timeout) > (erts_aint_t) msec) { ERTS_POLLSET_SET_INTERRUPTED(ps); wake_poller(ps); } @@ -1228,7 +1228,7 @@ int erts_poll_wait(ErtsPollSet ps, erts_mtx_unlock(&w->mtx); } done: - erts_smp_atomic_set(&ps->timeout, LONG_MAX); + erts_smp_atomic_set(&ps->timeout, ERTS_AINT_T_MAX); *len = num; ERTS_POLLSET_UNLOCK(ps); HARDTRACEF(("Out erts_poll_wait")); @@ -1314,7 +1314,7 @@ ErtsPollSet erts_poll_create_pollset(void) erts_smp_mtx_init(&ps->mtx, "pollset"); erts_smp_atomic_init(&ps->interrupt, 0); #endif - erts_smp_atomic_init(&ps->timeout, LONG_MAX); + erts_smp_atomic_init(&ps->timeout, ERTS_AINT_T_MAX); HARDTRACEF(("Out erts_poll_create_pollset")); return ps; diff --git a/erts/emulator/sys/win32/erl_win_dyn_driver.h b/erts/emulator/sys/win32/erl_win_dyn_driver.h index 4949998abc..1347eead91 100644 --- a/erts/emulator/sys/win32/erl_win_dyn_driver.h +++ b/erts/emulator/sys/win32/erl_win_dyn_driver.h @@ -87,15 +87,15 @@ WDD_TYPEDEF(unsigned long, erts_alc_test, (unsigned long, unsigned long, unsigned long, unsigned long)); -WDD_TYPEDEF(long, driver_binary_get_refc, (ErlDrvBinary *dbp)); -WDD_TYPEDEF(long, driver_binary_inc_refc, (ErlDrvBinary *dbp)); -WDD_TYPEDEF(long, driver_binary_dec_refc, (ErlDrvBinary *dbp)); +WDD_TYPEDEF(ErlDrvSInt, driver_binary_get_refc, (ErlDrvBinary *dbp)); +WDD_TYPEDEF(ErlDrvSInt, driver_binary_inc_refc, (ErlDrvBinary *dbp)); +WDD_TYPEDEF(ErlDrvSInt, driver_binary_dec_refc, (ErlDrvBinary *dbp)); WDD_TYPEDEF(ErlDrvPDL, driver_pdl_create, (ErlDrvPort)); WDD_TYPEDEF(void, driver_pdl_lock, (ErlDrvPDL)); WDD_TYPEDEF(void, driver_pdl_unlock, (ErlDrvPDL)); -WDD_TYPEDEF(long, driver_pdl_get_refc, (ErlDrvPDL)); -WDD_TYPEDEF(long, driver_pdl_inc_refc, (ErlDrvPDL)); -WDD_TYPEDEF(long, driver_pdl_dec_refc, (ErlDrvPDL)); +WDD_TYPEDEF(ErlDrvSInt, driver_pdl_get_refc, (ErlDrvPDL)); +WDD_TYPEDEF(ErlDrvSInt, driver_pdl_inc_refc, (ErlDrvPDL)); +WDD_TYPEDEF(ErlDrvSInt, driver_pdl_dec_refc, (ErlDrvPDL)); WDD_TYPEDEF(void, driver_system_info, (ErlDrvSysInfo *, size_t)); WDD_TYPEDEF(int, driver_get_now, (ErlDrvNowData *)); WDD_TYPEDEF(int, driver_monitor_process, (ErlDrvPort port, diff --git a/erts/emulator/sys/win32/sys.c b/erts/emulator/sys/win32/sys.c index 39b04b26a9..37041ed987 100644 --- a/erts/emulator/sys/win32/sys.c +++ b/erts/emulator/sys/win32/sys.c @@ -67,14 +67,17 @@ static void async_read_file(struct async_io* aio, LPVOID buf, DWORD numToRead); static int async_write_file(struct async_io* aio, LPVOID buf, DWORD numToWrite); static int get_overlapped_result(struct async_io* aio, LPDWORD pBytesRead, BOOL wait); -static BOOL CreateChildProcess(char *, HANDLE, HANDLE, +static BOOL create_child_process(char *, HANDLE, HANDLE, HANDLE, LPHANDLE, BOOL, LPVOID, LPTSTR, unsigned, char **, int *); static int create_pipe(LPHANDLE, LPHANDLE, BOOL, BOOL); -static int ApplicationType(const char* originalName, char fullPath[MAX_PATH], +static int application_type(const char* originalName, char fullPath[MAX_PATH], BOOL search_in_path, BOOL handle_quotes, int *error_return); +static int application_type_w(const char* originalName, WCHAR fullPath[MAX_PATH], + BOOL search_in_path, BOOL handle_quotes, + int *error_return); HANDLE erts_service_event; @@ -87,7 +90,7 @@ static erts_smp_atomic_t pipe_creation_counter; static erts_smp_mtx_t sys_driver_data_lock; -/* Results from ApplicationType is one of */ +/* Results from application_type(_w) is one of */ #define APPL_NONE 0 #define APPL_DOS 1 #define APPL_WIN3X 2 @@ -1235,8 +1238,10 @@ spawn_start(ErlDrvPort port_num, char* name, SysDriverOpts* opts) */ DEBUGF(("Spawning \"%s\"\n", name)); - envir = win_build_environment(envir); - ok = CreateChildProcess(name, + envir = win_build_environment(envir); /* Still an ansi environment, could be + converted to unicode for spawn_executable, but + that is not done (yet) */ + ok = create_child_process(name, hChildStdin, hChildStdout, hChildStderr, @@ -1315,7 +1320,7 @@ create_file_thread(AsyncIo* aio, int mode) } /* - * A helper function used by CreateChildProcess(). + * A helper function used by create_child_process(). * Parses a command line with arguments and returns the length of the * first part containing the program name. * Example: input = "\"Program Files\"\\erl arg1 arg2" @@ -1356,24 +1361,25 @@ int parse_command(char* cmd){ return i; } -BOOL need_quotes(char *str) +static BOOL need_quotes(WCHAR *str) { int in_quote = 0; int backslashed = 0; int naked_space = 0; - while (*str != '\0') { + + while (*str != L'\0') { switch (*str) { - case '\\' : + case L'\\' : backslashed = !backslashed; break; - case '"': + case L'"': if (backslashed) { backslashed=0; } else { in_quote = !in_quote; } break; - case ' ': + case L' ': backslashed = 0; if (!(backslashed || in_quote)) { naked_space++; @@ -1392,7 +1398,7 @@ BOOL need_quotes(char *str) /* *---------------------------------------------------------------------- * - * CreateChildProcess -- + * create_child_process -- * * Create a child process that has pipes as its * standard input, output, and error. The child process runs @@ -1417,7 +1423,7 @@ BOOL need_quotes(char *str) */ static BOOL -CreateChildProcess +create_child_process ( char *origcmd, /* Command line for child process (including * name of executable). Or whole executable if st is @@ -1436,14 +1442,12 @@ CreateChildProcess ) { PROCESS_INFORMATION piProcInfo = {0}; - STARTUPINFO siStartInfo = {0}; BOOL ok = FALSE; int applType; /* Not to be changed for different types of executables */ int staticCreateFlags = GetPriorityClass(GetCurrentProcess()); int createFlags = DETACHED_PROCESS; char *newcmdline = NULL; - char execPath[MAX_PATH]; int cmdlength; char* thecommand; LPTSTR appname = NULL; @@ -1451,14 +1455,17 @@ CreateChildProcess *errno_return = -1; - siStartInfo.cb = sizeof(STARTUPINFO); - siStartInfo.dwFlags = STARTF_USESTDHANDLES; - siStartInfo.hStdInput = hStdin; - siStartInfo.hStdOutput = hStdout; - siStartInfo.hStdError = hStderr; - if (st != ERTS_SPAWN_EXECUTABLE) { + STARTUPINFO siStartInfo = {0}; + char execPath[MAX_PATH]; + + siStartInfo.cb = sizeof(STARTUPINFO); + siStartInfo.dwFlags = STARTF_USESTDHANDLES; + siStartInfo.hStdInput = hStdin; + siStartInfo.hStdOutput = hStdout; + siStartInfo.hStdError = hStderr; + /* * Parse out the program name from the command line (it can be quoted and * contain spaces). @@ -1470,9 +1477,9 @@ CreateChildProcess thecommand[cmdlength] = '\0'; DEBUGF(("spawn command: %s\n", thecommand)); - applType = ApplicationType(thecommand, execPath, TRUE, + applType = application_type(thecommand, execPath, TRUE, TRUE, errno_return); - DEBUGF(("ApplicationType returned for (%s) is %d\n", thecommand, applType)); + DEBUGF(("application_type returned for (%s) is %d\n", thecommand, applType)); erts_free(ERTS_ALC_T_TMP, (void *) thecommand); if (applType == APPL_NONE) { erts_free(ERTS_ALC_T_TMP,newcmdline); @@ -1501,126 +1508,147 @@ CreateChildProcess strcat(newcmdline, execPath); strcat(newcmdline, origcmd+cmdlength); - } else { /* ERTS_SPAWN_EXECUTABLE */ + DEBUGF(("Creating child process: %s, createFlags = %d\n", newcmdline, createFlags)); + ok = CreateProcessA(appname, + newcmdline, + NULL, + NULL, + TRUE, + createFlags | staticCreateFlags, + env, + wd, + &siStartInfo, + &piProcInfo); + + } else { /* ERTS_SPAWN_EXECUTABLE, filename and args are in unicode ({utf16,little}) */ int run_cmd = 0; - applType = ApplicationType(origcmd, execPath, FALSE, FALSE, - errno_return); + STARTUPINFOW siStartInfo = {0}; + WCHAR execPath[MAX_PATH]; + + + siStartInfo.cb = sizeof(STARTUPINFOW); + siStartInfo.dwFlags = STARTF_USESTDHANDLES; + siStartInfo.hStdInput = hStdin; + siStartInfo.hStdOutput = hStdout; + siStartInfo.hStdError = hStderr; + + applType = application_type_w(origcmd, (char *) execPath, FALSE, FALSE, + errno_return); if (applType == APPL_NONE) { return FALSE; } if (applType == APPL_DOS) { - /* - * See comment above - */ + /* + * See comment above + */ - siStartInfo.wShowWindow = SW_HIDE; - siStartInfo.dwFlags |= STARTF_USESHOWWINDOW; - createFlags = CREATE_NEW_CONSOLE; - run_cmd = 1; + siStartInfo.wShowWindow = SW_HIDE; + siStartInfo.dwFlags |= STARTF_USESHOWWINDOW; + createFlags = CREATE_NEW_CONSOLE; + run_cmd = 1; } else if (hide) { - DEBUGF(("hiding window\n")); - siStartInfo.wShowWindow = SW_HIDE; - siStartInfo.dwFlags |= STARTF_USESHOWWINDOW; - createFlags = 0; + DEBUGF(("hiding window\n")); + siStartInfo.wShowWindow = SW_HIDE; + siStartInfo.dwFlags |= STARTF_USESHOWWINDOW; + createFlags = 0; } if (run_cmd) { - char cmdPath[MAX_PATH]; + WCHAR cmdPath[MAX_PATH]; int cmdType; - cmdType = ApplicationType("cmd.exe", cmdPath, TRUE, FALSE, errno_return); + cmdType = application_type_w((char *) L"cmd.exe", (char *) cmdPath, TRUE, FALSE, errno_return); if (cmdType == APPL_NONE || cmdType == APPL_DOS) { return FALSE; } - appname = (char *) erts_alloc(ERTS_ALC_T_TMP, strlen(cmdPath)+1); - strcpy(appname,cmdPath); + appname = (char *) erts_alloc(ERTS_ALC_T_TMP, (wcslen(cmdPath)+1)*sizeof(WCHAR)); + wcscpy((WCHAR *) appname,cmdPath); } else { - appname = (char *) erts_alloc(ERTS_ALC_T_TMP, strlen(execPath)+1); - strcpy(appname,execPath); + appname = (char *) erts_alloc(ERTS_ALC_T_TMP, (wcslen(execPath)+1)*sizeof(WCHAR)); + wcscpy((WCHAR *) appname, execPath); } - if (argv == NULL) { + if (argv == NULL) { BOOL orig_need_q = need_quotes(execPath); - char *ptr; - int ocl = strlen(execPath); + WCHAR *ptr; + int ocl = wcslen(execPath); if (run_cmd) { newcmdline = (char *) erts_alloc(ERTS_ALC_T_TMP, - ocl + ((orig_need_q) ? 3 : 1) - + 11); - memcpy(newcmdline,"cmd.exe /c ",11); - ptr = newcmdline + 11; + (ocl + ((orig_need_q) ? 3 : 1) + + 11)*sizeof(WCHAR)); + memcpy(newcmdline,L"cmd.exe /c ",11*sizeof(WCHAR)); + ptr = (WCHAR *) (newcmdline + (11*sizeof(WCHAR))); } else { newcmdline = (char *) erts_alloc(ERTS_ALC_T_TMP, - ocl + ((orig_need_q) ? 3 : 1)); - ptr = newcmdline; + (ocl + ((orig_need_q) ? 3 : 1))*sizeof(WCHAR)); + ptr = (WCHAR *) newcmdline; } if (orig_need_q) { - *ptr++ = '"'; + *ptr++ = L'"'; } - memcpy(ptr,execPath,ocl); + memcpy(ptr,execPath,ocl*sizeof(WCHAR)); ptr += ocl; if (orig_need_q) { - *ptr++ = '"'; + *ptr++ = L'"'; } - *ptr = '\0'; + *ptr = L'\0'; } else { int sum = 1; /* '\0' */ - char **ar = argv; - char *n; + WCHAR **ar = (WCHAR **) argv; + WCHAR *n; char *save_arg0 = NULL; if (argv[0] == erts_default_arg0 || run_cmd) { save_arg0 = argv[0]; - argv[0] = execPath; + argv[0] = (char *) execPath; } if (run_cmd) { sum += 11; /* cmd.exe /c */ } while (*ar != NULL) { - sum += strlen(*ar); + sum += wcslen(*ar); if (need_quotes(*ar)) { sum += 2; /* quotes */ } sum++; /* space */ ++ar; } - ar = argv; - newcmdline = erts_alloc(ERTS_ALC_T_TMP, sum); - n = newcmdline; + ar = (WCHAR **) argv; + newcmdline = erts_alloc(ERTS_ALC_T_TMP, sum*sizeof(WCHAR)); + n = (WCHAR *) newcmdline; if (run_cmd) { - memcpy(n,"cmd.exe /c ",11); + memcpy(n,L"cmd.exe /c ",11*sizeof(WCHAR)); n += 11; } while (*ar != NULL) { int q = need_quotes(*ar); - sum = strlen(*ar); + sum = wcslen(*ar); if (q) { - *n++ = '"'; + *n++ = L'"'; } - memcpy(n,*ar,sum); + memcpy(n,*ar,sum*sizeof(WCHAR)); n += sum; if (q) { - *n++ = '"'; + *n++ = L'"'; } - *n++ = ' '; + *n++ = L' '; ++ar; } - ASSERT(n > newcmdline); - *(n-1) = '\0'; + *(n-1) = L'\0'; if (save_arg0 != NULL) { argv[0] = save_arg0; } } - } - DEBUGF(("Creating child process: %s, createFlags = %d\n", newcmdline, createFlags)); - ok = CreateProcess(appname, - newcmdline, - NULL, - NULL, - TRUE, - createFlags | staticCreateFlags, - env, - wd, - &siStartInfo, - &piProcInfo); - + DEBUGF(("Creating child process: %s, createFlags = %d\n", newcmdline, createFlags)); + ok = CreateProcessW((WCHAR *) appname, + (WCHAR *) newcmdline, + NULL, + NULL, + TRUE, + createFlags | staticCreateFlags, + env, + (WCHAR *) wd, + &siStartInfo, + &piProcInfo); + + } /* end SPAWN_EXECUTABLE */ if (newcmdline != NULL) { erts_free(ERTS_ALC_T_TMP,newcmdline); } @@ -1739,7 +1767,7 @@ static int create_pipe(HANDLE *phRead, HANDLE *phWrite, BOOL inheritRead, BOOL o -static int ApplicationType +static int application_type ( const char *originalName, /* Name of the application to find. */ char fullPath[MAX_PATH], /* Filled with complete path to @@ -1893,6 +1921,146 @@ static int ApplicationType return applType; } +static int application_type_w (const char *originalName, /* Name of the application to find. */ + WCHAR wfullpath[MAX_PATH],/* Filled with complete path to + * application. */ + BOOL search_in_path, /* If we should search the system wide path */ + BOOL handle_quotes, /* If we should handle quotes around executable */ + int *error_return) /* A place to put an error code */ +{ + int applType, i; + HANDLE hFile; + WCHAR *ext, *rest; + char buf[2]; + DWORD read; + IMAGE_DOS_HEADER header; + static WCHAR extensions[][5] = {L"", L".com", L".exe", L".bat"}; + int is_quoted; + int len; + WCHAR *wname = (WCHAR *) originalName; + WCHAR xfullpath[MAX_PATH]; + + len = wcslen(wname); + is_quoted = handle_quotes && len > 0 && wname[0] == L'"' && + wname[len-1] == L'"'; + + applType = APPL_NONE; + *error_return = ENOENT; + for (i = 0; i < (int) (sizeof(extensions) / sizeof(extensions[0])); i++) { + if(is_quoted) { + lstrcpynW(xfullpath, wname+1, MAX_PATH - 7); /* Cannot start using StringCchCopy yet, we support + older platforms */ + len = wcslen(xfullpath); + if(len > 0) { + xfullpath[len-1] = L'\0'; + } + } else { + lstrcpynW(xfullpath, wname, MAX_PATH - 5); + } + wcscat(xfullpath, extensions[i]); + /* It seems that the Unicode version does not allow in and out parameter to overlap. */ + SearchPathW((search_in_path) ? NULL : L".", xfullpath, NULL, MAX_PATH, wfullpath, &rest); + + /* + * Ignore matches on directories or data files, return if identified + * a known type. + */ + + if (GetFileAttributesW(wfullpath) & FILE_ATTRIBUTE_DIRECTORY) { + continue; + } + + ext = wcsrchr(wfullpath, L'.'); + if ((ext != NULL) && (_wcsicmp(ext, L".bat") == 0)) { + *error_return = EACCES; + applType = APPL_DOS; + break; + } + + hFile = CreateFileW(wfullpath, GENERIC_READ, FILE_SHARE_READ, NULL, + OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL); + if (hFile == INVALID_HANDLE_VALUE) { + continue; + } + + *error_return = EACCES; /* If considered an error, + it's an access error */ + header.e_magic = 0; + ReadFile(hFile, (void *) &header, sizeof(header), &read, NULL); + if (header.e_magic != IMAGE_DOS_SIGNATURE) { + /* + * Doesn't have the magic number for relocatable executables. If + * filename ends with .com, assume it's a DOS application anyhow. + * Note that we didn't make this assumption at first, because some + * supposed .com files are really 32-bit executables with all the + * magic numbers and everything. + */ + + CloseHandle(hFile); + if ((ext != NULL) && (_wcsicmp(ext, L".com") == 0)) { + applType = APPL_DOS; + break; + } + continue; + } + if (header.e_lfarlc != sizeof(header)) { + /* + * All Windows 3.X and Win32 and some DOS programs have this value + * set here. If it doesn't, assume that since it already had the + * other magic number it was a DOS application. + */ + + CloseHandle(hFile); + applType = APPL_DOS; + break; + } + + /* + * The DWORD at header.e_lfanew points to yet another magic number. + */ + + buf[0] = '\0'; + SetFilePointer(hFile, header.e_lfanew, NULL, FILE_BEGIN); + ReadFile(hFile, (void *) buf, 2, &read, NULL); + CloseHandle(hFile); + + if ((buf[0] == 'L') && (buf[1] == 'E')) { + applType = APPL_DOS; + } else if ((buf[0] == 'N') && (buf[1] == 'E')) { + applType = APPL_WIN3X; + } else if ((buf[0] == 'P') && (buf[1] == 'E')) { + applType = APPL_WIN32; + } else { + continue; + } + break; + } + + if (applType == APPL_NONE) { + return APPL_NONE; + } + + if ((applType == APPL_DOS) || (applType == APPL_WIN3X)) { + /* + * Replace long path name of executable with short path name for + * 16-bit applications. Otherwise the application may not be able + * to correctly parse its own command line to separate off the + * application name from the arguments. + */ + + GetShortPathNameW(wfullpath, wfullpath, MAX_PATH); + } + if (is_quoted) { + /* restore quotes on quoted program name */ + len = wcslen(wfullpath); + memmove(wfullpath+1,wfullpath,len*sizeof(WCHAR)); + wfullpath[0]=L'"'; + wfullpath[len+1]=L'"'; + wfullpath[len+2]=L'\0'; + } + return applType; +} + /* * Thread function used to emulate overlapped reading. */ diff --git a/erts/emulator/sys/win32/sys_interrupt.c b/erts/emulator/sys/win32/sys_interrupt.c index d2449a1bdb..262f84babc 100644 --- a/erts/emulator/sys/win32/sys_interrupt.c +++ b/erts/emulator/sys/win32/sys_interrupt.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1997-2009. All Rights Reserved. + * Copyright Ericsson AB 1997-2010. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -33,9 +33,9 @@ #ifdef ERTS_SMP erts_smp_atomic_t erts_break_requested; #define ERTS_SET_BREAK_REQUESTED \ - erts_smp_atomic_set(&erts_break_requested, (long) 1) + erts_smp_atomic_set(&erts_break_requested, (erts_aint_t) 1) #define ERTS_UNSET_BREAK_REQUESTED \ - erts_smp_atomic_set(&erts_break_requested, (long) 0) + erts_smp_atomic_set(&erts_break_requested, (erts_aint_t) 0) #else volatile int erts_break_requested = 0; #define ERTS_SET_BREAK_REQUESTED (erts_break_requested = 1) diff --git a/erts/emulator/test/Makefile b/erts/emulator/test/Makefile index a4c02da626..7259e1b84d 100644 --- a/erts/emulator/test/Makefile +++ b/erts/emulator/test/Makefile @@ -83,6 +83,7 @@ MODULES= \ receive_SUITE \ ref_SUITE \ register_SUITE \ + mtx_SUITE \ save_calls_SUITE \ send_term_SUITE \ sensitive_SUITE \ diff --git a/erts/emulator/test/mtx_SUITE.erl b/erts/emulator/test/mtx_SUITE.erl new file mode 100644 index 0000000000..ae77fe4d89 --- /dev/null +++ b/erts/emulator/test/mtx_SUITE.erl @@ -0,0 +1,473 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +%% +%% Stress tests of rwmutex implementation. +%% +%% Author: Rickard Green +%% +-module(mtx_SUITE). + +%%-define(line_trace,true). + +-include("test_server.hrl"). + +-export([all/1, init_per_suite/1, end_per_suite/1, init_per_testcase/2, fin_per_testcase/2]). + +-export([long_rwlock/1, + hammer_ets_rwlock/1, + hammer_rwlock/1, + hammer_rwlock_check/1, + hammer_tryrwlock/1, + hammer_tryrwlock_check/1, + hammer_sched_long_rwlock/1, + hammer_sched_long_rwlock_check/1, + hammer_sched_long_freqread_rwlock/1, + hammer_sched_long_freqread_rwlock_check/1, + hammer_sched_long_tryrwlock/1, + hammer_sched_long_tryrwlock_check/1, + hammer_sched_long_freqread_tryrwlock/1, + hammer_sched_long_freqread_tryrwlock_check/1, + hammer_sched_rwlock/1, + hammer_sched_rwlock_check/1, + hammer_sched_freqread_rwlock/1, + hammer_sched_freqread_rwlock_check/1, + hammer_sched_tryrwlock/1, + hammer_sched_tryrwlock_check/1, + hammer_sched_freqread_tryrwlock/1, + hammer_sched_freqread_tryrwlock_check/1]). + +init_per_suite(Config) when is_list(Config) -> + DataDir = ?config(data_dir, Config), + Lib = filename:join([DataDir, atom_to_list(?MODULE)]), + ok = erlang:load_nif(Lib, none), + Config. + +end_per_suite(Config) when is_list(Config) -> + Config. + +init_per_testcase(_Case, Config) -> + Dog = ?t:timetrap(?t:minutes(15)), + [{watchdog, Dog}|Config]. + +fin_per_testcase(_Func, Config) -> + Dog = ?config(watchdog, Config), + ?t:timetrap_cancel(Dog). + +all(suite) -> + [ + long_rwlock, + hammer_rwlock_check, + hammer_rwlock, + hammer_tryrwlock_check, + hammer_tryrwlock, + hammer_ets_rwlock, + hammer_sched_long_rwlock_check, + hammer_sched_long_rwlock, + hammer_sched_long_freqread_rwlock_check, + hammer_sched_long_freqread_rwlock, + hammer_sched_long_tryrwlock_check, + hammer_sched_long_tryrwlock, + hammer_sched_long_freqread_tryrwlock_check, + hammer_sched_long_freqread_tryrwlock, + hammer_sched_rwlock_check, + hammer_sched_rwlock, + hammer_sched_freqread_rwlock_check, + hammer_sched_freqread_rwlock, + hammer_sched_tryrwlock_check, + hammer_sched_tryrwlock, + hammer_sched_freqread_tryrwlock_check, + hammer_sched_freqread_tryrwlock + ]. + +long_rwlock(Config) when is_list(Config) -> + statistics(runtime), + LLRes = long_rw_test(), + {_, RunTime} = statistics(runtime), + %% A very short run time is expected, since + %% threads in the test mostly wait + ?t:format("RunTime=~p~n", [RunTime]), + ?line true = RunTime < 100, + ?line RunTimeStr = "Run-time during test was "++integer_to_list(RunTime)++" ms.", + case LLRes of + ok -> + {comment, RunTimeStr}; + {comment, Comment} -> + {comment, Comment ++ " " ++ RunTimeStr} + end. + +hammer_rwlock(Config) when is_list(Config) -> + hammer_rw_test(false). + +hammer_rwlock_check(Config) when is_list(Config) -> + hammer_rw_test(true). + +hammer_tryrwlock(Config) when is_list(Config) -> + hammer_tryrw_test(false). + +hammer_tryrwlock_check(Config) when is_list(Config) -> + hammer_tryrw_test(true). + +hammer_sched_rwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, false, true, 0, 0). + +hammer_sched_rwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, true, true, 0, 0). + +hammer_sched_freqread_rwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, false, true, 0, 0). + +hammer_sched_freqread_rwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, true, true, 0, 0). + +hammer_sched_tryrwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, false, false, 0, 100). + +hammer_sched_tryrwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, true, false, 0, 100). + +hammer_sched_freqread_tryrwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, false, false, 0, 100). + +hammer_sched_freqread_tryrwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, true, false, 0, 100). + +hammer_sched_long_rwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, false, true, 100, 0). + +hammer_sched_long_rwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, true, true, 100, 0). + +hammer_sched_long_freqread_rwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, false, true, 100, 0). + +hammer_sched_long_freqread_rwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, true, true, 100, 0). + +hammer_sched_long_tryrwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, false, false, 100, 100). + +hammer_sched_long_tryrwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(false, true, false, 100, 100). + +hammer_sched_long_freqread_tryrwlock(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, false, false, 100, 100). + +hammer_sched_long_freqread_tryrwlock_check(Config) when is_list(Config) -> + hammer_sched_rwlock_test(true, true, false, 100, 100). + +hammer_sched_rwlock_test(FreqRead, LockCheck, Blocking, WaitLocked, WaitUnlocked) -> + case create_rwlock(FreqRead, LockCheck) of + enotsup -> + {skipped, "Not supported."}; + RWLock -> + Onln = erlang:system_info(schedulers_online), + NWPs = case Onln div 3 of + 1 -> case Onln < 4 of + true -> 1; + false -> 2 + end; + X -> X + end, + NRPs = Onln - NWPs, + NoLockOps = ((((50000000 div Onln) + div case {Blocking, WaitLocked} of + {false, 0} -> 1; + _ -> 10 + end) + div (case WaitLocked == 0 of + true -> 1; + false -> WaitLocked*250 + end)) + div handicap()), + ?t:format("NoLockOps=~p~n", [NoLockOps]), + Sleep = case Blocking of + true -> NoLockOps; + false -> NoLockOps div 10 + end, + WPs = lists:map( + fun (Sched) -> + spawn_opt( + fun () -> + io:format("Writer on scheduler ~p.~n", + [Sched]), + Sched = erlang:system_info(scheduler_id), + receive go -> gone end, + hammer_sched_rwlock_proc(RWLock, + Blocking, + true, + WaitLocked, + WaitUnlocked, + NoLockOps, + Sleep), + Sched = erlang:system_info(scheduler_id) + end, + [link, {scheduler, Sched}]) + end, + lists:seq(1, NWPs)), + RPs = lists:map( + fun (Sched) -> + spawn_opt( + fun () -> + io:format("Reader on scheduler ~p.~n", + [Sched]), + Sched = erlang:system_info(scheduler_id), + receive go -> gone end, + hammer_sched_rwlock_proc(RWLock, + Blocking, + false, + WaitLocked, + WaitUnlocked, + NoLockOps, + Sleep), + Sched = erlang:system_info(scheduler_id) + end, + [link, {scheduler, Sched}]) + end, + lists:seq(NWPs + 1, NWPs + NRPs)), + Procs = WPs ++ RPs, + case {Blocking, WaitLocked} of + {_, 0} -> ok; + {false, _} -> ok; + _ -> statistics(runtime) + end, + lists:foreach(fun (P) -> P ! go end, Procs), + lists:foreach(fun (P) -> + M = erlang:monitor(process, P), + receive + {'DOWN', M, process, P, _} -> + ok + end + end, + Procs), + case {Blocking, WaitLocked} of + {_, 0} -> ok; + {false, _} -> ok; + _ -> + {_, RunTime} = statistics(runtime), + ?t:format("RunTime=~p~n", [RunTime]), + ?line true = RunTime < 500, + {comment, + "Run-time during test was " + ++ integer_to_list(RunTime) + ++ " ms."} + end + end. + +hammer_sched_rwlock_proc(_RWLock, + _Blocking, + _WriteOp, + _WaitLocked, + _WaitUnlocked, + 0, + _Sleep) -> + ok; +hammer_sched_rwlock_proc(RWLock, + Blocking, + WriteOp, + WaitLocked, + WaitUnlocked, + Times, + Sleep) when Times rem Sleep == 0 -> + rwlock_op(RWLock, Blocking, WriteOp, WaitLocked, WaitUnlocked), + hammer_sched_rwlock_proc(RWLock, + Blocking, + WriteOp, + WaitLocked, + WaitUnlocked, + Times - 1, + Sleep); +hammer_sched_rwlock_proc(RWLock, + Blocking, + WriteOp, + WaitLocked, + WaitUnlocked, + Times, + Sleep) -> + rwlock_op(RWLock, Blocking, WriteOp, WaitLocked, 0), + hammer_sched_rwlock_proc(RWLock, + Blocking, + WriteOp, + WaitLocked, + WaitUnlocked, + Times - 1, + Sleep). + +-define(HAMMER_ETS_RWLOCK_REPEAT_TIMES, 1). +-define(HAMMER_ETS_RWLOCK_TSIZE, 500). + +hammer_ets_rwlock(Config) when is_list(Config) -> + {Ops, Procs} = case handicap() of + 1 -> {20000, 500}; + 2 -> {20000, 50}; + 3 -> {2000, 50}; + _ -> {200, 50} + end, + ?t:format("Procs=~p~nOps=~p~n", [Procs, Ops]), + lists:foreach(fun (XOpts) -> + ?t:format("Running with extra opts: ~p", [XOpts]), + hammer_ets_rwlock_test(XOpts, true, 2, Ops, + Procs, false) + end, + [[], + [{read_concurrency, true}], + [{write_concurrency, true}], + [{read_concurrency, true},{write_concurrency, true}]]), + ok. + +%% Aux funcs + +long_rw_test() -> + exit(no_nif_implementation). + +hammer_rw_test(_Arg) -> + exit(no_nif_implementation). + +hammer_tryrw_test(_Arg) -> + exit(no_nif_implementation). + +create_rwlock(_FreqRead, _LockCheck) -> + exit(no_nif_implementation). + +rwlock_op(_RWLock, _Blocking, _WriteOp, _WaitLocked, _WaitUnlocked) -> + exit(no_nif_implementation). + +hammer_ets_rwlock_put_data() -> + put(?MODULE, {"here are some", data, "to store", make_ref()}). + +hammer_ets_rwlock_get_data() -> + get(?MODULE). + +hammer_ets_rwlock_ops(_T, _UW, _N, _C, _SC, 0) -> + ok; +hammer_ets_rwlock_ops(T, UW, N, C, SC, Tot) when N >= ?HAMMER_ETS_RWLOCK_TSIZE -> + hammer_ets_rwlock_ops(T, UW, 0, C, SC, Tot); +hammer_ets_rwlock_ops(T, UW, N, 0, SC, Tot) -> + case UW of + true -> + true = ets:insert(T, {N, Tot, hammer_ets_rwlock_get_data()}); + false -> + [{N, _, _}] = ets:lookup(T, N) + end, + hammer_ets_rwlock_ops(T, UW, N+1, SC, SC, Tot-1); +hammer_ets_rwlock_ops(T, UW, N, C, SC, Tot) -> + case UW of + false -> + true = ets:insert(T, {N, Tot, hammer_ets_rwlock_get_data()}); + true -> + [{N, _, _}] = ets:lookup(T, N) + end, + hammer_ets_rwlock_ops(T, UW, N+1, C-1, SC, Tot-1). + +hammer_ets_rwlock_init(T, N) when N < ?HAMMER_ETS_RWLOCK_TSIZE -> + ets:insert(T, {N, N, N}), + hammer_ets_rwlock_init(T, N+1); +hammer_ets_rwlock_init(_T, _N) -> + ok. + +hammer_ets_rwlock_test(XOpts, UW, C, N, NP, SC) -> + receive after 100 -> ok end, + {TP, TM} = spawn_monitor( + fun () -> + _L = repeat_list( + fun () -> + Caller = self(), + T = fun () -> + Parent = self(), + hammer_ets_rwlock_put_data(), + T=ets:new(x, [public | XOpts]), + hammer_ets_rwlock_init(T, 0), + Ps0 = repeat_list( + fun () -> + spawn_link( + fun () -> + hammer_ets_rwlock_put_data(), + receive go -> ok end, + hammer_ets_rwlock_ops(T, UW, N, C, C, N), + Parent ! {done, self()}, + receive after infinity -> ok end + end) + end, + NP - case SC of + false -> 0; + _ -> 1 + end), + Ps = case SC of + false -> Ps0; + _ -> [spawn_link(fun () -> + hammer_ets_rwlock_put_data(), + receive go -> ok end, + hammer_ets_rwlock_ops(T, UW, N, SC, SC, N), + Parent ! {done, self()}, + receive after infinity -> ok end + end) | Ps0] + end, + Start = now(), + lists:foreach(fun (P) -> P ! go end, Ps), + lists:foreach(fun (P) -> receive {done, P} -> ok end end, Ps), + Stop = now(), + lists:foreach(fun (P) -> + unlink(P), + exit(P, bang), + M = erlang:monitor(process, P), + receive + {'DOWN', M, process, P, _} -> ok + end + end, Ps), + Res = timer:now_diff(Stop, Start)/1000000, + Caller ! {?MODULE, self(), Res} + end, + TP = spawn_link(T), + receive + {?MODULE, TP, Res} -> + Res + end + end, + ?HAMMER_ETS_RWLOCK_REPEAT_TIMES) + end), + receive + {'DOWN', TM, process, TP, _} -> ok + end. + +repeat_list(Fun, N) -> + repeat_list(Fun, N, []). + +repeat_list(_Fun, 0, Acc) -> + Acc; +repeat_list(Fun, N, Acc) -> + repeat_list(Fun, N-1, [Fun()|Acc]). + + +handicap() -> + X0 = case catch (erlang:system_info(logical_processors_available) >= + erlang:system_info(schedulers_online)) of + true -> 1; + _ -> 2 + end, + case erlang:system_info(build_type) of + opt -> + X0; + ReallySlow when ReallySlow == debug; + ReallySlow == valgrind; + ReallySlow == purify -> + X0*3; + _Slow -> + X0*2 + end. + diff --git a/erts/emulator/test/mtx_SUITE_data/Makefile.src b/erts/emulator/test/mtx_SUITE_data/Makefile.src new file mode 100644 index 0000000000..b6c843269c --- /dev/null +++ b/erts/emulator/test/mtx_SUITE_data/Makefile.src @@ -0,0 +1,30 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 2010. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# + +include @erts_lib_include_internal_generated@@[email protected] +include @erts_lib_include_internal_generated@@DS@erts_internal.mk + +NIF_LIBS = mtx_SUITE@dll@ + +SHLIB_EXTRA_CFLAGS = $(ETHR_DEFS) -I@erts_lib_include_internal@ -I@erts_lib_include_internal_generated@ +LIBS = @ERTS_LIBS@ + +all: $(NIF_LIBS) + +@SHLIB_RULES@ diff --git a/erts/emulator/test/mtx_SUITE_data/mtx_SUITE.c b/erts/emulator/test/mtx_SUITE_data/mtx_SUITE.c new file mode 100644 index 0000000000..818023211c --- /dev/null +++ b/erts/emulator/test/mtx_SUITE_data/mtx_SUITE.c @@ -0,0 +1,692 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2010. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +/* + * Stress tests of rwmutex implementation. + * + * Author: Rickard Green + */ + +#include "erl_nif.h" + +#ifdef __WIN32__ +# ifndef WIN32_LEAN_AND_MEAN +# define WIN32_LEAN_AND_MEAN +# endif +# include <windows.h> +#else +# include "ethread.h" +# include "erl_misc_utils.h" +# include <unistd.h> +#endif + +#include <errno.h> +#include <stdio.h> + +static int +fail(const char *file, int line, const char *function, const char *assertion); + +#undef ASSERT +#define ASSERT(X) ((void) ((X) ? 1 : fail(__FILE__, __LINE__, __func__, #X))) + +#ifdef __WIN32__ +/* + * We cannot access the ethread symbols directly; test + * what we got in the nif api instead... + */ +#define HAVE_FREQREAD_SUPPORT 0 +#define RWMUTEX_T ErlNifRWLock +#define RWMUTEX_CREATE(FR) enif_rwlock_create("dummy") +#define RWMUTEX_DESTROY enif_rwlock_destroy +#define RWMUTEX_WLOCK enif_rwlock_rwlock +#define RWMUTEX_TRYWLOCK enif_rwlock_tryrwlock +#define RWMUTEX_WUNLOCK enif_rwlock_rwunlock +#define RWMUTEX_TRYRLOCK enif_rwlock_tryrlock +#define RWMUTEX_RLOCK enif_rwlock_rlock +#define RWMUTEX_RUNLOCK enif_rwlock_runlock +#define THR_ID ErlNifTid +#define THR_CREATE(A, B, C, D) enif_thread_create("dummy", (A), (B), (C), (D)) +#define THR_JOIN enif_thread_join +#define ATOMIC_T volatile LONG +#define ATOMIC_INIT(VarP, Val) (*(VarP) = (Val)) +#define ATOMIC_SET(VarP, Val) (*(VarP) = (Val)) +#define ATOMIC_READ(VarP) (*(VarP)) +#define ATOMIC_INC InterlockedIncrement +#define ATOMIC_DEC InterlockedDecrement + +#else + +#ifdef ETHR_USE_OWN_RWMTX_IMPL__ +# define HAVE_FREQREAD_SUPPORT 1 +#else +# define HAVE_FREQREAD_SUPPORT 0 +#endif + +#define RWMUTEX_T ethr_rwmutex +static ethr_rwmutex * +RWMUTEX_CREATE(int freqread) +{ + ethr_rwmutex *rwmtx = enif_alloc(sizeof(ethr_rwmutex)); + ethr_rwmutex_opt rwmtx_opt = ETHR_RWMUTEX_OPT_DEFAULT_INITER; + if (freqread) + rwmtx_opt.type = ETHR_RWMUTEX_TYPE_FREQUENT_READ; + ASSERT(rwmtx); + ASSERT(ethr_rwmutex_init_opt(rwmtx, &rwmtx_opt) == 0); + return rwmtx; +} +static void +RWMUTEX_DESTROY(ethr_rwmutex *rwmtx) +{ + ASSERT(ethr_rwmutex_destroy(rwmtx) == 0); + enif_free(rwmtx); +} +#define RWMUTEX_TRYWLOCK ethr_rwmutex_tryrwlock +#define RWMUTEX_WLOCK ethr_rwmutex_rwlock +#define RWMUTEX_WUNLOCK ethr_rwmutex_rwunlock +#define RWMUTEX_TRYRLOCK ethr_rwmutex_tryrlock +#define RWMUTEX_RLOCK ethr_rwmutex_rlock +#define RWMUTEX_RUNLOCK ethr_rwmutex_runlock +#define THR_ID ethr_tid +#define THR_CREATE ethr_thr_create +#define THR_JOIN ethr_thr_join +#define ATOMIC_T ethr_atomic_t +#define ATOMIC_INIT ethr_atomic_init +#define ATOMIC_SET ethr_atomic_set +#define ATOMIC_READ ethr_atomic_read +#define ATOMIC_INC ethr_atomic_inc +#define ATOMIC_DEC ethr_atomic_dec + +#endif + + +#if !defined(__func__) +# if !defined(__STDC_VERSION__) || __STDC_VERSION__ < 199901L +# if !defined(__GNUC__) || __GNUC__ < 2 +# define __func__ "[unknown_function]" +# else +# define __func__ __FUNCTION__ +# endif +# endif +#endif + +static void milli_sleep(int ms); +static int get_bool(ErlNifEnv* env, ERL_NIF_TERM term); + +/* + * Long rwlock testcase + */ + +#define LONG_RW_NO_W_THREADS 6 +#define LONG_RW_NO_THREADS 20 +#define LONG_RW_NO_WLOCK_COUNT 100 + +typedef struct { + RWMUTEX_T *rwlock; + ATOMIC_T *is_wlocked; + ATOMIC_T *is_rlocked; + int *stop; + int *count; + int sleep; +} long_rw_t; + +static void * +long_rw_w(void *varg) +{ + long_rw_t *arg = varg; + int stop = 0; + do { + RWMUTEX_WLOCK(arg->rwlock); + ASSERT(!ATOMIC_READ(arg->is_wlocked)); + ATOMIC_SET(arg->is_wlocked, 1); + ASSERT(!ATOMIC_READ(arg->is_rlocked)); + milli_sleep(arg->sleep); + if (++(*arg->count) > LONG_RW_NO_WLOCK_COUNT) + stop = *arg->stop = 1; + ATOMIC_SET(arg->is_wlocked, 0); + ASSERT(!ATOMIC_READ(arg->is_rlocked)); + RWMUTEX_WUNLOCK(arg->rwlock); + } while (!stop); + return NULL; +} + +static void * +long_rw_r(void *varg) +{ + long_rw_t *arg = varg; + int stop; + do { + RWMUTEX_RLOCK(arg->rwlock); + ASSERT(!ATOMIC_READ(arg->is_wlocked)); + ATOMIC_INC(arg->is_rlocked); + milli_sleep(arg->sleep); + stop = *arg->stop; + ATOMIC_DEC(arg->is_rlocked); + ASSERT(!ATOMIC_READ(arg->is_wlocked)); + RWMUTEX_RUNLOCK(arg->rwlock); + } while (!stop); + return NULL; +} + + +static ERL_NIF_TERM long_rw_test(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) +{ + int res, freqread, i, count, stop; + ATOMIC_T is_wlocked, is_rlocked; + THR_ID tid[LONG_RW_NO_THREADS]; + long_rw_t arg; + long_rw_t targ[LONG_RW_NO_THREADS]; + + ATOMIC_INIT(&is_wlocked, 0); + ATOMIC_INIT(&is_rlocked, 0); + + freqread = 0; + + arg.is_wlocked = &is_wlocked; + arg.is_rlocked = &is_rlocked; + arg.count = &count; + arg.stop = &stop; + + restart: + + stop = 0; + count = 0; + + arg.rwlock = RWMUTEX_CREATE(freqread); + + ASSERT(arg.rwlock); + + for (i = 0; i < LONG_RW_NO_W_THREADS; i++) { + targ[i] = arg; + targ[i].sleep = 100 + i*10; + ASSERT(THR_CREATE(&tid[i], long_rw_w, &targ[i], NULL) == 0); + } + for (; i < LONG_RW_NO_THREADS; i++) { + targ[i] = arg; + targ[i].sleep = 100; + ASSERT(THR_CREATE(&tid[i], long_rw_r, &targ[i], NULL) == 0); + } + for (i = 0; i < LONG_RW_NO_THREADS; i++) + ASSERT(THR_JOIN(tid[i], NULL) == 0); + + ASSERT(!ATOMIC_READ(arg.is_wlocked)); + ASSERT(!ATOMIC_READ(arg.is_rlocked)); + + RWMUTEX_DESTROY(arg.rwlock); + + if (HAVE_FREQREAD_SUPPORT && !freqread) { + freqread = 1; + goto restart; + } + + if (freqread) + return enif_make_atom(env, "ok"); + else + return enif_make_tuple2(env, + enif_make_atom(env, + "comment"), + enif_make_string(env, + "No frequent read test made.", + ERL_NIF_LATIN1)); +} + +/* + * Hammer rwlock testcase + */ + +#define HAMMER_RW_NO_W_THREADS 6 +#define HAMMER_RW_NO_THREADS 20 +#define HAMMER_RW_NO_WLOCK_COUNT 1000000 + +typedef struct { + RWMUTEX_T *rwlock; + ATOMIC_T is_locked; + int lock_check; + int stop; + int count; +} hammer_rw_t; + +static void * +hammer_rw_w(void *varg) +{ + hammer_rw_t *arg = varg; + int stop = 0; + do { + RWMUTEX_WLOCK(arg->rwlock); + if (arg->lock_check) { + ASSERT(!ATOMIC_READ(&arg->is_locked)); + ATOMIC_SET(&arg->is_locked, -1); + } + if (++arg->count > HAMMER_RW_NO_WLOCK_COUNT) + stop = arg->stop = 1; + if (arg->lock_check) { + ASSERT(ATOMIC_READ(&arg->is_locked) == -1); + ATOMIC_SET(&arg->is_locked, 0); + } + RWMUTEX_WUNLOCK(arg->rwlock); + } while (!stop); + return NULL; +} + +static void * +hammer_rw_r(void *varg) +{ + hammer_rw_t *arg = varg; + int stop; + do { + RWMUTEX_RLOCK(arg->rwlock); + if (arg->lock_check) { + ASSERT(ATOMIC_READ(&arg->is_locked) >= 0); + ATOMIC_INC(&arg->is_locked); + } + stop = arg->stop; + if (arg->lock_check) { + ASSERT(ATOMIC_READ(&arg->is_locked) > 0); + ATOMIC_DEC(&arg->is_locked); + } + RWMUTEX_RUNLOCK(arg->rwlock); + } while (!stop); + return NULL; +} + + +static ERL_NIF_TERM hammer_rw_test(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) +{ + hammer_rw_t arg; + char buf[10]; + int res, freqread, i; + THR_ID tid[HAMMER_RW_NO_THREADS]; + + if (argc != 1) + goto badarg; + + arg.lock_check = get_bool(env, argv[0]); + if (arg.lock_check < 0) + goto badarg; + + ATOMIC_INIT(&arg.is_locked, 0); + + freqread = 0; + + restart: + arg.stop = 0; + arg.count = 0; + + arg.rwlock = RWMUTEX_CREATE(freqread); + + ASSERT(arg.rwlock); + + for (i = 0; i < HAMMER_RW_NO_W_THREADS; i++) + ASSERT(THR_CREATE(&tid[i], hammer_rw_w, &arg, NULL) == 0); + for (; i < HAMMER_RW_NO_THREADS; i++) + ASSERT(THR_CREATE(&tid[i], hammer_rw_r, &arg, NULL) == 0); + for (i = 0; i < HAMMER_RW_NO_THREADS; i++) + ASSERT(THR_JOIN(tid[i], NULL) == 0); + + ASSERT(!ATOMIC_READ(&arg.is_locked)); + + RWMUTEX_DESTROY(arg.rwlock); + + if (HAVE_FREQREAD_SUPPORT && !freqread) { + freqread = 1; + goto restart; + } + + if (freqread) + return enif_make_atom(env, "ok"); + else + return enif_make_tuple2(env, + enif_make_atom(env, + "comment"), + enif_make_string(env, + "No frequent read test made.", + ERL_NIF_LATIN1)); + badarg: + return enif_make_badarg(env); +} + +/* + * Hammer try rwlock testcase + */ + +#define HAMMER_TRYRW_NO_W_THREADS 10 +#define HAMMER_TRYRW_NO_THREADS 20 +#define HAMMER_TRYRW_NO_WLOCK_COUNT 10000000 +#define HAMMER_TRYRW_NO_RLOCK_COUNT 10000000 +#define HAMMER_TRYRW_NO_WLOCK_WAIT_COUNT ((10*HAMMER_TRYRW_NO_WLOCK_COUNT)/8) +#define HAMMER_TRYRW_NO_RLOCK_WAIT_COUNT ((10*HAMMER_TRYRW_NO_RLOCK_COUNT)/8) + +typedef struct { + RWMUTEX_T *rwlock; + ATOMIC_T is_locked; + int lock_check; + int w_count; + ATOMIC_T r_count; +} hammer_tryrw_t; + +static void * +hammer_tryrw_w(void *varg) +{ + hammer_tryrw_t *arg = varg; + int stop = 0; + int wait = 0; + do { + while (EBUSY == RWMUTEX_TRYWLOCK(arg->rwlock)); + if (arg->lock_check) { + ASSERT(!ATOMIC_READ(&arg->is_locked)); + ATOMIC_SET(&arg->is_locked, -1); + } + if (++arg->w_count > HAMMER_TRYRW_NO_WLOCK_COUNT) + stop = 1; + else if (arg->w_count > HAMMER_TRYRW_NO_RLOCK_WAIT_COUNT) + wait = 1; + if (arg->lock_check) { + ASSERT(ATOMIC_READ(&arg->is_locked) == -1); + ATOMIC_SET(&arg->is_locked, 0); + } + RWMUTEX_WUNLOCK(arg->rwlock); + if (wait) + milli_sleep(1); + } while (!stop); + return NULL; +} + +static void * +hammer_tryrw_r(void *varg) +{ + hammer_tryrw_t *arg = varg; + long r_count; + int stop = 0; + int wait = 0; + do { + while (EBUSY == RWMUTEX_TRYRLOCK(arg->rwlock)); + if (arg->lock_check) { + ASSERT(ATOMIC_READ(&arg->is_locked) >= 0); + ATOMIC_INC(&arg->is_locked); + } + ATOMIC_INC(&arg->r_count); + r_count = ATOMIC_READ(&arg->r_count); + if (r_count > HAMMER_TRYRW_NO_RLOCK_COUNT) + stop = 1; + else if (r_count > HAMMER_TRYRW_NO_RLOCK_WAIT_COUNT) + wait = 1; + if (arg->lock_check) { + ASSERT(ATOMIC_READ(&arg->is_locked) > 0); + ATOMIC_DEC(&arg->is_locked); + } + RWMUTEX_RUNLOCK(arg->rwlock); + if (wait) + milli_sleep(1); + } while (!stop); + return NULL; +} + + +static ERL_NIF_TERM hammer_tryrw_test(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) +{ + hammer_tryrw_t arg; + char buf[10]; + int res, freqread, i; + THR_ID tid[HAMMER_TRYRW_NO_THREADS]; + + if (argc != 1) + goto badarg; + + arg.lock_check = get_bool(env, argv[0]); + if (arg.lock_check < 0) + goto badarg; + + ATOMIC_INIT(&arg.is_locked, 0); + freqread = 0; + + restart: + + arg.w_count = 0; + ATOMIC_INIT(&arg.r_count, 0); + + arg.rwlock = RWMUTEX_CREATE(freqread); + + ASSERT(arg.rwlock); + + for (i = 0; i < HAMMER_TRYRW_NO_W_THREADS; i++) + ASSERT(THR_CREATE(&tid[i], hammer_tryrw_w, &arg, NULL) == 0); + for (; i < HAMMER_TRYRW_NO_THREADS; i++) + ASSERT(THR_CREATE(&tid[i], hammer_tryrw_r, &arg, NULL) == 0); + for (i = 0; i < HAMMER_TRYRW_NO_THREADS; i++) + ASSERT(THR_JOIN(tid[i], NULL) == 0); + + ASSERT(!ATOMIC_READ(&arg.is_locked)); + + RWMUTEX_DESTROY(arg.rwlock); + + if (HAVE_FREQREAD_SUPPORT && !freqread) { + freqread = 1; + goto restart; + } + + if (freqread) + return enif_make_atom(env, "ok"); + else + return enif_make_tuple2(env, + enif_make_atom(env, + "comment"), + enif_make_string(env, + "No frequent read test made.", + ERL_NIF_LATIN1)); + badarg: + return enif_make_badarg(env); +} + +typedef struct { + int lock_check; + ATOMIC_T is_locked; + RWMUTEX_T *rwlock; +} rwlock_resource_t; + +static void +rwlock_destructor(ErlNifEnv* env, void* obj) +{ + rwlock_resource_t *rwlr = obj; + if (rwlr->lock_check) + ASSERT(!ATOMIC_READ(&rwlr->is_locked)); + RWMUTEX_DESTROY(rwlr->rwlock); +} + +/* + * create_rwlock(FreqRead, LockCheck) + */ + +static ERL_NIF_TERM +create_rwlock(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) +{ + int lock_check, freqread; + ERL_NIF_TERM rwlock_term; + rwlock_resource_t *rwlr; + char buf[100]; + + if (argc != 2) + goto badarg; + + freqread = get_bool(env, argv[0]); + if (freqread < 0) + goto badarg; + + if (!HAVE_FREQREAD_SUPPORT && freqread) + return enif_make_atom(env, "enotsup"); + + lock_check = get_bool(env, argv[1]); + if (lock_check < 0) + goto badarg; + + rwlr = enif_alloc_resource(enif_priv_data(env), sizeof(rwlock_resource_t)); + rwlr->lock_check = lock_check; + ATOMIC_INIT(&rwlr->is_locked, 0); + rwlr->rwlock = RWMUTEX_CREATE(freqread); + rwlock_term = enif_make_resource(env, rwlr); + enif_release_resource(rwlr); + return rwlock_term; + + badarg: + return enif_make_badarg(env); +} + +/* + * rwlock_op(RWLock, Blocking, WriteOp, WaitTime) + */ + +static ERL_NIF_TERM +rwlock_op(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) +{ + rwlock_resource_t *rwlr; + int blocking, write, wait_locked, wait_unlocked; + + if (argc != 5) + goto badarg; + + if (!enif_get_resource(env, argv[0], enif_priv_data(env), (void **) &rwlr)) + goto badarg; + + blocking = get_bool(env, argv[1]); + if (blocking < 0) + goto badarg; + + write = get_bool(env, argv[2]); + if (write < 0) + goto badarg; + + if (!enif_get_int(env, argv[3], &wait_locked)) + goto badarg; + if (wait_locked < 0) + goto badarg; + + if (!enif_get_int(env, argv[4], &wait_unlocked)) + goto badarg; + if (wait_unlocked < 0) + goto badarg; + + if (write) { + if (blocking) + RWMUTEX_WLOCK(rwlr->rwlock); + else + while (EBUSY == RWMUTEX_TRYWLOCK(rwlr->rwlock)); + if (rwlr->lock_check) { + ASSERT(!ATOMIC_READ(&rwlr->is_locked)); + ATOMIC_SET(&rwlr->is_locked, -1); + } + } + else { + if (blocking) + RWMUTEX_RLOCK(rwlr->rwlock); + else + while (EBUSY == RWMUTEX_TRYRLOCK(rwlr->rwlock)); + if (rwlr->lock_check) { + ASSERT(ATOMIC_READ(&rwlr->is_locked) >= 0); + ATOMIC_INC(&rwlr->is_locked); + } + } + + if (wait_locked) + milli_sleep(wait_locked); + + if (write) { + if (rwlr->lock_check) { + ASSERT(ATOMIC_READ(&rwlr->is_locked) == -1); + ATOMIC_SET(&rwlr->is_locked, 0); + } + RWMUTEX_WUNLOCK(rwlr->rwlock); + } + else { + if (rwlr->lock_check) { + ASSERT(ATOMIC_READ(&rwlr->is_locked) > 0); + ATOMIC_DEC(&rwlr->is_locked); + } + RWMUTEX_RUNLOCK(rwlr->rwlock); + } + + if (wait_unlocked) + milli_sleep(wait_unlocked); + + return enif_make_atom(env, "ok"); + badarg: + return enif_make_badarg(env); +} + +static int load_nif_lib(ErlNifEnv* env, void** priv_data, ERL_NIF_TERM load_info) +{ + *priv_data = enif_open_resource_type(env, + NULL, + "rwlock_resource", + rwlock_destructor, + ERL_NIF_RT_CREATE, + NULL); + if (*priv_data) + return 0; + else + return -1; +} + +/* + * 0 -> false + * >0 -> true + * <0 -> error + */ + +static int +get_bool(ErlNifEnv* env, ERL_NIF_TERM term) +{ + int res; + char buf[10]; + + res = enif_get_atom(env, term, buf, sizeof(buf), ERL_NIF_LATIN1); + if (res == 0) + return -1; + if (strcmp("false", buf) == 0) + return 0; + else if (strcmp("true", buf) == 0) + return 1; + else + return -1; +} + +static int +fail(const char *file, int line, const char *function, const char *assertion) +{ + fprintf(stderr, "%s:%d: Assertion failed in %s(): %s\n", + file, line, function, assertion); + abort(); +} + +static void +milli_sleep(int ms) +{ +#ifdef __WIN32__ + Sleep(ms); +#else + while (erts_milli_sleep(ms) != 0); +#endif +} + +static ErlNifFunc nif_funcs[] = { + {"long_rw_test", 0, long_rw_test}, + {"hammer_rw_test", 1, hammer_rw_test}, + {"hammer_tryrw_test", 1, hammer_tryrw_test}, + {"create_rwlock", 2, create_rwlock}, + {"rwlock_op", 5, rwlock_op} +}; + +ERL_NIF_INIT(mtx_SUITE, nif_funcs, load_nif_lib, NULL, NULL, NULL) diff --git a/erts/emulator/zlib/zutil.h b/erts/emulator/zlib/zutil.h index d560382691..a8872e1c88 100644 --- a/erts/emulator/zlib/zutil.h +++ b/erts/emulator/zlib/zutil.h @@ -142,6 +142,7 @@ extern const char * const z_errmsg[10]; /* indexed by 2-zlib_error */ #ifdef WIN32 # ifndef __CYGWIN__ /* Cygwin is Unix, not Win32 */ # define OS_CODE 0x0b +# define F_OPEN(name, mode) _wfopen((WCHAR *)(name), (WCHAR *)(mode)) /* Unicode */ # endif #endif diff --git a/erts/etc/common/inet_gethost.c b/erts/etc/common/inet_gethost.c index e095836258..8bd9368aa1 100644 --- a/erts/etc/common/inet_gethost.c +++ b/erts/etc/common/inet_gethost.c @@ -65,10 +65,8 @@ #include <stdlib.h> /* These are not used even if they would exist which they should not */ -#undef HAVE_GETADDRINFO #undef HAVE_GETIPNODEBYNAME #undef HAVE_GETHOSTBYNAME2 -#undef HAVE_GETNAMEINFO #undef HAVE_GETIPNODEBYADDR #else /* Unix */ @@ -1762,7 +1760,7 @@ static int worker_loop(void) struct addrinfo hints; memset(&hints, 0, sizeof(hints)); - hints.ai_flags = (AI_CANONNAME|AI_V4MAPPED|AI_ADDRCONFIG); + hints.ai_flags = AI_CANONNAME; hints.ai_socktype = SOCK_STREAM; hints.ai_family = AF_INET6; DEBUGF(5, ("Starting getaddrinfo(%s, ...)", data)); diff --git a/erts/etc/unix/format_man_pages b/erts/etc/unix/format_man_pages index 2c4f6eee4f..93dcdcd8fa 100644 --- a/erts/etc/unix/format_man_pages +++ b/erts/etc/unix/format_man_pages @@ -3,7 +3,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 1996-2009. All Rights Reserved. +# Copyright Ericsson AB 1996-2010. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -59,34 +59,21 @@ esac # Create the 'cat' directories (probably not needed) # -cd $ERL_ROOT +cd $ERL_ROOT/man -if [ ! -d man/cat1 ] -then - mkdir man/cat1 -fi +for d in 0 1 2 3 4 5 6 7 8 9 +do + if [ ! -d cat$d ] + then + mkdir cat$d + fi -if [ ! -d man/cat3 ] -then - mkdir man/cat3 -fi - -if [ ! -d man/cat4 ] -then - mkdir man/cat4 -fi - -if [ ! -d man/cat6 ] -then - mkdir man/cat6 -fi +done # # Cleanup old formatting # -cd $ERL_ROOT/man - rm -f whatis windex # Remove old cat files diff --git a/erts/etc/win32/nsis/Makefile b/erts/etc/win32/nsis/Makefile index ebb3ad9a96..981a232c69 100644 --- a/erts/etc/win32/nsis/Makefile +++ b/erts/etc/win32/nsis/Makefile @@ -45,6 +45,7 @@ WTARGET_DIR=$(shell (cygpath -d $(TARGET_DIR) 2>/dev/null || cygpath -d $(TARGET REDIST_FILE=$(shell (sh ./find_redist.sh || echo "")) REDIST_DLL_VERSION=$(shell (sh ./dll_version_helper.sh || echo "")) +REDIST_DLL_NAME=$(shell (sh ./dll_version_helper.sh -n || echo "")) release_spec: @NSIS_VER=`makensis /hdrinfo | head -1 | awk '{print $$2}'`; \ @@ -73,6 +74,7 @@ release_spec: cp $(REDIST_FILE) $(RELEASE_PATH)/vcredist_x86.exe;\ echo '!define HAVE_REDIST_FILE 1' >> $(VERSION_HEADER); \ echo '!define REDIST_DLL_VERSION "$(REDIST_DLL_VERSION)"' >> $(VERSION_HEADER);\ + echo '!define REDIST_DLL_NAME "$(REDIST_DLL_NAME)"' >> $(VERSION_HEADER);\ fi;\ if [ -f $(RELEASE_PATH)/docs/doc/index.html ];\ then \ diff --git a/erts/etc/win32/nsis/dll_version_helper.sh b/erts/etc/win32/nsis/dll_version_helper.sh index e0047dea8b..571ee3e39e 100755 --- a/erts/etc/win32/nsis/dll_version_helper.sh +++ b/erts/etc/win32/nsis/dll_version_helper.sh @@ -2,7 +2,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2007-2009. All Rights Reserved. +# Copyright Ericsson AB 2007-2010. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -41,9 +41,15 @@ if [ '!' -f hello.exe.manifest ]; then exit 0 fi VERSION=`grep '<assemblyIdentity' hello.exe.manifest | sed 's,.*version=.\([0-9\.]*\).*,\1,g' | grep -v '<'` +NAME=`grep '<assemblyIdentity' hello.exe.manifest | sed 's,.*name=.[A-Za-z\.]*\([0-9]*\).*,msvcr\1.dll,g' | grep -v '<'` rm -f hello.c hello.obj hello.exe hello.exe.manifest -if [ -z "$VERSION" ]; then +if [ "$1" = "-n" ]; then + ASKEDFOR=$NAME +else + ASKEDFOR=$VERSION +fi +if [ -z "$ASKEDFOR" ]; then exit 1 fi -echo $VERSION +echo $ASKEDFOR exit 0 diff --git a/erts/etc/win32/nsis/erlang20.nsi b/erts/etc/win32/nsis/erlang20.nsi index 43e5d91604..941e8e6f5d 100644 --- a/erts/etc/win32/nsis/erlang20.nsi +++ b/erts/etc/win32/nsis/erlang20.nsi @@ -311,23 +311,23 @@ FunctionEnd Function .onInit
SectionGetFlags 0 $MYTEMP
-; MessageBox MB_YESNO "Found $SYSDIR\msvcr80.dll" IDYES FoundLbl
- IfFileExists $SYSDIR\msvcr80.dll MaybeFoundInSystemLbl
+ ;MessageBox MB_YESNO "Found $SYSDIR\${REDIST_DLL_NAME}" IDYES FoundLbl
+ IfFileExists $SYSDIR\${REDIST_DLL_NAME} MaybeFoundInSystemLbl
SearchSxsLbl:
FindFirst $0 $1 $WINDIR\WinSxS\x86*
LoopLbl:
StrCmp $1 "" NotFoundLbl
- IfFileExists $WINDIR\WinSxS\$1\msvcr80.dll MaybeFoundInSxsLbl
+ IfFileExists $WINDIR\WinSxS\$1\${REDIST_DLL_NAME} MaybeFoundInSxsLbl
FindNext $0 $1
Goto LoopLbl
MaybeFoundInSxsLbl:
- GetDllVersion $WINDIR\WinSxS\$1\msvcr80.dll $R0 $R1
+ GetDllVersion $WINDIR\WinSxS\$1\${REDIST_DLL_NAME} $R0 $R1
Call DllVersionGoodEnough
FindNext $0 $1
IntCmp 2 $R0 LoopLbl
Goto FoundLbl
MaybeFoundInSystemLbl:
- GetDllVersion $SYSDIR\msvcr80.dll $R0 $R1
+ GetDllVersion $SYSDIR\${REDIST_DLL_NAME} $R0 $R1
Call DllVersionGoodEnough
IntCmp 2 $R0 SearchSxSLbl
FoundLbl:
diff --git a/erts/etc/win32/nsis/find_redist.sh b/erts/etc/win32/nsis/find_redist.sh index c5572839c5..153977ded5 100755 --- a/erts/etc/win32/nsis/find_redist.sh +++ b/erts/etc/win32/nsis/find_redist.sh @@ -2,7 +2,7 @@ # # %CopyrightBegin% # -# Copyright Ericsson AB 2007-2009. All Rights Reserved. +# Copyright Ericsson AB 2007-2010. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in @@ -107,16 +107,56 @@ for x in cl bin vc; do fi BPATH="$NBPATH" done +BPATH_LIST=$BPATH + +# rc.exe is in the Microsoft SDK directory of VS2008 +RCPATH=`lookup_prog_in_path rc` +fail=false +if [ '!' -z "$RCPATH" ]; then + BPATH=$RCPATH + for x in rc bin v6.0A ; do + NBPATH=`remove_path_element $x "$BPATH"` + if [ "$NBPATH" = "$BPATH" ]; then + fail=true + break; + fi + BPATH="$NBPATH" + done + if [ $fail = false ]; then + BPATH_LIST="$BPATH_LIST $BPATH" + fi +fi + +# Frantic search through two roots with different +# version directories. We want to be very specific about the +# directory structures as we woildnt want to find the wrong +# redistributables... + #echo $BPATH -for x in sdk v2.0 bootstrapper packages vcredist_x86 vcredist_x86.exe; do - #echo "x=$x" - #echo "BPATH=$BPATH" - NBPATH=`add_path_element $x "$BPATH"` - if [ "$NBPATH" = "$BPATH" ]; then - echo "Failed to locate vcredist_x86.exe because directory structure was unexpected" >&2 - exit 3 +for BP in $BPATH_LIST; do + for verdir in "sdk v2.0" "sdk v3.5" "v6.0A"; do + BPATH=$BP + fail=false + for x in $verdir bootstrapper packages vcredist_x86 vcredist_x86.exe; do + #echo "x=$x" + #echo "BPATH=$BPATH" + NBPATH=`add_path_element $x "$BPATH"` + if [ "$NBPATH" = "$BPATH" ]; then + fail=true + break; + fi + BPATH="$NBPATH" + done + if [ $fail = false ]; then + break; + fi + done + if [ $fail = false ]; then + echo $BPATH + exit 0 fi - BPATH="$NBPATH" done -echo $BPATH -exit 0
\ No newline at end of file + +echo "Failed to locate vcredist_x86.exe because directory structure was unexpected" >&2 +exit 3 + diff --git a/erts/include/internal/ethr_atomics.h b/erts/include/internal/ethr_atomics.h new file mode 100644 index 0000000000..1caf4d0567 --- /dev/null +++ b/erts/include/internal/ethr_atomics.h @@ -0,0 +1,726 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2010. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +/* + * Description: The ethread atomic API + * Author: Rickard Green + */ + +#ifndef ETHR_ATOMIC_H__ +#define ETHR_ATOMIC_H__ + +#if !defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_ATOMIC_IMPL__) +# define ETHR_NEED_ATOMIC_PROTOTYPES__ +#endif + +#ifndef ETHR_HAVE_NATIVE_ATOMICS +/* + * No native atomic implementation available. :( + * Use fallback... + */ +typedef ethr_sint32_t ethr_atomic32_t; +typedef ethr_sint_t ethr_atomic_t; +#else +/* + * Map ethread native atomics to ethread API atomics. + * + * We do at least have a native atomic implementation that + * can handle integers of a size larger than or equal to + * the size of pointers. + */ + +/* -- Pointer size atomics -- */ + +#undef ETHR_NAINT_T__ +#undef ETHR_NATMC_FUNC__ +#undef ETHR_NATMC_ADDR_FUNC__ +#if ETHR_SIZEOF_PTR == 8 +# if defined(ETHR_HAVE_NATIVE_ATOMIC64) +# define ETHR_NATMC_ADDR_FUNC__ ethr_native_atomic64_addr +typedef ethr_native_atomic64_t ethr_atomic_t; +# define ETHR_NAINT_T__ ethr_sint64_t +# define ETHR_NATMC_FUNC__(X) ethr_native_atomic64_ ## X +# else +# error "Missing native atomic implementation" +# endif +#elif ETHR_SIZEOF_PTR == 4 +# define ETHR_NATMC_ADDR_FUNC__ ethr_native_atomic32_addr +# ifdef ETHR_HAVE_NATIVE_ATOMIC32 +typedef ethr_native_atomic32_t ethr_atomic_t; +# define ETHR_NAINT_T__ ethr_sint32_t +# define ETHR_NATMC_FUNC__(X) ethr_native_atomic32_ ## X +# elif defined(ETHR_HAVE_NATIVE_ATOMIC64) +typedef ethr_native_atomic64_t ethr_atomic_t; +# define ETHR_NATMC_T__ ethr_native_atomic64_t +# define ETHR_NAINT_T__ ethr_sint64_t +# define ETHR_NATMC_FUNC__(X) ethr_native_atomic64_ ## X +# else +# error "Missing native atomic implementation" +# endif +#endif + +/* -- 32-bit atomics -- */ + +#undef ETHR_NAINT32_T__ +#undef ETHR_NATMC32_FUNC__ +#if defined(ETHR_HAVE_NATIVE_ATOMIC32) +typedef ethr_native_atomic32_t ethr_atomic32_t; +# define ETHR_NAINT32_T__ ethr_sint32_t +# define ETHR_NATMC32_FUNC__(X) ethr_native_atomic32_ ## X +#elif defined(ETHR_HAVE_NATIVE_ATOMIC64) +typedef ethr_native_atomic64_t ethr_atomic32_t; +# define ETHR_NAINT32_T__ ethr_sint64_t +# define ETHR_NATMC32_FUNC__(X) ethr_native_atomic64_ ## X +#else +# error "Missing native atomic implementation" +#endif + +#endif + +#ifdef ETHR_NEED_ATOMIC_PROTOTYPES__ +ethr_sint_t *ethr_atomic_addr(ethr_atomic_t *); +void ethr_atomic_init(ethr_atomic_t *, ethr_sint_t); +void ethr_atomic_set(ethr_atomic_t *, ethr_sint_t); +ethr_sint_t ethr_atomic_read(ethr_atomic_t *); +ethr_sint_t ethr_atomic_inc_read(ethr_atomic_t *); +ethr_sint_t ethr_atomic_dec_read(ethr_atomic_t *); +void ethr_atomic_inc(ethr_atomic_t *); +void ethr_atomic_dec(ethr_atomic_t *); +ethr_sint_t ethr_atomic_add_read(ethr_atomic_t *, ethr_sint_t); +void ethr_atomic_add(ethr_atomic_t *, ethr_sint_t); +ethr_sint_t ethr_atomic_read_band(ethr_atomic_t *, ethr_sint_t); +ethr_sint_t ethr_atomic_read_bor(ethr_atomic_t *, ethr_sint_t); +ethr_sint_t ethr_atomic_xchg(ethr_atomic_t *, ethr_sint_t); +ethr_sint_t ethr_atomic_cmpxchg(ethr_atomic_t *, ethr_sint_t, ethr_sint_t); +ethr_sint_t ethr_atomic_read_acqb(ethr_atomic_t *); +ethr_sint_t ethr_atomic_inc_read_acqb(ethr_atomic_t *); +void ethr_atomic_set_relb(ethr_atomic_t *, ethr_sint_t); +void ethr_atomic_dec_relb(ethr_atomic_t *); +ethr_sint_t ethr_atomic_dec_read_relb(ethr_atomic_t *); +ethr_sint_t ethr_atomic_cmpxchg_acqb(ethr_atomic_t *, ethr_sint_t, ethr_sint_t); +ethr_sint_t ethr_atomic_cmpxchg_relb(ethr_atomic_t *, ethr_sint_t, ethr_sint_t); + +ethr_sint32_t *ethr_atomic32_addr(ethr_atomic32_t *); +void ethr_atomic32_init(ethr_atomic32_t *, ethr_sint32_t); +void ethr_atomic32_set(ethr_atomic32_t *, ethr_sint32_t); +ethr_sint32_t ethr_atomic32_read(ethr_atomic32_t *); +ethr_sint32_t ethr_atomic32_inc_read(ethr_atomic32_t *); +ethr_sint32_t ethr_atomic32_dec_read(ethr_atomic32_t *); +void ethr_atomic32_inc(ethr_atomic32_t *); +void ethr_atomic32_dec(ethr_atomic32_t *); +ethr_sint32_t ethr_atomic32_add_read(ethr_atomic32_t *, ethr_sint32_t); +void ethr_atomic32_add(ethr_atomic32_t *, ethr_sint32_t); +ethr_sint32_t ethr_atomic32_read_band(ethr_atomic32_t *, ethr_sint32_t); +ethr_sint32_t ethr_atomic32_read_bor(ethr_atomic32_t *, ethr_sint32_t); +ethr_sint32_t ethr_atomic32_xchg(ethr_atomic32_t *, ethr_sint32_t); +ethr_sint32_t ethr_atomic32_cmpxchg(ethr_atomic32_t *, + ethr_sint32_t, + ethr_sint32_t); +ethr_sint32_t ethr_atomic32_read_acqb(ethr_atomic32_t *); +ethr_sint32_t ethr_atomic32_inc_read_acqb(ethr_atomic32_t *); +void ethr_atomic32_set_relb(ethr_atomic32_t *, ethr_sint32_t); +void ethr_atomic32_dec_relb(ethr_atomic32_t *); +ethr_sint32_t ethr_atomic32_dec_read_relb(ethr_atomic32_t *); +ethr_sint32_t ethr_atomic32_cmpxchg_acqb(ethr_atomic32_t *, + ethr_sint32_t, + ethr_sint32_t); +ethr_sint32_t ethr_atomic32_cmpxchg_relb(ethr_atomic32_t *, + ethr_sint32_t, + ethr_sint32_t); +#endif + +int ethr_init_atomics(void); + +#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_ATOMIC_IMPL__) + +#ifndef ETHR_HAVE_NATIVE_ATOMICS +/* + * Fallbacks for atomics used in absence of a native implementation. + */ + +#define ETHR_ATOMIC_ADDR_BITS 10 +#define ETHR_ATOMIC_ADDR_SHIFT 6 + +typedef struct { + union { + ethr_spinlock_t lck; + char buf[ETHR_CACHE_LINE_SIZE]; + } u; +} ethr_atomic_protection_t; + +extern ethr_atomic_protection_t ethr_atomic_protection__[1 << ETHR_ATOMIC_ADDR_BITS]; + +#define ETHR_ATOMIC_PTR2LCK__(PTR) \ +(ðr_atomic_protection__[((((ethr_uint_t) (PTR)) >> ETHR_ATOMIC_ADDR_SHIFT) \ + & ((1 << ETHR_ATOMIC_ADDR_BITS) - 1))].u.lck) + + +#define ETHR_ATOMIC_OP_FALLBACK_IMPL__(AP, EXPS) \ +do { \ + ethr_spinlock_t *slp__ = ETHR_ATOMIC_PTR2LCK__((AP)); \ + ethr_spin_lock(slp__); \ + { EXPS; } \ + ethr_spin_unlock(slp__); \ +} while (0) + +#endif + +/* + * --- Pointer size atomics --------------------------------------------------- + */ + +static ETHR_INLINE ethr_sint_t * +ETHR_INLINE_FUNC_NAME_(ethr_atomic_addr)(ethr_atomic_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t *) ETHR_NATMC_ADDR_FUNC__(var); +#else + return (ethr_sint_t *) var; +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic_init)(ethr_atomic_t *var, ethr_sint_t i) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC_FUNC__(init)(var, (ETHR_NAINT_T__) i); +#else + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, *var = i); +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic_set)(ethr_atomic_t *var, ethr_sint_t i) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC_FUNC__(set)(var, (ETHR_NAINT_T__) i); +#else + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, *var = i); +#endif +} + +static ETHR_INLINE ethr_sint_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic_read)(ethr_atomic_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t) ETHR_NATMC_FUNC__(read)(var); +#else + ethr_sint_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = *var); + return res; +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic_add)(ethr_atomic_t *var, ethr_sint_t incr) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC_FUNC__(add)(var, (ETHR_NAINT_T__) incr); +#else + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, *var += incr); +#endif +} + +static ETHR_INLINE ethr_sint_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic_add_read)(ethr_atomic_t *var, ethr_sint_t i) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t) ETHR_NATMC_FUNC__(add_return)(var, (ETHR_NAINT_T__) i); +#else + ethr_sint_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, *var += i; res = *var); + return res; +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic_inc)(ethr_atomic_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC_FUNC__(inc)(var); +#else + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, ++(*var)); +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic_dec)(ethr_atomic_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC_FUNC__(dec)(var); +#else + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, --(*var)); +#endif +} + +static ETHR_INLINE ethr_sint_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic_inc_read)(ethr_atomic_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t) ETHR_NATMC_FUNC__(inc_return)(var); +#else + ethr_sint_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = ++(*var)); + return res; +#endif +} + +static ETHR_INLINE ethr_sint_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic_dec_read)(ethr_atomic_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t) ETHR_NATMC_FUNC__(dec_return)(var); +#else + ethr_sint_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = --(*var)); + return res; +#endif +} + +static ETHR_INLINE ethr_sint_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic_read_band)(ethr_atomic_t *var, + ethr_sint_t mask) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t) ETHR_NATMC_FUNC__(and_retold)(var, + (ETHR_NAINT_T__) mask); +#else + ethr_sint_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = *var; *var &= mask); + return res; +#endif +} + +static ETHR_INLINE ethr_sint_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic_read_bor)(ethr_atomic_t *var, + ethr_sint_t mask) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t) ETHR_NATMC_FUNC__(or_retold)(var, + (ETHR_NAINT_T__) mask); +#else + ethr_sint_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = *var; *var |= mask); + return res; +#endif +} + +static ETHR_INLINE ethr_sint_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic_xchg)(ethr_atomic_t *var, ethr_sint_t new) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t) ETHR_NATMC_FUNC__(xchg)(var, + (ETHR_NAINT_T__) new); +#else + ethr_sint_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = *var; *var = new); + return res; +#endif +} + +static ETHR_INLINE ethr_sint_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic_cmpxchg)(ethr_atomic_t *var, + ethr_sint_t new, + ethr_sint_t exp) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t) ETHR_NATMC_FUNC__(cmpxchg)(var, + (ETHR_NAINT_T__) new, + (ETHR_NAINT_T__) exp); +#else + ethr_sint_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, + { + res = *var; + if (__builtin_expect(res == exp, 1)) + *var = new; + }); + return res; +#endif +} + +/* + * Important memory barrier requirements. + * + * The following atomic operations *must* supply a memory barrier of + * at least the type specified by its suffix: + * _acqb = acquire barrier + * _relb = release barrier + */ + +static ETHR_INLINE ethr_sint_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic_read_acqb)(ethr_atomic_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t) ETHR_NATMC_FUNC__(read_acqb)(var); +#else + return ETHR_INLINE_FUNC_NAME_(ethr_atomic_read)(var); +#endif +} + +static ETHR_INLINE ethr_sint_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic_inc_read_acqb)(ethr_atomic_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t) ETHR_NATMC_FUNC__(inc_return_acqb)(var); +#else + return ETHR_INLINE_FUNC_NAME_(ethr_atomic_inc_read)(var); +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic_set_relb)(ethr_atomic_t *var, + ethr_sint_t val) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC_FUNC__(set_relb)(var, (ETHR_NAINT_T__) val); +#else + ETHR_INLINE_FUNC_NAME_(ethr_atomic_set)(var, val); +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic_dec_relb)(ethr_atomic_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC_FUNC__(dec_relb)(var); +#else + ETHR_INLINE_FUNC_NAME_(ethr_atomic_dec)(var); +#endif +} + +static ETHR_INLINE ethr_sint_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic_dec_read_relb)(ethr_atomic_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t) ETHR_NATMC_FUNC__(dec_return_relb)(var); +#else + return ETHR_INLINE_FUNC_NAME_(ethr_atomic_dec_read)(var); +#endif +} + +static ETHR_INLINE ethr_sint_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic_cmpxchg_acqb)(ethr_atomic_t *var, + ethr_sint_t new, + ethr_sint_t exp) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t) ETHR_NATMC_FUNC__(cmpxchg_acqb)(var, + (ETHR_NAINT_T__) new, + (ETHR_NAINT_T__) exp); +#else + return ETHR_INLINE_FUNC_NAME_(ethr_atomic_cmpxchg)(var, new, exp); +#endif +} + +static ETHR_INLINE ethr_sint_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic_cmpxchg_relb)(ethr_atomic_t *var, + ethr_sint_t new, + ethr_sint_t exp) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint_t) ETHR_NATMC_FUNC__(cmpxchg_relb)(var, + (ETHR_NAINT_T__) new, + (ETHR_NAINT_T__) exp); +#else + return ETHR_INLINE_FUNC_NAME_(ethr_atomic_cmpxchg)(var, new, exp); +#endif +} + +/* + * --- 32-bit atomics --------------------------------------------------------- + */ + +static ETHR_INLINE ethr_sint32_t * +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_addr)(ethr_atomic32_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return ethr_native_atomic32_addr(var); +#else + return (ethr_sint32_t *) var; +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_init)(ethr_atomic32_t *var, + ethr_sint32_t i) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC32_FUNC__(init)(var, (ETHR_NAINT32_T__) i); +#else + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, *var = i); +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_set)(ethr_atomic32_t *var, ethr_sint32_t i) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC32_FUNC__(set)(var, (ETHR_NAINT32_T__) i); +#else + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, *var = i); +#endif +} + +static ETHR_INLINE ethr_sint32_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_read)(ethr_atomic32_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint32_t) ETHR_NATMC32_FUNC__(read)(var); +#else + ethr_sint32_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = *var); + return res; +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_add)(ethr_atomic32_t *var, + ethr_sint32_t incr) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC32_FUNC__(add)(var, (ETHR_NAINT32_T__) incr); +#else + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, *var += incr); +#endif +} + +static ETHR_INLINE ethr_sint32_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_add_read)(ethr_atomic32_t *var, + ethr_sint32_t i) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint32_t) + ETHR_NATMC32_FUNC__(add_return)(var, (ETHR_NAINT32_T__) i); +#else + ethr_sint32_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, *var += i; res = *var); + return res; +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_inc)(ethr_atomic32_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC32_FUNC__(inc)(var); +#else + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, ++(*var)); +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_dec)(ethr_atomic32_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC32_FUNC__(dec)(var); +#else + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, --(*var)); +#endif +} + +static ETHR_INLINE ethr_sint32_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_inc_read)(ethr_atomic32_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint32_t) ETHR_NATMC32_FUNC__(inc_return)(var); +#else + ethr_sint32_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = ++(*var)); + return res; +#endif +} + +static ETHR_INLINE ethr_sint32_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_dec_read)(ethr_atomic32_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint32_t) ETHR_NATMC32_FUNC__(dec_return)(var); +#else + ethr_sint32_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = --(*var)); + return res; +#endif +} + +static ETHR_INLINE ethr_sint32_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_read_band)(ethr_atomic32_t *var, + ethr_sint32_t mask) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint32_t) + ETHR_NATMC32_FUNC__(and_retold)(var, (ETHR_NAINT32_T__) mask); +#else + ethr_sint32_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = *var; *var &= mask); + return res; +#endif +} + +static ETHR_INLINE ethr_sint32_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_read_bor)(ethr_atomic32_t *var, + ethr_sint32_t mask) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return + (ethr_sint32_t) ETHR_NATMC32_FUNC__(or_retold)(var, + (ETHR_NAINT32_T__) mask); +#else + ethr_sint32_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = *var; *var |= mask); + return res; +#endif +} + +static ETHR_INLINE ethr_sint32_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_xchg)(ethr_atomic32_t *var, + ethr_sint32_t new) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint32_t) ETHR_NATMC32_FUNC__(xchg)(var, + (ETHR_NAINT32_T__) new); +#else + ethr_sint32_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = *var; *var = new); + return res; +#endif +} + +static ETHR_INLINE ethr_sint32_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_cmpxchg)(ethr_atomic32_t *var, + ethr_sint32_t new, + ethr_sint32_t exp) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint32_t) ETHR_NATMC32_FUNC__(cmpxchg)(var, + (ETHR_NAINT32_T__) new, + (ETHR_NAINT32_T__) exp); +#else + ethr_sint32_t res; + ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, + { + res = *var; + if (__builtin_expect(res == exp, 1)) + *var = new; + }); + return res; +#endif +} + +/* + * Important memory barrier requirements. + * + * The following atomic operations *must* supply a memory barrier of + * at least the type specified by its suffix: + * _acqb = acquire barrier + * _relb = release barrier + */ + +static ETHR_INLINE ethr_sint32_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_read_acqb)(ethr_atomic32_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint32_t) ETHR_NATMC32_FUNC__(read_acqb)(var); +#else + return ETHR_INLINE_FUNC_NAME_(ethr_atomic32_read)(var); +#endif +} + +static ETHR_INLINE ethr_sint32_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_inc_read_acqb)(ethr_atomic32_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint32_t) ETHR_NATMC32_FUNC__(inc_return_acqb)(var); +#else + return ETHR_INLINE_FUNC_NAME_(ethr_atomic32_inc_read)(var); +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_set_relb)(ethr_atomic32_t *var, + ethr_sint32_t val) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC32_FUNC__(set_relb)(var, (ETHR_NAINT32_T__) val); +#else + ETHR_INLINE_FUNC_NAME_(ethr_atomic32_set)(var, val); +#endif +} + +static ETHR_INLINE void +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_dec_relb)(ethr_atomic32_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + ETHR_NATMC32_FUNC__(dec_relb)(var); +#else + ETHR_INLINE_FUNC_NAME_(ethr_atomic32_dec)(var); +#endif +} + +static ETHR_INLINE ethr_sint32_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_dec_read_relb)(ethr_atomic32_t *var) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint32_t) ETHR_NATMC32_FUNC__(dec_return_relb)(var); +#else + return ETHR_INLINE_FUNC_NAME_(ethr_atomic32_dec_read)(var); +#endif +} + +static ETHR_INLINE ethr_sint32_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_cmpxchg_acqb)(ethr_atomic32_t *var, + ethr_sint32_t new, + ethr_sint32_t exp) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint32_t) + ETHR_NATMC32_FUNC__(cmpxchg_acqb)(var, + (ETHR_NAINT32_T__) new, + (ETHR_NAINT32_T__) exp); +#else + return ETHR_INLINE_FUNC_NAME_(ethr_atomic32_cmpxchg)(var, new, exp); +#endif +} + +static ETHR_INLINE ethr_sint32_t +ETHR_INLINE_FUNC_NAME_(ethr_atomic32_cmpxchg_relb)(ethr_atomic32_t *var, + ethr_sint32_t new, + ethr_sint32_t exp) +{ +#ifdef ETHR_HAVE_NATIVE_ATOMICS + return (ethr_sint32_t) + ETHR_NATMC32_FUNC__(cmpxchg_relb)(var, + (ETHR_NAINT32_T__) new, + (ETHR_NAINT32_T__) exp); +#else + return ETHR_INLINE_FUNC_NAME_(ethr_atomic32_cmpxchg)(var, new, exp); +#endif +} + + +#endif /* ETHR_TRY_INLINE_FUNCS */ + +#undef ETHR_NAINT_T__ +#undef ETHR_NATMC_FUNC__ +#undef ETHR_NATMC_ADDR_FUNC__ + +#undef ETHR_NAINT32_T__ +#undef ETHR_NATMC32_FUNC__ + +#endif diff --git a/erts/include/internal/ethr_mutex.h b/erts/include/internal/ethr_mutex.h index 8d9d5e3d08..fadaf1e2a4 100644 --- a/erts/include/internal/ethr_mutex.h +++ b/erts/include/internal/ethr_mutex.h @@ -33,6 +33,13 @@ # define ETHR_MTX_HARD_DEBUG #endif +#if 0 +# define ETHR_MTX_CHK_EXCL +#if 1 +# define ETHR_MTX_CHK_NON_EXCL +#endif +#endif + #ifdef ETHR_MTX_HARD_DEBUG # ifdef __GNUC__ # warning ETHR_MTX_HARD_DEBUG @@ -49,6 +56,15 @@ #if defined(ETHR_USE_OWN_RWMTX_IMPL__) || defined(ETHR_USE_OWN_MTX_IMPL__) +#ifdef ETHR_DEBUG +# ifndef ETHR_MTX_CHK_EXCL +# define ETHR_MTX_CHK_EXCL +# endif +# ifndef ETHR_MTX_CHK_NON_EXCL +# define ETHR_MTX_CHK_NON_EXCL +# endif +#endif + #if 0 # define ETHR_MTX_Q_LOCK_SPINLOCK__ # define ETHR_MTX_QLOCK_TYPE__ ethr_spinlock_t @@ -62,14 +78,14 @@ # error Need a qlock implementation #endif -#define ETHR_RWMTX_W_FLG__ (((long) 1) << 31) -#define ETHR_RWMTX_W_WAIT_FLG__ (((long) 1) << 30) -#define ETHR_RWMTX_R_WAIT_FLG__ (((long) 1) << 29) +#define ETHR_RWMTX_W_FLG__ (((ethr_sint32_t) 1) << 31) +#define ETHR_RWMTX_W_WAIT_FLG__ (((ethr_sint32_t) 1) << 30) +#define ETHR_RWMTX_R_WAIT_FLG__ (((ethr_sint32_t) 1) << 29) /* frequent read kind */ -#define ETHR_RWMTX_R_FLG__ (((long) 1) << 28) -#define ETHR_RWMTX_R_PEND_UNLCK_MASK__ (ETHR_RWMTX_R_FLG__ - 1) -#define ETHR_RWMTX_R_MASK__ (ETHR_RWMTX_R_WAIT_FLG__ - 1) +#define ETHR_RWMTX_R_FLG__ (((ethr_sint32_t) 1) << 28) +#define ETHR_RWMTX_R_ABRT_UNLCK_FLG__ (((ethr_sint32_t) 1) << 27) +#define ETHR_RWMTX_R_PEND_UNLCK_MASK__ (ETHR_RWMTX_R_ABRT_UNLCK_FLG__ - 1) /* normal kind */ #define ETHR_RWMTX_RS_MASK__ (ETHR_RWMTX_R_WAIT_FLG__ - 1) @@ -79,20 +95,39 @@ #define ETHR_CND_WAIT_FLG__ ETHR_RWMTX_R_WAIT_FLG__ +#ifdef ETHR_DEBUG +#define ETHR_DBG_CHK_UNUSED_FLG_BITS(V) \ + ETHR_ASSERT(!((V) & ~(ETHR_RWMTX_W_FLG__ \ + | ETHR_RWMTX_W_WAIT_FLG__ \ + | ETHR_RWMTX_R_WAIT_FLG__ \ + | ETHR_RWMTX_RS_MASK__))) +#else +#define ETHR_DBG_CHK_UNUSED_FLG_BITS(V) +#endif + +#define ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(MTX) \ + ETHR_DBG_CHK_UNUSED_FLG_BITS(ethr_atomic32_read(&(MTX)->mtxb.flgs)) + struct ethr_mutex_base_ { #ifdef ETHR_MTX_HARD_DEBUG_FENCE long pre_fence; #endif - ethr_atomic_t flgs; - ETHR_MTX_QLOCK_TYPE__ qlck; - ethr_ts_event *q; + ethr_atomic32_t flgs; short aux_scnt; short main_scnt; + ETHR_MTX_QLOCK_TYPE__ qlck; + ethr_ts_event *q; #ifdef ETHR_MTX_HARD_DEBUG_WSQ int ws; #endif +#ifdef ETHR_MTX_CHK_EXCL + ethr_atomic32_t exclusive; +#endif +#ifdef ETHR_MTX_CHK_NON_EXCL + ethr_atomic32_t non_exclusive; +#endif #ifdef ETHR_MTX_HARD_DEBUG_LFS - ethr_atomic_t hdbg_lfs; + ethr_atomic32_t hdbg_lfs; #endif }; @@ -201,7 +236,7 @@ typedef struct { typedef union { struct { - ethr_atomic_t readers; + ethr_atomic32_t readers; int waiting_readers; int byte_offset; ethr_rwmutex_lived lived; @@ -263,13 +298,13 @@ void ethr_rwmutex_rwunlock(ethr_rwmutex *); #ifdef ETHR_MTX_HARD_DEBUG_LFS # define ETHR_MTX_HARD_DEBUG_LFS_INIT(MTXB) \ do { \ - ethr_atomic_init(&(MTXB)->hdbg_lfs, 0); \ + ethr_atomic32_init(&(MTXB)->hdbg_lfs, 0); \ } while (0) # define ETHR_MTX_HARD_DEBUG_LFS_RLOCK(MTXB) \ do { \ - long val__; \ + ethr_sint32_t val__; \ ETHR_COMPILER_BARRIER; \ - val__ = ethr_atomic_inc_read(&(MTXB)->hdbg_lfs); \ + val__ = ethr_atomic32_inc_read(&(MTXB)->hdbg_lfs); \ ETHR_MTX_HARD_ASSERT(val__ > 0); \ } while (0) # define ETHR_MTX_HARD_DEBUG_LFS_TRYRLOCK(MTXB, RES) \ @@ -282,15 +317,15 @@ do { \ } while (0) # define ETHR_MTX_HARD_DEBUG_LFS_RUNLOCK(MTXB) \ do { \ - long val__ = ethr_atomic_dec_read(&(MTXB)->hdbg_lfs); \ + ethr_sint32_t val__ = ethr_atomic32_dec_read(&(MTXB)->hdbg_lfs); \ ETHR_MTX_HARD_ASSERT(val__ >= 0); \ ETHR_COMPILER_BARRIER; \ } while (0) # define ETHR_MTX_HARD_DEBUG_LFS_RWLOCK(MTXB) \ do { \ - long val__; \ + ethr_sint32_t val__; \ ETHR_COMPILER_BARRIER; \ - val__ = ethr_atomic_dec_read(&(MTXB)->hdbg_lfs); \ + val__ = ethr_atomic32_dec_read(&(MTXB)->hdbg_lfs); \ ETHR_MTX_HARD_ASSERT(val__ == -1); \ } while (0) # define ETHR_MTX_HARD_DEBUG_LFS_TRYRWLOCK(MTXB, RES) \ @@ -303,7 +338,7 @@ do { \ } while (0) # define ETHR_MTX_HARD_DEBUG_LFS_RWUNLOCK(MTXB) \ do { \ - long val__ = ethr_atomic_inctest(&(MTXB)->hdbg_lfs); \ + ethr_sint32_t val__ = ethr_atomic32_inctest(&(MTXB)->hdbg_lfs); \ ETHR_MTX_HARD_ASSERT(val__ == 0); \ ETHR_COMPILER_BARRIER; \ } while (0) @@ -344,6 +379,116 @@ do { \ #define ETHR_MTX_HARD_DEBUG_FENCE_INIT(X) #endif +#ifdef ETHR_MTX_CHK_EXCL + +#if !defined(ETHR_DEBUG) && defined(__GNUC__) +#warning "check exclusive is enabled" +#endif + +# define ETHR_MTX_CHK_EXCL_INIT__(MTXB) \ + ethr_atomic32_init(&(MTXB)->exclusive, 0) + +# define ETHR_MTX_CHK_EXCL_IS_EXCL(MTXB) \ +do { \ + ETHR_COMPILER_BARRIER; \ + if (!ethr_atomic32_read(&(MTXB)->exclusive)) \ + ethr_assert_failed(__FILE__, __LINE__, __func__,\ + "is exclusive"); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_IS_NOT_EXCL(MTXB) \ +do { \ + ETHR_COMPILER_BARRIER; \ + if (ethr_atomic32_read(&(MTXB)->exclusive)) \ + ethr_assert_failed(__FILE__, __LINE__, __func__,\ + "is not exclusive"); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_SET_EXCL(MTXB) \ +do { \ + ETHR_MTX_CHK_EXCL_IS_NOT_EXCL((MTXB)); \ + ethr_atomic32_set(&(MTXB)->exclusive, 1); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_UNSET_EXCL(MTXB) \ +do { \ + ETHR_MTX_CHK_EXCL_IS_EXCL((MTXB)); \ + ethr_atomic32_set(&(MTXB)->exclusive, 0); \ + ETHR_COMPILER_BARRIER; \ +} while (0) + +#ifdef ETHR_MTX_CHK_NON_EXCL + +#if !defined(ETHR_DEBUG) && defined(__GNUC__) +#warning "check non-exclusive is enabled" +#endif + +# define ETHR_MTX_CHK_NON_EXCL_INIT__(MTXB) \ + ethr_atomic32_init(&(MTXB)->non_exclusive, 0) +# define ETHR_MTX_CHK_EXCL_IS_NON_EXCL(MTXB) \ +do { \ + ETHR_COMPILER_BARRIER; \ + if (!ethr_atomic32_read(&(MTXB)->non_exclusive)) \ + ethr_assert_failed(__FILE__, __LINE__, __func__,\ + "is non-exclusive"); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_IS_NOT_NON_EXCL(MTXB) \ +do { \ + ETHR_COMPILER_BARRIER; \ + if (ethr_atomic32_read(&(MTXB)->non_exclusive)) \ + ethr_assert_failed(__FILE__, __LINE__, __func__,\ + "is not non-exclusive"); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_SET_NON_EXCL(MTXB) \ +do { \ + ETHR_COMPILER_BARRIER; \ + ethr_atomic32_inc(&(MTXB)->non_exclusive); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_SET_NON_EXCL_NO(MTXB, NO) \ +do { \ + ETHR_COMPILER_BARRIER; \ + ethr_atomic32_add(&(MTXB)->non_exclusive, (NO)); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +# define ETHR_MTX_CHK_EXCL_UNSET_NON_EXCL(MTXB) \ +do { \ + ETHR_COMPILER_BARRIER; \ + ethr_atomic32_dec(&(MTXB)->non_exclusive); \ + ETHR_COMPILER_BARRIER; \ +} while (0) +#else +# define ETHR_MTX_CHK_NON_EXCL_INIT__(MTXB) +# define ETHR_MTX_CHK_EXCL_IS_NON_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_IS_NOT_NON_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_SET_NON_EXCL_NO(MTXB, NO) +# define ETHR_MTX_CHK_EXCL_SET_NON_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_UNSET_NON_EXCL(MTXB) +#endif + +#else +# define ETHR_MTX_CHK_EXCL_INIT__(MTXB) +# define ETHR_MTX_CHK_EXCL_IS_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_IS_NOT_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_SET_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_UNSET_EXCL(MTXB) +# define ETHR_MTX_CHK_NON_EXCL_INIT__(MTXB) +# define ETHR_MTX_CHK_EXCL_IS_NON_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_IS_NOT_NON_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_SET_NON_EXCL_NO(MTXB, NO) +# define ETHR_MTX_CHK_EXCL_SET_NON_EXCL(MTXB) +# define ETHR_MTX_CHK_EXCL_UNSET_NON_EXCL(MTXB) +#endif + +# define ETHR_MTX_CHK_EXCL_INIT(MTXB) \ +do { \ + ETHR_MTX_CHK_EXCL_INIT__((MTXB)); \ + ETHR_MTX_CHK_NON_EXCL_INIT__((MTXB)); \ +} while (0) + + #ifdef ETHR_USE_OWN_MTX_IMPL__ #define ETHR_MTX_DEFAULT_MAIN_SPINCOUNT_MAX 2000 @@ -356,21 +501,28 @@ do { \ #if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_MUTEX_IMPL__) -void ethr_mutex_lock_wait__(ethr_mutex *, long); -void ethr_mutex_unlock_wake__(ethr_mutex *, long); +void ethr_mutex_lock_wait__(ethr_mutex *, ethr_sint32_t); +void ethr_mutex_unlock_wake__(ethr_mutex *, ethr_sint32_t); static ETHR_INLINE int ETHR_INLINE_FUNC_NAME_(ethr_mutex_trylock)(ethr_mutex *mtx) { - long act; + ethr_sint32_t act; int res; ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(mtx); - act = ethr_atomic_cmpxchg_acqb(&mtx->mtxb.flgs, ETHR_RWMTX_W_FLG__, 0); + act = ethr_atomic32_cmpxchg_acqb(&mtx->mtxb.flgs, ETHR_RWMTX_W_FLG__, 0); res = (act == 0) ? 0 : EBUSY; +#ifdef ETHR_MTX_CHK_EXCL + if (res == 0) + ETHR_MTX_CHK_EXCL_SET_EXCL(&mtx->mtxb); +#endif + ETHR_MTX_HARD_DEBUG_LFS_TRYRWLOCK(&mtx->mtxb, res); ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(mtx); ETHR_COMPILER_BARRIER; return res; @@ -379,15 +531,19 @@ ETHR_INLINE_FUNC_NAME_(ethr_mutex_trylock)(ethr_mutex *mtx) static ETHR_INLINE void ETHR_INLINE_FUNC_NAME_(ethr_mutex_lock)(ethr_mutex *mtx) { - long act; + ethr_sint32_t act; ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(mtx); - act = ethr_atomic_cmpxchg_acqb(&mtx->mtxb.flgs, ETHR_RWMTX_W_FLG__, 0); + act = ethr_atomic32_cmpxchg_acqb(&mtx->mtxb.flgs, ETHR_RWMTX_W_FLG__, 0); if (act != 0) ethr_mutex_lock_wait__(mtx, act); + ETHR_MTX_CHK_EXCL_SET_EXCL(&mtx->mtxb); + ETHR_MTX_HARD_DEBUG_LFS_RWLOCK(&mtx->mtxb); ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(mtx); ETHR_COMPILER_BARRIER; } @@ -395,16 +551,20 @@ ETHR_INLINE_FUNC_NAME_(ethr_mutex_lock)(ethr_mutex *mtx) static ETHR_INLINE void ETHR_INLINE_FUNC_NAME_(ethr_mutex_unlock)(ethr_mutex *mtx) { - long act; + ethr_sint32_t act; ETHR_COMPILER_BARRIER; ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); ETHR_MTX_HARD_DEBUG_LFS_RWUNLOCK(&mtx->mtxb); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(mtx); + + ETHR_MTX_CHK_EXCL_UNSET_EXCL(&mtx->mtxb); - act = ethr_atomic_cmpxchg_relb(&mtx->mtxb.flgs, 0, ETHR_RWMTX_W_FLG__); + act = ethr_atomic32_cmpxchg_relb(&mtx->mtxb.flgs, 0, ETHR_RWMTX_W_FLG__); if (act != ETHR_RWMTX_W_FLG__) ethr_mutex_unlock_wake__(mtx, act); ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(mtx); } #endif /* ETHR_TRY_INLINE_FUNCS */ diff --git a/erts/include/internal/ethr_optimized_fallbacks.h b/erts/include/internal/ethr_optimized_fallbacks.h index 2f9f987d0b..8e04692856 100644 --- a/erts/include/internal/ethr_optimized_fallbacks.h +++ b/erts/include/internal/ethr_optimized_fallbacks.h @@ -71,36 +71,46 @@ ethr_opt_spin_lock(ethr_opt_spinlock_t *lock) #define ETHR_HAVE_NATIVE_SPINLOCKS 1 #define ETHR_HAVE_OPTIMIZED_SPINLOCKS 1 -typedef ethr_native_atomic_t ethr_native_spinlock_t; +#if defined(ETHR_HAVE_NATIVE_ATOMIC32) +typedef ethr_native_atomic32_t ethr_native_spinlock_t; +# define ETHR_NATMC_FUNC__(X) ethr_native_atomic32_ ## X +#elif defined(ETHR_HAVE_NATIVE_ATOMIC64) +typedef ethr_native_atomic64_t ethr_native_spinlock_t; +# define ETHR_NATMC_FUNC__(X) ethr_native_atomic64_ ## X +#else +# error "Missing native atomic implementation" +#endif #if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_AUX_IMPL__) static ETHR_INLINE void ethr_native_spinlock_init(ethr_native_spinlock_t *lock) { - ethr_native_atomic_init((ethr_native_atomic_t *) lock, 0); + ETHR_NATMC_FUNC__(init)(lock, 0); } static ETHR_INLINE void ethr_native_spin_unlock(ethr_native_spinlock_t *lock) { ETHR_COMPILER_BARRIER; - ETHR_ASSERT(ethr_native_atomic_read((ethr_native_atomic_t *) lock) == 1); - ethr_native_atomic_set_relb((ethr_native_atomic_t *) lock, 0); + ETHR_ASSERT(ETHR_NATMC_FUNC__(read)(lock) == 1); + ETHR_NATMC_FUNC__(set_relb)(lock, 0); } static ETHR_INLINE void ethr_native_spin_lock(ethr_native_spinlock_t *lock) { - while (ethr_native_atomic_cmpxchg_acqb((ethr_native_atomic_t *) lock, - (long) 1, (long) 0) != 0) { - ETHR_SPIN_BODY; + while (ETHR_NATMC_FUNC__(cmpxchg_acqb)(lock, 1, 0) != 0) { + while (ETHR_NATMC_FUNC__(read)(lock) != 0) + ETHR_SPIN_BODY; } ETHR_COMPILER_BARRIER; } #endif +#undef ETHR_NATMC_FUNC__ + #endif @@ -111,16 +121,26 @@ ethr_native_spin_lock(ethr_native_spinlock_t *lock) #define ETHR_HAVE_NATIVE_RWSPINLOCKS 1 #define ETHR_HAVE_OPTIMIZED_RWSPINLOCKS 1 -typedef ethr_native_atomic_t ethr_native_rwlock_t; +#if defined(ETHR_HAVE_NATIVE_ATOMIC32) +typedef ethr_native_atomic32_t ethr_native_rwlock_t; +# define ETHR_NAINT_T__ ethr_sint32_t +# define ETHR_WLOCK_FLAG__ (((ethr_sint32_t) 1) << 30) +# define ETHR_NATMC_FUNC__(X) ethr_native_atomic32_ ## X +#elif defined(ETHR_HAVE_NATIVE_ATOMIC64) +typedef ethr_native_atomic64_t ethr_native_rwlock_t; +# define ETHR_NAINT_T__ ethr_sint64_t +# define ETHR_WLOCK_FLAG__ (((ethr_sint64_t) 1) << 62) +# define ETHR_NATMC_FUNC__(X) ethr_native_atomic64_ ## X +#else +# error "Missing native atomic implementation" +#endif #if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_AUX_IMPL__) -#define ETHR_WLOCK_FLAG__ (((long) 1) << 30) - static ETHR_INLINE void ethr_native_rwlock_init(ethr_native_rwlock_t *lock) { - ethr_native_atomic_init((ethr_native_atomic_t *) lock, 0); + ETHR_NATMC_FUNC__(init)(lock, 0); } static ETHR_INLINE void @@ -128,22 +148,24 @@ ethr_native_read_unlock(ethr_native_rwlock_t *lock) { ETHR_COMPILER_BARRIER; #ifdef DEBUG - ETHR_ASSERT(ethr_native_atomic_read((ethr_native_atomic_t *) lock) >= 0); + ETHR_ASSERT(ETHR_NATMC_FUNC__(read)(lock) >= 0); #endif - ethr_native_atomic_dec_relb((ethr_native_atomic_t *) lock); + ETHR_NATMC_FUNC__(dec_relb)(lock); } static ETHR_INLINE void ethr_native_read_lock(ethr_native_rwlock_t *lock) { - long act, exp = 0; + ETHR_NAINT_T__ act, exp = 0; while (1) { - act = ethr_native_atomic_cmpxchg_acqb((ethr_native_atomic_t *) lock, - exp+1, exp); + act = ETHR_NATMC_FUNC__(cmpxchg_acqb)(lock, exp+1, exp); if (act == exp) break; - ETHR_SPIN_BODY; - exp = (act & ETHR_WLOCK_FLAG__) ? 0 : act; + while (act & ETHR_WLOCK_FLAG__) { + ETHR_SPIN_BODY; + act = ETHR_NATMC_FUNC__(read)(lock); + } + exp = act; } ETHR_COMPILER_BARRIER; } @@ -152,18 +174,16 @@ static ETHR_INLINE void ethr_native_write_unlock(ethr_native_rwlock_t *lock) { ETHR_COMPILER_BARRIER; - ETHR_ASSERT(ethr_native_atomic_read((ethr_native_atomic_t *) lock) - == ETHR_WLOCK_FLAG__); - ethr_native_atomic_set_relb((ethr_native_atomic_t *) lock, 0); + ETHR_ASSERT(ETHR_NATMC_FUNC__(read)(lock) == ETHR_WLOCK_FLAG__); + ETHR_NATMC_FUNC__(set_relb)(lock, 0); } static ETHR_INLINE void ethr_native_write_lock(ethr_native_rwlock_t *lock) { - long act, exp = 0; + ETHR_NAINT_T__ act, exp = 0; while (1) { - act = ethr_native_atomic_cmpxchg_acqb((ethr_native_atomic_t *) lock, - exp|ETHR_WLOCK_FLAG__, exp); + act = ETHR_NATMC_FUNC__(cmpxchg_acqb)(lock, exp|ETHR_WLOCK_FLAG__, exp); if (act == exp) break; ETHR_SPIN_BODY; @@ -173,13 +193,17 @@ ethr_native_write_lock(ethr_native_rwlock_t *lock) /* Wait for readers to leave */ while (act != ETHR_WLOCK_FLAG__) { ETHR_SPIN_BODY; - act = ethr_native_atomic_read_acqb((ethr_native_atomic_t *) lock); + act = ETHR_NATMC_FUNC__(read_acqb)(lock); } ETHR_COMPILER_BARRIER; } #endif +#undef ETHR_NAINT_T__ +#undef ETHR_NATMC_FUNC__ +#undef ETHR_WLOCK_FLAG__ + #endif #endif diff --git a/erts/include/internal/ethread.h b/erts/include/internal/ethread.h index 53fa1acdc2..4cd95faf6a 100644 --- a/erts/include/internal/ethread.h +++ b/erts/include/internal/ethread.h @@ -37,11 +37,6 @@ #undef ETHR_HAVE_OPTIMIZED_SPINLOCK #undef ETHR_HAVE_OPTIMIZED_RWSPINLOCK -typedef struct { - long tv_sec; - long tv_nsec; -} ethr_timeval; - #if defined(DEBUG) # define ETHR_DEBUG #endif @@ -73,7 +68,7 @@ typedef struct { #endif /* Assume 64-byte cache line size */ -#define ETHR_CACHE_LINE_SIZE 64L +#define ETHR_CACHE_LINE_SIZE ((ethr_uint_t) 64) #define ETHR_CACHE_LINE_MASK (ETHR_CACHE_LINE_SIZE - 1) #define ETHR_CACHE_LINE_ALIGN_SIZE(SZ) \ @@ -171,6 +166,22 @@ typedef pthread_key_t ethr_tsd_key; # undef WIN32_LEAN_AND_MEAN #endif +#if defined(_MSC_VER) + +#if ETHR_SIZEOF_LONG == 4 +#define ETHR_HAVE_INT32_T 1 +typedef long ethr_sint32_t; +typedef unsigned long ethr_uint32_t; +#endif + +#if ETHR_SIZEOF___INT64 == 8 +#define ETHR_HAVE_INT64_T 1 +typedef __int64 ethr_sint64_t; +typedef unsigned __int64 ethr_uint64_t; +#endif + +#endif + struct ethr_join_data_; /* Types */ @@ -198,12 +209,48 @@ typedef DWORD ethr_tsd_key; #endif -#ifdef SIZEOF_LONG -#if SIZEOF_LONG < ETHR_SIZEOF_PTR -#error size of long currently needs to be at least the same as size of void * +#ifndef ETHR_HAVE_INT32_T +#if ETHR_SIZEOF_INT == 4 +#define ETHR_HAVE_INT32_T 1 +typedef int ethr_sint32_t; +typedef unsigned int ethr_uint32_t; +#elif ETHR_SIZEOF_LONG == 4 +#define ETHR_HAVE_INT32_T 1 +typedef long ethr_sint32_t; +typedef unsigned long ethr_uint32_t; #endif #endif +#ifndef ETHR_HAVE_INT64_T +#if ETHR_SIZEOF_INT == 8 +#define ETHR_HAVE_INT64_T 1 +typedef int ethr_sint64_t; +typedef unsigned int ethr_uint64_t; +#elif ETHR_SIZEOF_LONG == 8 +#define ETHR_HAVE_INT64_T 1 +typedef long ethr_sint64_t; +typedef unsigned long ethr_uint64_t; +#elif ETHR_SIZEOF_LONG_LONG == 8 +#define ETHR_HAVE_INT64_T 1 +typedef long long ethr_sint64_t; +typedef unsigned long long ethr_uint64_t; +#endif +#endif + +#if ETHR_SIZEOF_PTR == 4 +#ifndef ETHR_HAVE_INT32_T +#error "No 32-bit integer type found" +#endif +typedef ethr_sint32_t ethr_sint_t; +typedef ethr_uint32_t ethr_uint_t; +#elif ETHR_SIZEOF_PTR == 8 +#ifndef ETHR_HAVE_INT64_T +#error "No 64-bit integer type found" +#endif +typedef ethr_sint64_t ethr_sint_t; +typedef ethr_uint64_t ethr_uint_t; +#endif + /* __builtin_expect() is needed by both native atomics code * and the fallback code */ #if !defined(__GNUC__) || (__GNUC__ < 2) || (__GNUC__ == 2 && __GNUC_MINOR__ < 96) @@ -386,7 +433,6 @@ typedef struct { #if !defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_AUX_IMPL__) # define ETHR_NEED_SPINLOCK_PROTOTYPES__ # define ETHR_NEED_RWSPINLOCK_PROTOTYPES__ -# define ETHR_NEED_ATOMIC_PROTOTYPES__ #endif int ethr_init(ethr_init_data *); @@ -399,7 +445,6 @@ void ethr_thr_exit(void *); ethr_tid ethr_self(void); int ethr_equal_tids(ethr_tid, ethr_tid); -int ethr_time_now(ethr_timeval *); int ethr_tsd_key_create(ethr_tsd_key *); int ethr_tsd_key_delete(ethr_tsd_key); int ethr_tsd_set(ethr_tsd_key, void *); @@ -502,312 +547,7 @@ ETHR_INLINE_FUNC_NAME_(ethr_spin_lock)(ethr_spinlock_t *lock) #endif /* ETHR_TRY_INLINE_FUNCS */ -#ifdef ETHR_HAVE_NATIVE_ATOMICS -/* - * Map ethread native atomics to ethread API atomics. - */ -typedef ethr_native_atomic_t ethr_atomic_t; -#else -typedef long ethr_atomic_t; -#endif - -#ifdef ETHR_NEED_ATOMIC_PROTOTYPES__ -void ethr_atomic_init(ethr_atomic_t *, long); -void ethr_atomic_set(ethr_atomic_t *, long); -long ethr_atomic_read(ethr_atomic_t *); -long ethr_atomic_inc_read(ethr_atomic_t *); -long ethr_atomic_dec_read(ethr_atomic_t *); -void ethr_atomic_inc(ethr_atomic_t *); -void ethr_atomic_dec(ethr_atomic_t *); -long ethr_atomic_add_read(ethr_atomic_t *, long); -void ethr_atomic_add(ethr_atomic_t *, long); -long ethr_atomic_read_band(ethr_atomic_t *, long); -long ethr_atomic_read_bor(ethr_atomic_t *, long); -long ethr_atomic_xchg(ethr_atomic_t *, long); -long ethr_atomic_cmpxchg(ethr_atomic_t *, long, long); -long ethr_atomic_read_acqb(ethr_atomic_t *); -long ethr_atomic_inc_read_acqb(ethr_atomic_t *); -void ethr_atomic_set_relb(ethr_atomic_t *, long); -void ethr_atomic_dec_relb(ethr_atomic_t *); -long ethr_atomic_dec_read_relb(ethr_atomic_t *); -long ethr_atomic_cmpxchg_acqb(ethr_atomic_t *, long, long); -long ethr_atomic_cmpxchg_relb(ethr_atomic_t *, long, long); -#endif - -#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_AUX_IMPL__) - -#ifndef ETHR_HAVE_NATIVE_ATOMICS -/* - * Fallbacks for atomics used in absence of a native implementation. - */ - -#define ETHR_ATOMIC_ADDR_BITS 10 -#define ETHR_ATOMIC_ADDR_SHIFT 6 - -typedef struct { - union { - ethr_spinlock_t lck; - char buf[ETHR_CACHE_LINE_SIZE]; - } u; -} ethr_atomic_protection_t; - -extern ethr_atomic_protection_t ethr_atomic_protection__[1 << ETHR_ATOMIC_ADDR_BITS]; - -#define ETHR_ATOMIC_PTR2LCK__(PTR) \ -(ðr_atomic_protection__[((((unsigned long) (PTR)) >> ETHR_ATOMIC_ADDR_SHIFT) \ - & ((1 << ETHR_ATOMIC_ADDR_BITS) - 1))].u.lck) - - -#define ETHR_ATOMIC_OP_FALLBACK_IMPL__(AP, EXPS) \ -do { \ - ethr_spinlock_t *slp__ = ETHR_ATOMIC_PTR2LCK__((AP)); \ - ethr_spin_lock(slp__); \ - { EXPS; } \ - ethr_spin_unlock(slp__); \ -} while (0) - -#endif - -static ETHR_INLINE void -ETHR_INLINE_FUNC_NAME_(ethr_atomic_init)(ethr_atomic_t *var, long i) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - ethr_native_atomic_init(var, i); -#else - ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, *var = i); -#endif -} - -static ETHR_INLINE void -ETHR_INLINE_FUNC_NAME_(ethr_atomic_set)(ethr_atomic_t *var, long i) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - ethr_native_atomic_set(var, i); -#else - ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, *var = i); -#endif -} - -static ETHR_INLINE long -ETHR_INLINE_FUNC_NAME_(ethr_atomic_read)(ethr_atomic_t *var) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - return ethr_native_atomic_read(var); -#else - long res; - ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = (long) *var); - return res; -#endif -} - -static ETHR_INLINE void -ETHR_INLINE_FUNC_NAME_(ethr_atomic_add)(ethr_atomic_t *var, long incr) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - ethr_native_atomic_add(var, incr); -#else - ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, *var += incr); -#endif -} - -static ETHR_INLINE long -ETHR_INLINE_FUNC_NAME_(ethr_atomic_add_read)(ethr_atomic_t *var, long i) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - return ethr_native_atomic_add_return(var, i); -#else - long res; - ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, *var += i; res = *var); - return res; -#endif -} - -static ETHR_INLINE void -ETHR_INLINE_FUNC_NAME_(ethr_atomic_inc)(ethr_atomic_t *var) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - ethr_native_atomic_inc(var); -#else - ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, ++(*var)); -#endif -} - -static ETHR_INLINE void -ETHR_INLINE_FUNC_NAME_(ethr_atomic_dec)(ethr_atomic_t *var) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - ethr_native_atomic_dec(var); -#else - ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, --(*var)); -#endif -} - -static ETHR_INLINE long -ETHR_INLINE_FUNC_NAME_(ethr_atomic_inc_read)(ethr_atomic_t *var) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - return ethr_native_atomic_inc_return(var); -#else - long res; - ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = (long) ++(*var)); - return res; -#endif -} - -static ETHR_INLINE long -ETHR_INLINE_FUNC_NAME_(ethr_atomic_dec_read)(ethr_atomic_t *var) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - return ethr_native_atomic_dec_return(var); -#else - long res; - ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = (long) --(*var)); - return res; -#endif -} - -static ETHR_INLINE long -ETHR_INLINE_FUNC_NAME_(ethr_atomic_read_band)(ethr_atomic_t *var, - long mask) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - return ethr_native_atomic_and_retold(var, mask); -#else - long res; - ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = *var; *var &= mask); - return res; -#endif -} - -static ETHR_INLINE long -ETHR_INLINE_FUNC_NAME_(ethr_atomic_read_bor)(ethr_atomic_t *var, - long mask) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - return ethr_native_atomic_or_retold(var, mask); -#else - long res; - ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = *var; *var |= mask); - return res; -#endif -} - -static ETHR_INLINE long -ETHR_INLINE_FUNC_NAME_(ethr_atomic_xchg)(ethr_atomic_t *var, - long new) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - return ethr_native_atomic_xchg(var, new); -#else - long res; - ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, res = *var; *var = new); - return res; -#endif -} - -static ETHR_INLINE long -ETHR_INLINE_FUNC_NAME_(ethr_atomic_cmpxchg)(ethr_atomic_t *var, - long new, - long exp) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - return ethr_native_atomic_cmpxchg(var, new, exp); -#else - long res; - ETHR_ATOMIC_OP_FALLBACK_IMPL__(var, - { - res = *var; - if (__builtin_expect(res == exp, 1)) - *var = new; - }); - return res; -#endif -} - -/* - * Important memory barrier requirements. - * - * The following atomic operations *must* supply a memory barrier of - * at least the type specified by its suffix: - * _acqb = acquire barrier - * _relb = release barrier - */ - -static ETHR_INLINE long -ETHR_INLINE_FUNC_NAME_(ethr_atomic_read_acqb)(ethr_atomic_t *var) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - return ethr_native_atomic_read_acqb(var); -#else - return ETHR_INLINE_FUNC_NAME_(ethr_atomic_read)(var); -#endif -} - -static ETHR_INLINE long -ETHR_INLINE_FUNC_NAME_(ethr_atomic_inc_read_acqb)(ethr_atomic_t *var) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - return ethr_native_atomic_inc_return_acqb(var); -#else - return ETHR_INLINE_FUNC_NAME_(ethr_atomic_inc_read)(var); -#endif -} - -static ETHR_INLINE void -ETHR_INLINE_FUNC_NAME_(ethr_atomic_set_relb)(ethr_atomic_t *var, long val) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - ethr_native_atomic_set_relb(var, val); -#else - ETHR_INLINE_FUNC_NAME_(ethr_atomic_set)(var, val); -#endif -} - -static ETHR_INLINE void -ETHR_INLINE_FUNC_NAME_(ethr_atomic_dec_relb)(ethr_atomic_t *var) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - ethr_native_atomic_dec_relb(var); -#else - ETHR_INLINE_FUNC_NAME_(ethr_atomic_dec)(var); -#endif -} - -static ETHR_INLINE long -ETHR_INLINE_FUNC_NAME_(ethr_atomic_dec_read_relb)(ethr_atomic_t *var) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - return ethr_native_atomic_dec_return_relb(var); -#else - return ETHR_INLINE_FUNC_NAME_(ethr_atomic_dec_read)(var); -#endif -} - -static ETHR_INLINE long -ETHR_INLINE_FUNC_NAME_(ethr_atomic_cmpxchg_acqb)(ethr_atomic_t *var, - long new, - long exp) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - return ethr_native_atomic_cmpxchg_acqb(var, new, exp); -#else - return ETHR_INLINE_FUNC_NAME_(ethr_atomic_cmpxchg)(var, new, exp); -#endif -} - -static ETHR_INLINE long -ETHR_INLINE_FUNC_NAME_(ethr_atomic_cmpxchg_relb)(ethr_atomic_t *var, - long new, - long exp) -{ -#ifdef ETHR_HAVE_NATIVE_ATOMICS - return ethr_native_atomic_cmpxchg_relb(var, new, exp); -#else - return ETHR_INLINE_FUNC_NAME_(ethr_atomic_cmpxchg)(var, new, exp); -#endif -} - -#endif /* ETHR_TRY_INLINE_FUNCS */ +#include "ethr_atomics.h" typedef struct ethr_ts_event_ ethr_ts_event; /* Needed by ethr_mutex.h */ @@ -825,7 +565,7 @@ struct ethr_ts_event_ { ethr_ts_event *prev; ethr_event event; void *udata; - ethr_atomic_t uaflgs; + ethr_atomic32_t uaflgs; unsigned uflgs; unsigned iflgs; /* for ethr lib only */ short rgix; /* for ethr lib only */ diff --git a/erts/include/internal/ethread_header_config.h.in b/erts/include/internal/ethread_header_config.h.in index 5debb44756..f394d790d2 100644 --- a/erts/include/internal/ethread_header_config.h.in +++ b/erts/include/internal/ethread_header_config.h.in @@ -20,6 +20,21 @@ /* Define to the size of pointers */ #undef ETHR_SIZEOF_PTR +/* Define to the size of int */ +#undef ETHR_SIZEOF_INT + +/* Define to the size of long */ +#undef ETHR_SIZEOF_LONG + +/* Define to the size of long long */ +#undef ETHR_SIZEOF_LONG_LONG + +/* Define to the size of __int64 */ +#undef ETHR_SIZEOF___INT64 + +/* Define if bigendian */ +#undef ETHR_BIGENDIAN + /* Define if you want to disable native ethread implementations */ #undef ETHR_DISABLE_NATIVE_IMPLS @@ -100,6 +115,27 @@ /* Define to the size of AO_t if libatomic_ops is used */ #undef ETHR_SIZEOF_AO_T +/* Define if you have _InterlockedCompareExchange64() */ +#undef ETHR_HAVE__INTERLOCKEDCOMPAREEXCHANGE64 + +/* Define if you have _InterlockedDecrement64() */ +#undef ETHR_HAVE__INTERLOCKEDDECREMENT64 + +/* Define if you have _InterlockedIncrement64() */ +#undef ETHR_HAVE__INTERLOCKEDINCREMENT64 + +/* Define if you have _InterlockedExchangeAdd64() */ +#undef ETHR_HAVE__INTERLOCKEDEXCHANGEADD64 + +/* Define if you have _InterlockedExchange64() */ +#undef ETHR_HAVE__INTERLOCKEDEXCHANGE64 + +/* Define if you have _InterlockedAnd64() */ +#undef ETHR_HAVE__INTERLOCKEDAND64 + +/* Define if you have _InterlockedOr64() */ +#undef ETHR_HAVE__INTERLOCKEDOR64 + /* Define if you want to turn on extra sanity checking in the ethread library */ #undef ETHR_XCHK diff --git a/erts/include/internal/gcc/ethr_atomic.h b/erts/include/internal/gcc/ethr_atomic.h index e8e529dd48..16935084b1 100644 --- a/erts/include/internal/gcc/ethr_atomic.h +++ b/erts/include/internal/gcc/ethr_atomic.h @@ -22,24 +22,35 @@ * Author: Rickard Green */ -#ifndef ETHR_GCC_ATOMIC_H__ -#define ETHR_GCC_ATOMIC_H__ +#undef ETHR_INCLUDE_ATOMIC_IMPL__ +#if !defined(ETHR_GCC_ATOMIC32_H__) && defined(ETHR_ATOMIC_WANT_32BIT_IMPL__) +#define ETHR_GCC_ATOMIC32_H__ +#define ETHR_INCLUDE_ATOMIC_IMPL__ 4 +#undef ETHR_ATOMIC_WANT_32BIT_IMPL__ +#elif !defined(ETHR_GCC_ATOMIC64_H__) && defined(ETHR_ATOMIC_WANT_64BIT_IMPL__) +#define ETHR_GCC_ATOMIC64_H__ +#define ETHR_INCLUDE_ATOMIC_IMPL__ 8 +#undef ETHR_ATOMIC_WANT_64BIT_IMPL__ +#endif + +#ifdef ETHR_INCLUDE_ATOMIC_IMPL__ -#if !defined(ETHR_HAVE_NATIVE_ATOMICS) && defined(ETHR_HAVE_GCC_ATOMIC_OPS) -#define ETHR_HAVE_NATIVE_ATOMICS 1 +#ifndef ETHR_GCC_ATOMIC_COMMON__ +#define ETHR_GCC_ATOMIC_COMMON__ -#define ETHR_IMMED_ATOMIC_SET_GET_SAFE__ 0 -/* Enable immediate read/write on platforms where we know it is safe */ +#define ETHR_READ_AND_SET_WITHOUT_SYNC_OP__ 0 #if defined(__i386__) || defined(__x86_64__) || defined(__sparc__) \ || defined(__powerpc__) || defined(__ppc__) || defined(__mips__) -# undef ETHR_IMMED_ATOMIC_SET_GET_SAFE__ -# define ETHR_IMMED_ATOMIC_SET_GET_SAFE__ 1 +# undef ETHR_READ_AND_SET_WITHOUT_SYNC_OP__ +# define ETHR_READ_AND_SET_WITHOUT_SYNC_OP__ 1 #endif -typedef struct { - volatile long counter; -} ethr_native_atomic_t; - +#if defined(__x86_64__) || (defined(__i386__) \ + && !defined(ETHR_PRE_PENTIUM4_COMPAT)) +# define ETHR_READ_ACQB_AND_SET_RELB_COMPILER_BARRIER_ONLY__ 1 +#else +# define ETHR_READ_ACQB_AND_SET_RELB_COMPILER_BARRIER_ONLY__ 0 +#endif /* * According to the documentation this is what we want: @@ -47,34 +58,73 @@ typedef struct { * However, __sync_synchronize() is known to erroneously be * a noop on at least some platforms with some gcc versions. * This has suposedly been fixed in some gcc version, but we - * don't know from which version. Therefore, we use the - * workaround implemented below on all gcc versions except - * for gcc 4.2 or above for MIPS, where it's been verified. + * don't know from which version. Therefore, we only use + * it when it has been verified to work. Otherwise + * we use a workaround. */ #if defined(__mips__) && (__GNUC__ > 4 || (__GNUC__ == 4 && __GNUC_MINOR__ >= 2)) +/* __sync_synchronize() has been verified to work here */ #define ETHR_MEMORY_BARRIER __sync_synchronize() +#define ETHR_READ_DEPEND_MEMORY_BARRIER __sync_synchronize() +#elif defined(__x86_64__) || (defined(__i386__) \ + && !defined(ETHR_PRE_PENTIUM4_COMPAT)) +/* Use fence instructions directly instead of workaround */ +#define ETHR_MEMORY_BARRIER __asm__ __volatile__("mfence" : : : "memory") +#define ETHR_WRITE_MEMORY_BARRIER __asm__ __volatile__("sfence" : : : "memory") +#define ETHR_READ_MEMORY_BARRIER __asm__ __volatile__("lfence" : : : "memory") +#define ETHR_READ_DEPEND_MEMORY_BARRIER __asm__ __volatile__("" : : : "memory") #else +/* Workaround */ #define ETHR_MEMORY_BARRIER \ do { \ - volatile long x___ = 0; \ - (void) __sync_val_compare_and_swap(&x___, (long) 0, (long) 1); \ + volatile ethr_sint32_t x___ = 0; \ + (void) __sync_val_compare_and_swap(&x___, (ethr_sint32_t) 0, (ethr_sint32_t) 1); \ } while (0) -#endif #define ETHR_READ_DEPEND_MEMORY_BARRIER ETHR_MEMORY_BARRIER +#endif + +#define ETHR_COMPILER_BARRIER __asm__ __volatile__("" : : : "memory") -#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_AUX_IMPL__) +#endif /* ETHR_GCC_ATOMIC_COMMON__ */ + +#if ETHR_INCLUDE_ATOMIC_IMPL__ == 4 +#define ETHR_HAVE_NATIVE_ATOMIC32 1 +#define ETHR_NATMC_FUNC__(X) ethr_native_atomic32_ ## X +#define ETHR_ATMC_T__ ethr_native_atomic32_t +#define ETHR_AINT_T__ ethr_sint32_t +#elif ETHR_INCLUDE_ATOMIC_IMPL__ == 8 +#define ETHR_HAVE_NATIVE_ATOMIC64 1 +#define ETHR_NATMC_FUNC__(X) ethr_native_atomic64_ ## X +#define ETHR_ATMC_T__ ethr_native_atomic64_t +#define ETHR_AINT_T__ ethr_sint64_t +#else +#error "Unsupported integer size" +#endif + +typedef struct { + volatile ETHR_AINT_T__ counter; +} ETHR_ATMC_T__; + + +#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_ATOMIC_IMPL__) + +static ETHR_INLINE ETHR_AINT_T__ * +ETHR_NATMC_FUNC__(addr)(ETHR_ATMC_T__ *var) +{ + return (ETHR_AINT_T__ *) &var->counter; +} static ETHR_INLINE void -ethr_native_atomic_set(ethr_native_atomic_t *var, long value) +ETHR_NATMC_FUNC__(set)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ value) { -#if ETHR_IMMED_ATOMIC_SET_GET_SAFE__ +#if ETHR_READ_AND_SET_WITHOUT_SYNC_OP__ var->counter = value; #else /* * Unfortunately no __sync_store() or similar exist in the gcc atomic * op interface. We therefore have to simulate it this way... */ - long act = 0, exp; + ETHR_AINT_T__ act = 0, exp; do { exp = act; act = __sync_val_compare_and_swap(&var->counter, exp, value); @@ -82,80 +132,86 @@ ethr_native_atomic_set(ethr_native_atomic_t *var, long value) #endif } -#define ethr_native_atomic_init ethr_native_atomic_set +static ETHR_INLINE void +ETHR_NATMC_FUNC__(init)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ value) +{ + ETHR_NATMC_FUNC__(set)(var, value); +} -static ETHR_INLINE long -ethr_native_atomic_read(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(read)(ETHR_ATMC_T__ *var) { -#if ETHR_IMMED_ATOMIC_SET_GET_SAFE__ +#if ETHR_READ_AND_SET_WITHOUT_SYNC_OP__ return var->counter; #else /* * Unfortunately no __sync_fetch() or similar exist in the gcc atomic * op interface. We therefore have to simulate it this way... */ - return __sync_add_and_fetch(&var->counter, (long) 0); + return __sync_add_and_fetch(&var->counter, (ETHR_AINT_T__) 0); #endif } static ETHR_INLINE void -ethr_native_atomic_add(ethr_native_atomic_t *var, long incr) +ETHR_NATMC_FUNC__(add)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ incr) { (void) __sync_add_and_fetch(&var->counter, incr); } -static ETHR_INLINE long -ethr_native_atomic_add_return(ethr_native_atomic_t *var, long incr) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(add_return)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ incr) { return __sync_add_and_fetch(&var->counter, incr); } static ETHR_INLINE void -ethr_native_atomic_inc(ethr_native_atomic_t *var) +ETHR_NATMC_FUNC__(inc)(ETHR_ATMC_T__ *var) { - (void) __sync_add_and_fetch(&var->counter, (long) 1); + (void) __sync_add_and_fetch(&var->counter, (ETHR_AINT_T__) 1); } static ETHR_INLINE void -ethr_native_atomic_dec(ethr_native_atomic_t *var) +ETHR_NATMC_FUNC__(dec)(ETHR_ATMC_T__ *var) { - (void) __sync_sub_and_fetch(&var->counter, (long) 1); + (void) __sync_sub_and_fetch(&var->counter, (ETHR_AINT_T__) 1); } -static ETHR_INLINE long -ethr_native_atomic_inc_return(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(inc_return)(ETHR_ATMC_T__ *var) { - return __sync_add_and_fetch(&var->counter, (long) 1); + return __sync_add_and_fetch(&var->counter, (ETHR_AINT_T__) 1); } -static ETHR_INLINE long -ethr_native_atomic_dec_return(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(dec_return)(ETHR_ATMC_T__ *var) { - return __sync_sub_and_fetch(&var->counter, (long) 1); + return __sync_sub_and_fetch(&var->counter, (ETHR_AINT_T__) 1); } -static ETHR_INLINE long -ethr_native_atomic_and_retold(ethr_native_atomic_t *var, long mask) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(and_retold)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ mask) { return __sync_fetch_and_and(&var->counter, mask); } -static ETHR_INLINE long -ethr_native_atomic_or_retold(ethr_native_atomic_t *var, long mask) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(or_retold)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ mask) { - return (long) __sync_fetch_and_or(&var->counter, mask); + return (ETHR_AINT_T__) __sync_fetch_and_or(&var->counter, mask); } -static ETHR_INLINE long -ethr_native_atomic_cmpxchg(ethr_native_atomic_t *var, long new, long old) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg)(ETHR_ATMC_T__ *var, + ETHR_AINT_T__ new, + ETHR_AINT_T__ old) { return __sync_val_compare_and_swap(&var->counter, old, new); } -static ETHR_INLINE long -ethr_native_atomic_xchg(ethr_native_atomic_t *var, long new) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(xchg)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ new) { - long exp, act = 0; + ETHR_AINT_T__ exp, act = 0; do { exp = act; act = __sync_val_compare_and_swap(&var->counter, exp, new); @@ -167,22 +223,68 @@ ethr_native_atomic_xchg(ethr_native_atomic_t *var, long new) * Atomic ops with at least specified barriers. */ -static ETHR_INLINE long -ethr_native_atomic_read_acqb(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(read_acqb)(ETHR_ATMC_T__ *var) { - return __sync_add_and_fetch(&var->counter, (long) 0); +#if ETHR_READ_ACQB_AND_SET_RELB_COMPILER_BARRIER_ONLY__ + ETHR_AINT_T__ val = var->counter; + ETHR_COMPILER_BARRIER; + return val; +#else + return __sync_add_and_fetch(&var->counter, (ETHR_AINT_T__) 0); +#endif } -#define ethr_native_atomic_inc_return_acqb ethr_native_atomic_inc_return -#define ethr_native_atomic_set_relb ethr_native_atomic_xchg -#define ethr_native_atomic_dec_relb ethr_native_atomic_dec_return -#define ethr_native_atomic_dec_return_relb ethr_native_atomic_dec_return +static ETHR_INLINE void +ETHR_NATMC_FUNC__(set_relb)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ i) +{ +#if ETHR_READ_ACQB_AND_SET_RELB_COMPILER_BARRIER_ONLY__ + ETHR_COMPILER_BARRIER; + var->counter = i; +#else + (void) ETHR_NATMC_FUNC__(xchg)(var, i); +#endif +} -#define ethr_native_atomic_cmpxchg_acqb ethr_native_atomic_cmpxchg -#define ethr_native_atomic_cmpxchg_relb ethr_native_atomic_cmpxchg +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(inc_return_acqb)(ETHR_ATMC_T__ *var) +{ + return ETHR_NATMC_FUNC__(inc_return)(var); +} -#endif +static ETHR_INLINE void +ETHR_NATMC_FUNC__(dec_relb)(ETHR_ATMC_T__ *var) +{ + ETHR_NATMC_FUNC__(dec)(var); +} + +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(dec_return_relb)(ETHR_ATMC_T__ *var) +{ + return ETHR_NATMC_FUNC__(dec_return)(var); +} + +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg_acqb)(ETHR_ATMC_T__ *var, + ETHR_AINT_T__ new, + ETHR_AINT_T__ old) +{ + return ETHR_NATMC_FUNC__(cmpxchg)(var, new, old); +} + +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg_relb)(ETHR_ATMC_T__ *var, + ETHR_AINT_T__ new, + ETHR_AINT_T__ old) +{ + return ETHR_NATMC_FUNC__(cmpxchg)(var, new, old); +} #endif +#undef ETHR_NATMC_FUNC__ +#undef ETHR_ATMC_T__ +#undef ETHR_AINT_T__ +#undef ETHR_AINT_SUFFIX__ + #endif diff --git a/erts/include/internal/gcc/ethread.h b/erts/include/internal/gcc/ethread.h index bb378e31e0..392a1aa2b2 100644 --- a/erts/include/internal/gcc/ethread.h +++ b/erts/include/internal/gcc/ethread.h @@ -25,6 +25,16 @@ #ifndef ETHREAD_GCC_H__ #define ETHREAD_GCC_H__ +#if !defined(ETHR_HAVE_NATIVE_ATOMICS) && defined(ETHR_HAVE_GCC_ATOMIC_OPS) +#define ETHR_HAVE_NATIVE_ATOMICS 1 + +#define ETHR_ATOMIC_WANT_32BIT_IMPL__ #include "ethr_atomic.h" +#if ETHR_SIZEOF_PTR == 8 +# define ETHR_ATOMIC_WANT_64BIT_IMPL__ +# include "ethr_atomic.h" +#endif + +#endif #endif diff --git a/erts/include/internal/i386/atomic.h b/erts/include/internal/i386/atomic.h index f28258059f..4e402f261a 100644 --- a/erts/include/internal/i386/atomic.h +++ b/erts/include/internal/i386/atomic.h @@ -23,14 +23,24 @@ * * This code requires a 486 or newer processor. */ -#ifndef ETHREAD_I386_ATOMIC_H -#define ETHREAD_I386_ATOMIC_H -/* An atomic is an aligned long accessed via locked operations. - */ -typedef struct { - volatile long counter; -} ethr_native_atomic_t; +#undef ETHR_INCLUDE_ATOMIC_IMPL__ +#if !defined(ETHR_X86_ATOMIC32_H__) && defined(ETHR_ATOMIC_WANT_32BIT_IMPL__) +#define ETHR_X86_ATOMIC32_H__ +#define ETHR_INCLUDE_ATOMIC_IMPL__ 4 +#undef ETHR_ATOMIC_WANT_32BIT_IMPL__ +#elif !defined(ETHR_X86_ATOMIC64_H__) && defined(ETHR_ATOMIC_WANT_64BIT_IMPL__) +#define ETHR_X86_ATOMIC64_H__ +#define ETHR_INCLUDE_ATOMIC_IMPL__ 8 +#undef ETHR_ATOMIC_WANT_64BIT_IMPL__ +#endif + +#ifdef ETHR_INCLUDE_ATOMIC_IMPL__ + +#ifndef ETHR_X86_ATOMIC_COMMON__ +#define ETHR_X86_ATOMIC_COMMON__ + +#define ETHR_ATOMIC_HAVE_INC_DEC_INSTRUCTIONS 1 #if defined(__x86_64__) || !defined(ETHR_PRE_PENTIUM4_COMPAT) #define ETHR_MEMORY_BARRIER __asm__ __volatile__("mfence" : : : "memory") @@ -40,123 +50,161 @@ typedef struct { #else #define ETHR_MEMORY_BARRIER \ do { \ - volatile long x___ = 0; \ + volatile ethr_sint32_t x___ = 0; \ __asm__ __volatile__("lock; incl %0" : "=m"(x___) : "m"(x___) : "memory"); \ } while (0) #endif -#define ETHR_ATOMIC_HAVE_INC_DEC_INSTRUCTIONS 1 - -#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_AUX_IMPL__) +#endif /* ETHR_X86_ATOMIC_COMMON__ */ -#ifdef __x86_64__ -#define LONG_SUFFIX "q" +#if ETHR_INCLUDE_ATOMIC_IMPL__ == 4 +#define ETHR_HAVE_NATIVE_ATOMIC32 1 +#define ETHR_NATMC_FUNC__(X) ethr_native_atomic32_ ## X +#define ETHR_ATMC_T__ ethr_native_atomic32_t +#define ETHR_AINT_T__ ethr_sint32_t +#define ETHR_AINT_SUFFIX__ "l" +#elif ETHR_INCLUDE_ATOMIC_IMPL__ == 8 +#define ETHR_HAVE_NATIVE_ATOMIC64 1 +#define ETHR_NATMC_FUNC__(X) ethr_native_atomic64_ ## X +#define ETHR_ATMC_T__ ethr_native_atomic64_t +#define ETHR_AINT_T__ ethr_sint64_t +#define ETHR_AINT_SUFFIX__ "q" #else -#define LONG_SUFFIX "l" +#error "Unsupported integer size" #endif +/* An atomic is an aligned ETHR_AINT_T__ accessed via locked operations. + */ +typedef struct { + volatile ETHR_AINT_T__ counter; +} ETHR_ATMC_T__; + +#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_ATOMIC_IMPL__) + +static ETHR_INLINE ETHR_AINT_T__ * +ETHR_NATMC_FUNC__(addr)(ETHR_ATMC_T__ *var) +{ + return (ETHR_AINT_T__ *) &var->counter; +} + static ETHR_INLINE void -ethr_native_atomic_init(ethr_native_atomic_t *var, long i) +ETHR_NATMC_FUNC__(init)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ i) { var->counter = i; } -#define ethr_native_atomic_set(v, i) ethr_native_atomic_init((v), (i)) -static ETHR_INLINE long -ethr_native_atomic_read(ethr_native_atomic_t *var) +static ETHR_INLINE void +ETHR_NATMC_FUNC__(set)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ i) +{ + var->counter = i; +} + +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(read)(ETHR_ATMC_T__ *var) { return var->counter; } static ETHR_INLINE void -ethr_native_atomic_add(ethr_native_atomic_t *var, long incr) +ETHR_NATMC_FUNC__(add)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ incr) { __asm__ __volatile__( - "lock; add" LONG_SUFFIX " %1, %0" + "lock; add" ETHR_AINT_SUFFIX__ " %1, %0" : "=m"(var->counter) : "ir"(incr), "m"(var->counter)); } static ETHR_INLINE void -ethr_native_atomic_inc(ethr_native_atomic_t *var) +ETHR_NATMC_FUNC__(inc)(ETHR_ATMC_T__ *var) { __asm__ __volatile__( - "lock; inc" LONG_SUFFIX " %0" + "lock; inc" ETHR_AINT_SUFFIX__ " %0" : "=m"(var->counter) : "m"(var->counter)); } static ETHR_INLINE void -ethr_native_atomic_dec(ethr_native_atomic_t *var) +ETHR_NATMC_FUNC__(dec)(ETHR_ATMC_T__ *var) { __asm__ __volatile__( - "lock; dec" LONG_SUFFIX " %0" + "lock; dec" ETHR_AINT_SUFFIX__ " %0" : "=m"(var->counter) : "m"(var->counter)); } -static ETHR_INLINE long -ethr_native_atomic_add_return(ethr_native_atomic_t *var, long incr) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(add_return)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ incr) { - long tmp; + ETHR_AINT_T__ tmp; tmp = incr; __asm__ __volatile__( - "lock; xadd" LONG_SUFFIX " %0, %1" /* xadd didn't exist prior to the 486 */ + "lock; xadd" ETHR_AINT_SUFFIX__ " %0, %1" /* xadd didn't exist prior to the 486 */ : "=r"(tmp) : "m"(var->counter), "0"(tmp)); /* now tmp is the atomic's previous value */ return tmp + incr; } -#define ethr_native_atomic_inc_return(var) ethr_native_atomic_add_return((var), 1) -#define ethr_native_atomic_dec_return(var) ethr_native_atomic_add_return((var), -1) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(inc_return)(ETHR_ATMC_T__ *var) +{ + return ETHR_NATMC_FUNC__(add_return)(var, (ETHR_AINT_T__) 1); +} + +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(dec_return)(ETHR_ATMC_T__ *var) +{ + return ETHR_NATMC_FUNC__(add_return)(var, (ETHR_AINT_T__) -1); +} -static ETHR_INLINE long -ethr_native_atomic_cmpxchg(ethr_native_atomic_t *var, long new, long old) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg)(ETHR_ATMC_T__ *var, + ETHR_AINT_T__ new, + ETHR_AINT_T__ old) { __asm__ __volatile__( - "lock; cmpxchg" LONG_SUFFIX " %2, %3" + "lock; cmpxchg" ETHR_AINT_SUFFIX__ " %2, %3" : "=a"(old), "=m"(var->counter) : "r"(new), "m"(var->counter), "0"(old) : "cc", "memory"); /* full memory clobber to make this a compiler barrier */ return old; } -static ETHR_INLINE long -ethr_native_atomic_and_retold(ethr_native_atomic_t *var, long mask) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(and_retold)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ mask) { - long tmp, old; + ETHR_AINT_T__ tmp, old; tmp = var->counter; do { old = tmp; - tmp = ethr_native_atomic_cmpxchg(var, tmp & mask, tmp); + tmp = ETHR_NATMC_FUNC__(cmpxchg)(var, tmp & mask, tmp); } while (__builtin_expect(tmp != old, 0)); /* now tmp is the atomic's previous value */ return tmp; } -static ETHR_INLINE long -ethr_native_atomic_or_retold(ethr_native_atomic_t *var, long mask) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(or_retold)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ mask) { - long tmp, old; + ETHR_AINT_T__ tmp, old; tmp = var->counter; do { old = tmp; - tmp = ethr_native_atomic_cmpxchg(var, tmp | mask, tmp); + tmp = ETHR_NATMC_FUNC__(cmpxchg)(var, tmp | mask, tmp); } while (__builtin_expect(tmp != old, 0)); /* now tmp is the atomic's previous value */ return tmp; } -static ETHR_INLINE long -ethr_native_atomic_xchg(ethr_native_atomic_t *var, long val) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(xchg)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ val) { - long tmp = val; + ETHR_AINT_T__ tmp = val; __asm__ __volatile__( - "xchg" LONG_SUFFIX " %0, %1" + "xchg" ETHR_AINT_SUFFIX__ " %0, %1" : "=r"(tmp) : "m"(var->counter), "0"(tmp)); /* now tmp is the atomic's previous value */ @@ -167,20 +215,73 @@ ethr_native_atomic_xchg(ethr_native_atomic_t *var, long val) * Atomic ops with at least specified barriers. */ -#define ethr_native_atomic_read_acqb ethr_native_atomic_read -#define ethr_native_atomic_inc_return_acqb ethr_native_atomic_inc_return +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(read_acqb)(ETHR_ATMC_T__ *var) +{ + ETHR_AINT_T__ val; #if defined(__x86_64__) || !defined(ETHR_PRE_PENTIUM4_COMPAT) -#define ethr_native_atomic_set_relb ethr_native_atomic_set + val = var->counter; #else -#define ethr_native_atomic_set_relb ethr_native_atomic_xchg + val = ETHR_NATMC_FUNC__(add_return)(var, 0); #endif -#define ethr_native_atomic_dec_relb ethr_native_atomic_dec -#define ethr_native_atomic_dec_return_relb ethr_native_atomic_dec_return -#define ethr_native_atomic_cmpxchg_acqb ethr_native_atomic_cmpxchg -#define ethr_native_atomic_cmpxchg_relb ethr_native_atomic_cmpxchg + __asm__ __volatile__("" : : : "memory"); + return val; +} + +static ETHR_INLINE void +ETHR_NATMC_FUNC__(set_relb)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ i) +{ + __asm__ __volatile__("" : : : "memory"); +#if defined(__x86_64__) || !defined(ETHR_PRE_PENTIUM4_COMPAT) + var->counter = i; +#else + (void) ETHR_NATMC_FUNC__(xchg)(var, i); +#endif +} + +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(inc_return_acqb)(ETHR_ATMC_T__ *var) +{ + ETHR_AINT_T__ res = ETHR_NATMC_FUNC__(inc_return)(var); + __asm__ __volatile__("" : : : "memory"); + return res; +} -#undef LONG_SUFFIX +static ETHR_INLINE void +ETHR_NATMC_FUNC__(dec_relb)(ETHR_ATMC_T__ *var) +{ + __asm__ __volatile__("" : : : "memory"); + ETHR_NATMC_FUNC__(dec)(var); +} + +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(dec_return_relb)(ETHR_ATMC_T__ *var) +{ + __asm__ __volatile__("" : : : "memory"); + return ETHR_NATMC_FUNC__(dec_return)(var); +} + +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg_acqb)(ETHR_ATMC_T__ *var, + ETHR_AINT_T__ new, + ETHR_AINT_T__ old) +{ + return ETHR_NATMC_FUNC__(cmpxchg)(var, new, old); +} + +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg_relb)(ETHR_ATMC_T__ *var, + ETHR_AINT_T__ new, + ETHR_AINT_T__ old) +{ + return ETHR_NATMC_FUNC__(cmpxchg)(var, new, old); +} #endif /* ETHR_TRY_INLINE_FUNCS */ -#endif /* ETHREAD_I386_ATOMIC_H */ +#undef ETHR_NATMC_FUNC__ +#undef ETHR_ATMC_T__ +#undef ETHR_AINT_T__ +#undef ETHR_AINT_SUFFIX__ + +#endif /* ETHR_INCLUDE_ATOMIC_IMPL__ */ diff --git a/erts/include/internal/i386/ethread.h b/erts/include/internal/i386/ethread.h index ed43e77279..b5a17caefb 100644 --- a/erts/include/internal/i386/ethread.h +++ b/erts/include/internal/i386/ethread.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2005-2009. All Rights Reserved. + * Copyright Ericsson AB 2005-2010. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -24,7 +24,12 @@ #ifndef ETHREAD_I386_ETHREAD_H #define ETHREAD_I386_ETHREAD_H +#define ETHR_ATOMIC_WANT_32BIT_IMPL__ #include "atomic.h" +#if ETHR_SIZEOF_PTR == 8 +# define ETHR_ATOMIC_WANT_64BIT_IMPL__ +# include "atomic.h" +#endif #include "spinlock.h" #include "rwlock.h" diff --git a/erts/include/internal/libatomic_ops/ethr_atomic.h b/erts/include/internal/libatomic_ops/ethr_atomic.h index a6eb43a0bd..d56693dbf8 100644 --- a/erts/include/internal/libatomic_ops/ethr_atomic.h +++ b/erts/include/internal/libatomic_ops/ethr_atomic.h @@ -46,17 +46,39 @@ * - AO_store() * - AO_compare_and_swap() * - * The `AO_t' type also have to be at least as large as - * `void *' and `long' types. + * The `AO_t' type also have to be at least as large as the `void *' type. */ #if ETHR_SIZEOF_AO_T < ETHR_SIZEOF_PTR #error The AO_t type is too small #endif +#if ETHR_SIZEOF_AO_T == 4 +#define ETHR_HAVE_NATIVE_ATOMIC32 1 +#define ETHR_NATMC_FUNC__(X) ethr_native_atomic32_ ## X +#define ETHR_ATMC_T__ ethr_native_atomic32_t +#define ETHR_AINT_T__ ethr_sint32_t +#define ETHR_AINT_SUFFIX__ "l" +#elif ETHR_SIZEOF_AO_T == 8 +#define ETHR_HAVE_NATIVE_ATOMIC64 1 +#define ETHR_NATMC_FUNC__(X) ethr_native_atomic64_ ## X +#define ETHR_ATMC_T__ ethr_native_atomic64_t +#define ETHR_AINT_T__ ethr_sint64_t +#define ETHR_AINT_SUFFIX__ "q" +#else +#error "Unsupported integer size" +#endif + +#if ETHR_SIZEOF_AO_T == 8 +typedef union { + volatile AO_t counter; + ethr_sint32_t sint32[2]; +} ETHR_ATMC_T__; +#else typedef struct { volatile AO_t counter; -} ethr_native_atomic_t; +} ETHR_ATMC_T__; +#endif #define ETHR_MEMORY_BARRIER AO_nop_full() #ifdef AO_HAVE_nop_write @@ -72,123 +94,151 @@ typedef struct { #ifdef AO_NO_DD_ORDERING # define ETHR_READ_DEPEND_MEMORY_BARRIER ETHR_READ_MEMORY_BARRIER #else -# define ETHR_READ_DEPEND_MEMORY_BARRIER __asm__ __volatile__("":::"memory") +# define ETHR_READ_DEPEND_MEMORY_BARRIER AO_compiler_barrier() +#endif + +#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_ATOMIC_IMPL__) + +static ETHR_INLINE ETHR_AINT_T__ * +ETHR_NATMC_FUNC__(addr)(ETHR_ATMC_T__ *var) +{ + return (ETHR_AINT_T__ *) &var->counter; +} + +#if ETHR_SIZEOF_AO_T == 8 +/* + * We also need to provide an ethr_native_atomic32_addr(), since + * this 64-bit implementation will be used implementing 32-bit + * native atomics. + */ + +static ETHR_INLINE ethr_sint32_t * +ethr_native_atomic32_addr(ETHR_ATMC_T__ *var) +{ + ETHR_ASSERT(((void *) &var->sint32[0]) == ((void *) &var->counter)); +#ifdef ETHR_BIGENDIAN + return &var->sint32[1]; +#else + return &var->sint32[0]; #endif +} -#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_AUX_IMPL__) +#endif /* ETHR_SIZEOF_AO_T == 8 */ static ETHR_INLINE void -ethr_native_atomic_set(ethr_native_atomic_t *var, long value) +ETHR_NATMC_FUNC__(set)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ value) { AO_store(&var->counter, (AO_t) value); } static ETHR_INLINE void -ethr_native_atomic_init(ethr_native_atomic_t *var, long value) +ETHR_NATMC_FUNC__(init)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ value) { - ethr_native_atomic_set(var, value); + ETHR_NATMC_FUNC__(set)(var, value); } -static ETHR_INLINE long -ethr_native_atomic_read(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(read)(ETHR_ATMC_T__ *var) { - return (long) AO_load(&var->counter); + return (ETHR_AINT_T__) AO_load(&var->counter); } -static ETHR_INLINE long -ethr_native_atomic_add_return(ethr_native_atomic_t *var, long incr) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(add_return)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ incr) { #ifdef AO_HAVE_fetch_and_add - return ((long) AO_fetch_and_add(&var->counter, (AO_t) incr)) + incr; + return ((ETHR_AINT_T__) AO_fetch_and_add(&var->counter, (AO_t) incr)) + incr; #else while (1) { AO_t exp = AO_load(&var->counter); AO_t new = exp + (AO_t) incr; if (AO_compare_and_swap(&var->counter, exp, new)) - return (long) new; + return (ETHR_AINT_T__) new; } #endif } static ETHR_INLINE void -ethr_native_atomic_add(ethr_native_atomic_t *var, long incr) +ETHR_NATMC_FUNC__(add)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ incr) { - (void) ethr_native_atomic_add_return(var, incr); + (void) ETHR_NATMC_FUNC__(add_return)(var, incr); } -static ETHR_INLINE long -ethr_native_atomic_inc_return(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(inc_return)(ETHR_ATMC_T__ *var) { #ifdef AO_HAVE_fetch_and_add1 - return ((long) AO_fetch_and_add1(&var->counter)) + 1; + return ((ETHR_AINT_T__) AO_fetch_and_add1(&var->counter)) + 1; #else - return ethr_native_atomic_add_return(var, 1); + return ETHR_NATMC_FUNC__(add_return)(var, 1); #endif } static ETHR_INLINE void -ethr_native_atomic_inc(ethr_native_atomic_t *var) +ETHR_NATMC_FUNC__(inc)(ETHR_ATMC_T__ *var) { - (void) ethr_native_atomic_inc_return(var); + (void) ETHR_NATMC_FUNC__(inc_return)(var); } -static ETHR_INLINE long -ethr_native_atomic_dec_return(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(dec_return)(ETHR_ATMC_T__ *var) { #ifdef AO_HAVE_fetch_and_sub1 - return ((long) AO_fetch_and_sub1(&var->counter)) - 1; + return ((ETHR_AINT_T__) AO_fetch_and_sub1(&var->counter)) - 1; #else - return ethr_native_atomic_add_return(var, -1); + return ETHR_NATMC_FUNC__(add_return)(var, -1); #endif } static ETHR_INLINE void -ethr_native_atomic_dec(ethr_native_atomic_t *var) +ETHR_NATMC_FUNC__(dec)(ETHR_ATMC_T__ *var) { - (void) ethr_native_atomic_dec_return(var); + (void) ETHR_NATMC_FUNC__(dec_return)(var); } -static ETHR_INLINE long -ethr_native_atomic_and_retold(ethr_native_atomic_t *var, long mask) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(and_retold)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ mask) { while (1) { AO_t exp = AO_load(&var->counter); AO_t new = exp & ((AO_t) mask); if (AO_compare_and_swap(&var->counter, exp, new)) - return (long) exp; + return (ETHR_AINT_T__) exp; } } -static ETHR_INLINE long -ethr_native_atomic_or_retold(ethr_native_atomic_t *var, long mask) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(or_retold)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ mask) { while (1) { AO_t exp = AO_load(&var->counter); AO_t new = exp | ((AO_t) mask); if (AO_compare_and_swap(&var->counter, exp, new)) - return (long) exp; + return (ETHR_AINT_T__) exp; } } -static ETHR_INLINE long -ethr_native_atomic_cmpxchg(ethr_native_atomic_t *var, long new, long exp) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg)(ETHR_ATMC_T__ *var, + ETHR_AINT_T__ new, + ETHR_AINT_T__ exp) { - long act; + ETHR_AINT_T__ act; do { if (AO_compare_and_swap(&var->counter, (AO_t) exp, (AO_t) new)) return exp; - act = (long) AO_load(&var->counter); + act = (ETHR_AINT_T__) AO_load(&var->counter); } while (act == exp); return act; } -static ETHR_INLINE long -ethr_native_atomic_xchg(ethr_native_atomic_t *var, long new) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(xchg)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ new) { while (1) { AO_t exp = AO_load(&var->counter); if (AO_compare_and_swap(&var->counter, exp, (AO_t) new)) - return (long) exp; + return (ETHR_AINT_T__) exp; } } @@ -196,97 +246,105 @@ ethr_native_atomic_xchg(ethr_native_atomic_t *var, long new) * Atomic ops with at least specified barriers. */ -static ETHR_INLINE long -ethr_native_atomic_read_acqb(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(read_acqb)(ETHR_ATMC_T__ *var) { #ifdef AO_HAVE_load_acquire - return (long) AO_load_acquire(&var->counter); + return (ETHR_AINT_T__) AO_load_acquire(&var->counter); #else - long res = ethr_native_atomic_read(var); + ETHR_AINT_T__ res = ETHR_NATMC_FUNC__(read)(var); ETHR_MEMORY_BARRIER; return res; #endif } -static ETHR_INLINE long -ethr_native_atomic_inc_return_acqb(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(inc_return_acqb)(ETHR_ATMC_T__ *var) { #ifdef AO_HAVE_fetch_and_add1_acquire - return ((long) AO_fetch_and_add1_acquire(&var->counter)) + 1; + return ((ETHR_AINT_T__) AO_fetch_and_add1_acquire(&var->counter)) + 1; #else - long res = ethr_native_atomic_add_return(var, 1); + ETHR_AINT_T__ res = ETHR_NATMC_FUNC__(add_return)(var, 1); ETHR_MEMORY_BARRIER; return res; #endif } static ETHR_INLINE void -ethr_native_atomic_set_relb(ethr_native_atomic_t *var, long value) +ETHR_NATMC_FUNC__(set_relb)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ value) { #ifdef AO_HAVE_store_release AO_store_release(&var->counter, (AO_t) value); #else ETHR_MEMORY_BARRIER; - ethr_native_atomic_set(var, value); + ETHR_NATMC_FUNC__(set)(var, value); #endif } -static ETHR_INLINE long -ethr_native_atomic_dec_return_relb(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(dec_return_relb)(ETHR_ATMC_T__ *var) { #ifdef AO_HAVE_fetch_and_sub1_release - return ((long) AO_fetch_and_sub1_release(&var->counter)) - 1; + return ((ETHR_AINT_T__) AO_fetch_and_sub1_release(&var->counter)) - 1; #else ETHR_MEMORY_BARRIER; - return ethr_native_atomic_dec_return(var); + return ETHR_NATMC_FUNC__(dec_return)(var); #endif } static ETHR_INLINE void -ethr_native_atomic_dec_relb(ethr_native_atomic_t *var) +ETHR_NATMC_FUNC__(dec_relb)(ETHR_ATMC_T__ *var) { - (void) ethr_native_atomic_dec_return_relb(var); + (void) ETHR_NATMC_FUNC__(dec_return_relb)(var); } -static ETHR_INLINE long -ethr_native_atomic_cmpxchg_acqb(ethr_native_atomic_t *var, long new, long exp) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg_acqb)(ETHR_ATMC_T__ *var, + ETHR_AINT_T__ new, + ETHR_AINT_T__ exp) { #ifdef AO_HAVE_compare_and_swap_acquire - long act; + ETHR_AINT_T__ act; do { if (AO_compare_and_swap_acquire(&var->counter, (AO_t) exp, (AO_t) new)) return exp; - act = (long) AO_load(&var->counter); + act = (ETHR_AINT_T__) AO_load(&var->counter); } while (act == exp); AO_nop_full(); return act; #else - long act = ethr_native_atomic_cmpxchg(var, new, exp); + ETHR_AINT_T__ act = ETHR_NATMC_FUNC__(cmpxchg)(var, new, exp); ETHR_MEMORY_BARRIER; return act; #endif } -static ETHR_INLINE long -ethr_native_atomic_cmpxchg_relb(ethr_native_atomic_t *var, long new, long exp) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg_relb)(ETHR_ATMC_T__ *var, + ETHR_AINT_T__ new, + ETHR_AINT_T__ exp) { #ifdef AO_HAVE_compare_and_swap_release - long act; + ETHR_AINT_T__ act; do { if (AO_compare_and_swap_release(&var->counter, (AO_t) exp, (AO_t) new)) return exp; - act = (long) AO_load(&var->counter); + act = (ETHR_AINT_T__) AO_load(&var->counter); } while (act == exp); return act; #else ETHR_MEMORY_BARRIER; - return ethr_native_atomic_cmpxchg(var, new, exp); + return ETHR_NATMC_FUNC__(cmpxchg)(var, new, exp); #endif } -#endif +#endif /* ETHR_TRY_INLINE_FUNCS */ -#endif +#undef ETHR_NATMC_FUNC__ +#undef ETHR_ATMC_T__ +#undef ETHR_AINT_T__ -#endif +#endif /* !defined(ETHR_HAVE_NATIVE_ATOMICS) && defined(ETHR_HAVE_LIBATOMIC_OPS) */ + +#endif /* ETHR_LIBATOMIC_OPS_ATOMIC_H__ */ diff --git a/erts/include/internal/ppc32/atomic.h b/erts/include/internal/ppc32/atomic.h index f21f7c9588..522f433649 100644 --- a/erts/include/internal/ppc32/atomic.h +++ b/erts/include/internal/ppc32/atomic.h @@ -28,31 +28,39 @@ #ifndef ETHREAD_PPC_ATOMIC_H #define ETHREAD_PPC_ATOMIC_H +#define ETHR_HAVE_NATIVE_ATOMIC32 1 + typedef struct { - volatile int counter; -} ethr_native_atomic_t; + volatile ethr_sint32_t counter; +} ethr_native_atomic32_t; #define ETHR_MEMORY_BARRIER __asm__ __volatile__("sync" : : : "memory") -#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_AUX_IMPL__) +#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_ATOMIC_IMPL__) + +static ETHR_INLINE ethr_sint32_t * +ethr_native_atomic32_addr(ethr_native_atomic32_t *var) +{ + return (ethr_sint32_t *) &var->counter; +} static ETHR_INLINE void -ethr_native_atomic_init(ethr_native_atomic_t *var, int i) +ethr_native_atomic32_init(ethr_native_atomic32_t *var, ethr_sint32_t i) { var->counter = i; } -#define ethr_native_atomic_set(v, i) ethr_native_atomic_init((v), (i)) +#define ethr_native_atomic32_set(v, i) ethr_native_atomic32_init((v), (i)) -static ETHR_INLINE int -ethr_native_atomic_read(ethr_native_atomic_t *var) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_read(ethr_native_atomic32_t *var) { return var->counter; } -static ETHR_INLINE int -ethr_native_atomic_add_return(ethr_native_atomic_t *var, int incr) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_add_return(ethr_native_atomic32_t *var, ethr_sint32_t incr) { - int tmp; + ethr_sint32_t tmp; __asm__ __volatile__( "eieio\n\t" @@ -69,16 +77,16 @@ ethr_native_atomic_add_return(ethr_native_atomic_t *var, int incr) } static ETHR_INLINE void -ethr_native_atomic_add(ethr_native_atomic_t *var, int incr) +ethr_native_atomic32_add(ethr_native_atomic32_t *var, ethr_sint32_t incr) { /* XXX: could use weaker version here w/o eieio+isync */ - (void)ethr_native_atomic_add_return(var, incr); + (void)ethr_native_atomic32_add_return(var, incr); } -static ETHR_INLINE int -ethr_native_atomic_inc_return(ethr_native_atomic_t *var) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_inc_return(ethr_native_atomic32_t *var) { - int tmp; + ethr_sint32_t tmp; __asm__ __volatile__( "eieio\n\t" @@ -95,16 +103,16 @@ ethr_native_atomic_inc_return(ethr_native_atomic_t *var) } static ETHR_INLINE void -ethr_native_atomic_inc(ethr_native_atomic_t *var) +ethr_native_atomic32_inc(ethr_native_atomic32_t *var) { /* XXX: could use weaker version here w/o eieio+isync */ - (void)ethr_native_atomic_inc_return(var); + (void)ethr_native_atomic32_inc_return(var); } -static ETHR_INLINE int -ethr_native_atomic_dec_return(ethr_native_atomic_t *var) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_dec_return(ethr_native_atomic32_t *var) { - int tmp; + ethr_sint32_t tmp; __asm__ __volatile__( "eieio\n\t" @@ -121,16 +129,16 @@ ethr_native_atomic_dec_return(ethr_native_atomic_t *var) } static ETHR_INLINE void -ethr_native_atomic_dec(ethr_native_atomic_t *var) +ethr_native_atomic32_dec(ethr_native_atomic32_t *var) { /* XXX: could use weaker version here w/o eieio+isync */ - (void)ethr_native_atomic_dec_return(var); + (void)ethr_native_atomic32_dec_return(var); } -static ETHR_INLINE int -ethr_native_atomic_and_retold(ethr_native_atomic_t *var, int mask) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_and_retold(ethr_native_atomic32_t *var, ethr_sint32_t mask) { - int old, new; + ethr_sint32_t old, new; __asm__ __volatile__( "eieio\n\t" @@ -146,10 +154,10 @@ ethr_native_atomic_and_retold(ethr_native_atomic_t *var, int mask) return old; } -static ETHR_INLINE int -ethr_native_atomic_or_retold(ethr_native_atomic_t *var, int mask) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_or_retold(ethr_native_atomic32_t *var, ethr_sint32_t mask) { - int old, new; + ethr_sint32_t old, new; __asm__ __volatile__( "eieio\n\t" @@ -165,10 +173,10 @@ ethr_native_atomic_or_retold(ethr_native_atomic_t *var, int mask) return old; } -static ETHR_INLINE int -ethr_native_atomic_xchg(ethr_native_atomic_t *var, int val) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_xchg(ethr_native_atomic32_t *var, ethr_sint32_t val) { - int tmp; + ethr_sint32_t tmp; __asm__ __volatile__( "eieio\n\t" @@ -183,10 +191,12 @@ ethr_native_atomic_xchg(ethr_native_atomic_t *var, int val) return tmp; } -static ETHR_INLINE int -ethr_native_atomic_cmpxchg(ethr_native_atomic_t *var, int new, int expected) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_cmpxchg(ethr_native_atomic32_t *var, + ethr_sint32_t new, + ethr_sint32_t expected) { - int old; + ethr_sint32_t old; __asm__ __volatile__( "eieio\n\t" @@ -210,20 +220,20 @@ ethr_native_atomic_cmpxchg(ethr_native_atomic_t *var, int new, int expected) */ static ETHR_INLINE long -ethr_native_atomic_read_acqb(ethr_native_atomic_t *var) +ethr_native_atomic32_read_acqb(ethr_native_atomic32_t *var) { - long res = ethr_native_atomic_read(var); + long res = ethr_native_atomic32_read(var); ETHR_MEMORY_BARRIER; return res; } -#define ethr_native_atomic_set_relb ethr_native_atomic_xchg -#define ethr_native_atomic_inc_return_acqb ethr_native_atomic_inc_return -#define ethr_native_atomic_dec_relb ethr_native_atomic_dec_return -#define ethr_native_atomic_dec_return_relb ethr_native_atomic_dec_return +#define ethr_native_atomic32_set_relb ethr_native_atomic32_xchg +#define ethr_native_atomic32_inc_return_acqb ethr_native_atomic32_inc_return +#define ethr_native_atomic32_dec_relb ethr_native_atomic32_dec_return +#define ethr_native_atomic32_dec_return_relb ethr_native_atomic32_dec_return -#define ethr_native_atomic_cmpxchg_acqb ethr_native_atomic_cmpxchg -#define ethr_native_atomic_cmpxchg_relb ethr_native_atomic_cmpxchg +#define ethr_native_atomic32_cmpxchg_acqb ethr_native_atomic32_cmpxchg +#define ethr_native_atomic32_cmpxchg_relb ethr_native_atomic32_cmpxchg #endif /* ETHR_TRY_INLINE_FUNCS */ diff --git a/erts/include/internal/pthread/ethr_event.h b/erts/include/internal/pthread/ethr_event.h index 104ec287e0..93da8a0429 100644 --- a/erts/include/internal/pthread/ethr_event.h +++ b/erts/include/internal/pthread/ethr_event.h @@ -30,31 +30,9 @@ #include <linux/futex.h> #include <sys/time.h> -/* - * Note: Linux futexes operate on 32-bit integers, but - * ethr_native_atomic_t are 64-bits on 64-bit - * platforms. This has to be taken into account. - * Therefore, in each individual value used each - * byte look the same. - */ - -#if ETHR_SIZEOF_PTR == 8 - -#define ETHR_EVENT_OFF_WAITER__ 0xffffffffffffffffL -#define ETHR_EVENT_OFF__ 0x7777777777777777L -#define ETHR_EVENT_ON__ 0L - -#elif ETHR_SIZEOF_PTR == 4 - -#define ETHR_EVENT_OFF_WAITER__ 0xffffffffL -#define ETHR_EVENT_OFF__ 0x77777777L -#define ETHR_EVENT_ON__ 0L - -#else - -#error ehrm... - -#endif +#define ETHR_EVENT_OFF_WAITER__ ((ethr_sint32_t) -1) +#define ETHR_EVENT_OFF__ ((ethr_sint32_t) 1) +#define ETHR_EVENT_ON__ ((ethr_sint32_t) 0) #if defined(FUTEX_WAIT_PRIVATE) && defined(FUTEX_WAKE_PRIVATE) # define ETHR_FUTEX_WAIT__ FUTEX_WAIT_PRIVATE @@ -65,11 +43,17 @@ #endif typedef struct { - ethr_atomic_t futex; + ethr_atomic32_t futex; } ethr_event; -#define ETHR_FUTEX__(FTX, OP, VAL) \ - (-1 == syscall(__NR_futex, (void *) (FTX), (OP), (int) (VAL), NULL, NULL, 0)\ +#define ETHR_FUTEX__(FTX, OP, VAL) \ + (-1 == syscall(__NR_futex, \ + (void *) ethr_atomic32_addr((FTX)), \ + (OP), \ + (int) (VAL), \ + NULL, \ + NULL, \ + 0) \ ? errno : 0) #if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_EVENT_IMPL__) @@ -77,9 +61,9 @@ typedef struct { static void ETHR_INLINE ETHR_INLINE_FUNC_NAME_(ethr_event_set)(ethr_event *e) { - long val; + ethr_sint32_t val; ETHR_WRITE_MEMORY_BARRIER; - val = ethr_atomic_xchg(&e->futex, ETHR_EVENT_ON__); + val = ethr_atomic32_xchg(&e->futex, ETHR_EVENT_ON__); if (val == ETHR_EVENT_OFF_WAITER__) { int res = ETHR_FUTEX__(&e->futex, ETHR_FUTEX_WAKE__, 1); if (res != 0) @@ -90,7 +74,7 @@ ETHR_INLINE_FUNC_NAME_(ethr_event_set)(ethr_event *e) static void ETHR_INLINE ETHR_INLINE_FUNC_NAME_(ethr_event_reset)(ethr_event *e) { - ethr_atomic_set(&e->futex, ETHR_EVENT_OFF__); + ethr_atomic32_set(&e->futex, ETHR_EVENT_OFF__); ETHR_MEMORY_BARRIER; } @@ -100,7 +84,7 @@ ETHR_INLINE_FUNC_NAME_(ethr_event_reset)(ethr_event *e) /* --- Posix mutex/cond implementation of events ---------------------------- */ typedef struct { - ethr_atomic_t state; + ethr_atomic32_t state; pthread_mutex_t mtx; pthread_cond_t cnd; } ethr_event; @@ -114,9 +98,9 @@ typedef struct { static void ETHR_INLINE ETHR_INLINE_FUNC_NAME_(ethr_event_set)(ethr_event *e) { - long val; + ethr_sint32_t val; ETHR_WRITE_MEMORY_BARRIER; - val = ethr_atomic_xchg(&e->state, ETHR_EVENT_ON__); + val = ethr_atomic32_xchg(&e->state, ETHR_EVENT_ON__); if (val == ETHR_EVENT_OFF_WAITER__) { int res = pthread_mutex_lock(&e->mtx); if (res != 0) @@ -133,7 +117,7 @@ ETHR_INLINE_FUNC_NAME_(ethr_event_set)(ethr_event *e) static void ETHR_INLINE ETHR_INLINE_FUNC_NAME_(ethr_event_reset)(ethr_event *e) { - ethr_atomic_set(&e->state, ETHR_EVENT_OFF__); + ethr_atomic32_set(&e->state, ETHR_EVENT_OFF__); ETHR_MEMORY_BARRIER; } diff --git a/erts/include/internal/sparc32/atomic.h b/erts/include/internal/sparc32/atomic.h index 2a995d4465..00380dbf07 100644 --- a/erts/include/internal/sparc32/atomic.h +++ b/erts/include/internal/sparc32/atomic.h @@ -21,49 +21,86 @@ * Native ethread atomics on SPARC V9. * Author: Mikael Pettersson. */ -#ifndef ETHR_SPARC32_ATOMIC_H -#define ETHR_SPARC32_ATOMIC_H -typedef struct { - volatile long counter; -} ethr_native_atomic_t; +#undef ETHR_INCLUDE_ATOMIC_IMPL__ +#if !defined(ETHR_SPARC_V9_ATOMIC32_H__) && defined(ETHR_ATOMIC_WANT_32BIT_IMPL__) +#define ETHR_SPARC_V9_ATOMIC32_H__ +#define ETHR_INCLUDE_ATOMIC_IMPL__ 4 +#undef ETHR_ATOMIC_WANT_32BIT_IMPL__ +#elif !defined(ETHR_SPARC_V9_ATOMIC64_H__) && defined(ETHR_ATOMIC_WANT_64BIT_IMPL__) +#define ETHR_SPARC_V9_ATOMIC64_H__ +#define ETHR_INCLUDE_ATOMIC_IMPL__ 8 +#undef ETHR_ATOMIC_WANT_64BIT_IMPL__ +#endif + +#ifdef ETHR_INCLUDE_ATOMIC_IMPL__ + +#ifndef ETHR_SPARC_V9_ATOMIC_COMMON__ +#define ETHR_SPARC_V9_ATOMIC_COMMON__ #define ETHR_MEMORY_BARRIER \ __asm__ __volatile__("membar #LoadLoad|#LoadStore|#StoreLoad|#StoreStore\n" \ : : : "memory") -#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_AUX_IMPL__) +#endif /* ETHR_SPARC_V9_ATOMIC_COMMON__ */ -#if defined(__arch64__) -#define CASX "casx" +#if ETHR_INCLUDE_ATOMIC_IMPL__ == 4 +#define ETHR_HAVE_NATIVE_ATOMIC32 1 +#define ETHR_NATMC_FUNC__(X) ethr_native_atomic32_ ## X +#define ETHR_ATMC_T__ ethr_native_atomic32_t +#define ETHR_AINT_T__ ethr_sint32_t +#define ETHR_CAS__ "cas" +#elif ETHR_INCLUDE_ATOMIC_IMPL__ == 8 +#define ETHR_HAVE_NATIVE_ATOMIC64 1 +#define ETHR_NATMC_FUNC__(X) ethr_native_atomic64_ ## X +#define ETHR_ATMC_T__ ethr_native_atomic64_t +#define ETHR_AINT_T__ ethr_sint64_t +#define ETHR_CAS__ "casx" #else -#define CASX "cas" +#error "Unsupported integer size" #endif +typedef struct { + volatile ETHR_AINT_T__ counter; +} ETHR_ATMC_T__; + +#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_ATOMIC_IMPL__) + +static ETHR_INLINE ETHR_AINT_T__ * +ETHR_NATMC_FUNC__(addr)(ETHR_ATMC_T__ *var) +{ + return (ETHR_AINT_T__ *) &var->counter; +} + +static ETHR_INLINE void +ETHR_NATMC_FUNC__(init)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ i) +{ + var->counter = i; +} + static ETHR_INLINE void -ethr_native_atomic_init(ethr_native_atomic_t *var, long i) +ETHR_NATMC_FUNC__(set)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ i) { var->counter = i; } -#define ethr_native_atomic_set(v, i) ethr_native_atomic_init((v), (i)) -static ETHR_INLINE long -ethr_native_atomic_read(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(read)(ETHR_ATMC_T__ *var) { return var->counter; } -static ETHR_INLINE long -ethr_native_atomic_add_return(ethr_native_atomic_t *var, long incr) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(add_return)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ incr) { - long old, tmp; + ETHR_AINT_T__ old, tmp; __asm__ __volatile__("membar #LoadLoad|#StoreLoad\n"); do { old = var->counter; tmp = old+incr; __asm__ __volatile__( - CASX " [%2], %1, %0" + ETHR_CAS__ " [%2], %1, %0" : "=&r"(tmp) : "r"(old), "r"(&var->counter), "0"(tmp) : "memory"); @@ -73,46 +110,46 @@ ethr_native_atomic_add_return(ethr_native_atomic_t *var, long incr) } static ETHR_INLINE void -ethr_native_atomic_add(ethr_native_atomic_t *var, long incr) +ETHR_NATMC_FUNC__(add)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ incr) { - (void)ethr_native_atomic_add_return(var, incr); + (void)ETHR_NATMC_FUNC__(add_return)(var, incr); } -static ETHR_INLINE long -ethr_native_atomic_inc_return(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(inc_return)(ETHR_ATMC_T__ *var) { - return ethr_native_atomic_add_return(var, 1); + return ETHR_NATMC_FUNC__(add_return)(var, 1); } static ETHR_INLINE void -ethr_native_atomic_inc(ethr_native_atomic_t *var) +ETHR_NATMC_FUNC__(inc)(ETHR_ATMC_T__ *var) { - (void)ethr_native_atomic_add_return(var, 1); + (void)ETHR_NATMC_FUNC__(add_return)(var, 1); } -static ETHR_INLINE long -ethr_native_atomic_dec_return(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(dec_return)(ETHR_ATMC_T__ *var) { - return ethr_native_atomic_add_return(var, -1); + return ETHR_NATMC_FUNC__(add_return)(var, -1); } static ETHR_INLINE void -ethr_native_atomic_dec(ethr_native_atomic_t *var) +ETHR_NATMC_FUNC__(dec)(ETHR_ATMC_T__ *var) { - (void)ethr_native_atomic_add_return(var, -1); + (void)ETHR_NATMC_FUNC__(add_return)(var, -1); } -static ETHR_INLINE long -ethr_native_atomic_and_retold(ethr_native_atomic_t *var, long mask) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(and_retold)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ mask) { - long old, tmp; + ETHR_AINT_T__ old, tmp; __asm__ __volatile__("membar #LoadLoad|#StoreLoad\n"); do { old = var->counter; tmp = old & mask; __asm__ __volatile__( - CASX " [%2], %1, %0" + ETHR_CAS__ " [%2], %1, %0" : "=&r"(tmp) : "r"(old), "r"(&var->counter), "0"(tmp) : "memory"); @@ -121,17 +158,17 @@ ethr_native_atomic_and_retold(ethr_native_atomic_t *var, long mask) return old; } -static ETHR_INLINE long -ethr_native_atomic_or_retold(ethr_native_atomic_t *var, long mask) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(or_retold)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ mask) { - long old, tmp; + ETHR_AINT_T__ old, tmp; __asm__ __volatile__("membar #LoadLoad|#StoreLoad\n"); do { old = var->counter; tmp = old | mask; __asm__ __volatile__( - CASX " [%2], %1, %0" + ETHR_CAS__ " [%2], %1, %0" : "=&r"(tmp) : "r"(old), "r"(&var->counter), "0"(tmp) : "memory"); @@ -140,17 +177,17 @@ ethr_native_atomic_or_retold(ethr_native_atomic_t *var, long mask) return old; } -static ETHR_INLINE long -ethr_native_atomic_xchg(ethr_native_atomic_t *var, long val) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(xchg)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ val) { - long old, new; + ETHR_AINT_T__ old, new; __asm__ __volatile__("membar #LoadLoad|#StoreLoad"); do { old = var->counter; new = val; __asm__ __volatile__( - CASX " [%2], %1, %0" + ETHR_CAS__ " [%2], %1, %0" : "=&r"(new) : "r"(old), "r"(&var->counter), "0"(new) : "memory"); @@ -159,12 +196,12 @@ ethr_native_atomic_xchg(ethr_native_atomic_t *var, long val) return old; } -static ETHR_INLINE long -ethr_native_atomic_cmpxchg(ethr_native_atomic_t *var, long new, long old) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ new, ETHR_AINT_T__ old) { __asm__ __volatile__("membar #LoadLoad|#StoreLoad\n"); __asm__ __volatile__( - CASX " [%2], %1, %0" + ETHR_CAS__ " [%2], %1, %0" : "=&r"(new) : "r"(old), "r"(&var->counter), "0"(new) : "memory"); @@ -176,39 +213,65 @@ ethr_native_atomic_cmpxchg(ethr_native_atomic_t *var, long new, long old) * Atomic ops with at least specified barriers. */ -static ETHR_INLINE long -ethr_native_atomic_read_acqb(ethr_native_atomic_t *var) +/* TODO: relax acquire barriers */ + +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(read_acqb)(ETHR_ATMC_T__ *var) { - long res = ethr_native_atomic_read(var); - __asm__ __volatile__("membar #StoreLoad|#StoreStore"); + ETHR_AINT_T__ res = ETHR_NATMC_FUNC__(read)(var); + __asm__ __volatile__("membar #LoadLoad|#LoadStore|#StoreLoad|#StoreStore" : : : "memory"); return res; } static ETHR_INLINE void -ethr_native_atomic_set_relb(ethr_native_atomic_t *var, long i) +ETHR_NATMC_FUNC__(set_relb)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ i) { - __asm__ __volatile__("membar #LoadStore|#StoreStore"); - ethr_native_atomic_set(var, i); + __asm__ __volatile__("membar #LoadStore|#StoreStore" : : : "memory"); + ETHR_NATMC_FUNC__(set)(var, i); +} + +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(inc_return_acqb)(ETHR_ATMC_T__ *var) +{ + ETHR_AINT_T__ res = ETHR_NATMC_FUNC__(inc_return)(var); + __asm__ __volatile__("membar #LoadLoad|#LoadStore" : : : "memory"); + return res; } static ETHR_INLINE void -ethr_native_atomic_dec_relb(ethr_native_atomic_t *var) +ETHR_NATMC_FUNC__(dec_relb)(ETHR_ATMC_T__ *var) { - __asm__ __volatile__("membar #LoadStore|#StoreStore"); - ethr_native_atomic_dec(var); + __asm__ __volatile__("membar #LoadStore|#StoreStore" : : : "memory"); + ETHR_NATMC_FUNC__(dec)(var); } -static ETHR_INLINE long -ethr_native_atomic_dec_return_relb(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(dec_return_relb)(ETHR_ATMC_T__ *var) { - __asm__ __volatile__("membar #LoadStore|#StoreStore"); - return ethr_native_atomic_dec_return(var); + __asm__ __volatile__("membar #LoadStore|#StoreStore" : : : "memory"); + return ETHR_NATMC_FUNC__(dec_return)(var); } -#define ethr_native_atomic_inc_return_acqb ethr_native_atomic_inc_return -#define ethr_native_atomic_cmpxchg_acqb ethr_native_atomic_cmpxchg -#define ethr_native_atomic_cmpxchg_relb ethr_native_atomic_cmpxchg +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg_acqb)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ new, ETHR_AINT_T__ old) +{ + ETHR_AINT_T__ res = ETHR_NATMC_FUNC__(cmpxchg)(var, new, old); + __asm__ __volatile__("membar #LoadLoad|#LoadStore" : : : "memory"); + return res; +} + +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg_relb)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ new, ETHR_AINT_T__ old) +{ + __asm__ __volatile__("membar #LoadStore|#StoreStore" : : : "memory"); + return ETHR_NATMC_FUNC__(cmpxchg)(var, new, old); +} #endif /* ETHR_TRY_INLINE_FUNCS */ -#endif /* ETHR_SPARC32_ATOMIC_H */ +#undef ETHR_NATMC_FUNC__ +#undef ETHR_ATMC_T__ +#undef ETHR_AINT_T__ +#undef ETHR_CAS__ + +#endif /* ETHR_INCLUDE_ATOMIC_IMPL__ */ diff --git a/erts/include/internal/sparc32/ethread.h b/erts/include/internal/sparc32/ethread.h index dca113b4d6..aea9794390 100644 --- a/erts/include/internal/sparc32/ethread.h +++ b/erts/include/internal/sparc32/ethread.h @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 2005-2009. All Rights Reserved. + * Copyright Ericsson AB 2005-2010. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -24,7 +24,12 @@ #ifndef ETHREAD_SPARC32_ETHREAD_H #define ETHREAD_SPARC32_ETHREAD_H +#define ETHR_ATOMIC_WANT_32BIT_IMPL__ #include "atomic.h" +#if ETHR_SIZEOF_PTR == 8 +# define ETHR_ATOMIC_WANT_64BIT_IMPL__ +# include "atomic.h" +#endif #include "spinlock.h" #include "rwlock.h" diff --git a/erts/include/internal/tile/atomic.h b/erts/include/internal/tile/atomic.h index 69569d82d1..48e4c0c6c8 100644 --- a/erts/include/internal/tile/atomic.h +++ b/erts/include/internal/tile/atomic.h @@ -24,92 +24,102 @@ #ifndef ETHREAD_TILE_ATOMIC_H #define ETHREAD_TILE_ATOMIC_H +#define ETHR_HAVE_NATIVE_ATOMIC32 1 + #include <atomic.h> /* An atomic is an aligned int accessed via locked operations. */ typedef struct { - volatile long counter; -} ethr_native_atomic_t; + volatile ethr_sint32_t counter; +} ethr_native_atomic32_t; #define ETHR_MEMORY_BARRIER __insn_mf() -#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_AUX_IMPL__) +#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_ATOMIC_IMPL__) + +static ETHR_INLINE ethr_sint32_t * +ethr_native_atomic32_addr(ethr_native_atomic32_t *var) +{ + return (ethr_sint32_t *) &var->counter; +} static ETHR_INLINE void -ethr_native_atomic_init(ethr_native_atomic_t *var, long i) +ethr_native_atomic32_init(ethr_native_atomic32_t *var, ethr_sint32_t i) { var->counter = i; } static ETHR_INLINE void -ethr_native_atomic_set(ethr_native_atomic_t *var, long i) +ethr_native_atomic32_set(ethr_native_atomic32_t *var, ethr_sint32_t i) { atomic_exchange_acq(&var->counter, i); } -static ETHR_INLINE long -ethr_native_atomic_read(ethr_native_atomic_t *var) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_read(ethr_native_atomic32_t *var) { return var->counter; } static ETHR_INLINE void -ethr_native_atomic_add(ethr_native_atomic_t *var, long incr) +ethr_native_atomic32_add(ethr_native_atomic32_t *var, ethr_sint32_t incr) { atomic_add(&var->counter, incr); } static ETHR_INLINE void -ethr_native_atomic_inc(ethr_native_atomic_t *var) +ethr_native_atomic32_inc(ethr_native_atomic32_t *var) { atomic_increment(&var->counter); } static ETHR_INLINE void -ethr_native_atomic_dec(ethr_native_atomic_t *var) +ethr_native_atomic32_dec(ethr_native_atomic32_t *var) { atomic_decrement(&var->counter); } -static ETHR_INLINE long -ethr_native_atomic_add_return(ethr_native_atomic_t *var, long incr) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_add_return(ethr_native_atomic32_t *var, ethr_sint32_t incr) { return atomic_exchange_and_add(&var->counter, incr) + incr; } -static ETHR_INLINE long -ethr_native_atomic_inc_return(ethr_native_atomic_t *var) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_inc_return(ethr_native_atomic32_t *var) { - return ethr_native_atomic_add_return(var, 1); + return ethr_native_atomic32_add_return(var, 1); } -static ETHR_INLINE long -ethr_native_atomic_dec_return(ethr_native_atomic_t *var) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_dec_return(ethr_native_atomic32_t *var) { - return ethr_native_atomic_add_return(var, -1); + return ethr_native_atomic32_add_return(var, -1); } -static ETHR_INLINE long -ethr_native_atomic_and_retold(ethr_native_atomic_t *var, long mask) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_and_retold(ethr_native_atomic32_t *var, ethr_sint32_t mask) { return atomic_and_val(&var->counter, mask); } -static ETHR_INLINE long -ethr_native_atomic_or_retold(ethr_native_atomic_t *var, long mask) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_or_retold(ethr_native_atomic32_t *var, ethr_sint32_t mask) { return atomic_or_val(&var->counter, mask); } -static ETHR_INLINE long -ethr_native_atomic_xchg(ethr_native_atomic_t *var, long val) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_xchg(ethr_native_atomic32_t *var, ethr_sint32_t val) { return atomic_exchange_acq(&var->counter, val); } -static ETHR_INLINE long -ethr_native_atomic_cmpxchg(ethr_native_atomic_t *var, long new, long expected) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_cmpxchg(ethr_native_atomic32_t *var, + ethr_sint32_t new, + ethr_sint32_t expected) { return atomic_compare_and_exchange_val_acq(&var->counter, new, expected); } @@ -118,54 +128,58 @@ ethr_native_atomic_cmpxchg(ethr_native_atomic_t *var, long new, long expected) * Atomic ops with at least specified barriers. */ -static ETHR_INLINE long -ethr_native_atomic_read_acqb(ethr_native_atomic_t *var) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_read_acqb(ethr_native_atomic32_t *var) { - long res = ethr_native_atomic_read(var); + ethr_sint32_t res = ethr_native_atomic32_read(var); ETHR_MEMORY_BARRIER; return res; } -static ETHR_INLINE long -ethr_native_atomic_inc_return_acqb(ethr_native_atomic_t *var) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_inc_return_acqb(ethr_native_atomic32_t *var) { - long res = ethr_native_atomic_inc_return(var); + ethr_sint32_t res = ethr_native_atomic32_inc_return(var); ETHR_MEMORY_BARRIER; return res; } static ETHR_INLINE void -ethr_native_atomic_set_relb(ethr_native_atomic_t *var, long val) +ethr_native_atomic32_set_relb(ethr_native_atomic32_t *var, ethr_sint32_t val) { ETHR_MEMORY_BARRIER; - ethr_native_atomic_set(var, val); + ethr_native_atomic32_set(var, val); } static ETHR_INLINE void -ethr_native_atomic_dec_relb(ethr_native_atomic_t *var) +ethr_native_atomic32_dec_relb(ethr_native_atomic32_t *var) { ETHR_MEMORY_BARRIER; - ethr_native_atomic_dec(var); + ethr_native_atomic32_dec(var); } -static ETHR_INLINE long -ethr_native_atomic_dec_return_relb(ethr_native_atomic_t *var) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_dec_return_relb(ethr_native_atomic32_t *var) { ETHR_MEMORY_BARRIER; - return ethr_native_atomic_dec_return(var); + return ethr_native_atomic32_dec_return(var); } -static ETHR_INLINE long -ethr_native_atomic_cmpxchg_acqb(ethr_native_atomic_t *var, long new, long exp) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_cmpxchg_acqb(ethr_native_atomic32_t *var, + ethr_sint32_t new, + ethr_sint32_t exp) { - return ethr_native_atomic_cmpxchg(var, new, exp); + return ethr_native_atomic32_cmpxchg(var, new, exp); } -static ETHR_INLINE long -ethr_native_atomic_cmpxchg_relb(ethr_native_atomic_t *var, long new, long exp) +static ETHR_INLINE ethr_sint32_t +ethr_native_atomic32_cmpxchg_relb(ethr_native_atomic32_t *var, + ethr_sint32_t new, + ethr_sint32_t exp) { ETHR_MEMORY_BARRIER; - return ethr_native_atomic_cmpxchg(var, new, exp); + return ethr_native_atomic32_cmpxchg(var, new, exp); } #endif /* ETHR_TRY_INLINE_FUNCS */ diff --git a/erts/include/internal/win/ethr_atomic.h b/erts/include/internal/win/ethr_atomic.h index 500459dd6c..60def01a7e 100644 --- a/erts/include/internal/win/ethr_atomic.h +++ b/erts/include/internal/win/ethr_atomic.h @@ -22,223 +22,394 @@ * Author: Rickard Green */ -#ifndef ETHR_WIN_ATOMIC_H__ -#define ETHR_WIN_ATOMIC_H__ - -#ifdef _MSC_VER -# if _MSC_VER < 1300 -# define ETHR_IMMED_ATOMIC_SET_GET_SAFE__ 0 /* Dont trust really old compilers */ -# else -# if defined(_M_IX86) -# define ETHR_IMMED_ATOMIC_SET_GET_SAFE__ 1 -# else /* I.e. IA64 */ -# if _MSC_VER >= 1400 -# define ETHR_IMMED_ATOMIC_SET_GET_SAFE__ 1 -# else -# define ETHR_IMMED_ATOMIC_SET_GET_SAFE__ 0 -# endif -# endif -# endif -# if _MSC_VER >= 1400 -# include <intrin.h> -# undef ETHR_COMPILER_BARRIER -# define ETHR_COMPILER_BARRIER _ReadWriteBarrier() -# endif -#pragma intrinsic(_ReadWriteBarrier) -#pragma intrinsic(_InterlockedAnd) -#pragma intrinsic(_InterlockedOr) +#undef ETHR_INCLUDE_ATOMIC_IMPL__ +#if !defined(ETHR_WIN_ATOMIC32_H__) && defined(ETHR_ATOMIC_WANT_32BIT_IMPL__) +#define ETHR_WIN_ATOMIC32_H__ +#define ETHR_INCLUDE_ATOMIC_IMPL__ 4 +#undef ETHR_ATOMIC_WANT_32BIT_IMPL__ +#elif !defined(ETHR_WIN_ATOMIC64_H__) && defined(ETHR_ATOMIC_WANT_64BIT_IMPL__) +#define ETHR_WIN_ATOMIC64_H__ +#ifdef ETHR_HAVE__INTERLOCKEDCOMPAREEXCHANGE64 +/* _InterlockedCompareExchange64() required... */ +#define ETHR_INCLUDE_ATOMIC_IMPL__ 8 +#endif +#undef ETHR_ATOMIC_WANT_64BIT_IMPL__ +#endif + +#ifdef ETHR_INCLUDE_ATOMIC_IMPL__ + +#if defined(_MSC_VER) && _MSC_VER >= 1400 + +#ifndef ETHR_WIN_ATOMIC_COMMON__ +#define ETHR_WIN_ATOMIC_COMMON__ + +#define ETHR_HAVE_NATIVE_ATOMICS 1 + +#if defined(_M_IX86) || defined(_M_AMD64) || defined(_M_IA64) +# define ETHR_READ_AND_SET_WITHOUT_INTERLOCKED_OP__ 1 #else -# define ETHR_IMMED_ATOMIC_SET_GET_SAFE__ 0 +# define ETHR_READ_AND_SET_WITHOUT_INTERLOCKED_OP__ 0 #endif +#if defined(_M_AMD64) || (defined(_M_IX86) \ + && !defined(ETHR_PRE_PENTIUM4_COMPAT)) +# define ETHR_READ_ACQB_AND_SET_RELB_COMPILER_BARRIER_ONLY__ 1 +#else +# define ETHR_READ_ACQB_AND_SET_RELB_COMPILER_BARRIER_ONLY__ 0 +#endif /* - * No configure test checking for _Interlocked*_{acq,rel} and - * Interlocked*{Acquire,Release} have been written yet... + * No configure test checking for interlocked acquire/release + * versions have been written, yet. It should define + * ETHR_HAVE_INTERLOCKED_ACQUIRE_RELEASE_BARRIERS if, and + * only if, all used interlocked operations with barriers + * exists. * * Note, that these are pure optimizations for the itanium * processor. */ -#ifdef ETHR_HAVE_INTERLOCKEDCOMPAREEXCHANGE_ACQ -#pragma intrinsic(_InterlockedCompareExchange_acq) +#include <intrin.h> +#undef ETHR_COMPILER_BARRIER +#define ETHR_COMPILER_BARRIER _ReadWriteBarrier() +#pragma intrinsic(_ReadWriteBarrier) +#pragma intrinsic(_InterlockedCompareExchange) + +#if defined(_M_AMD64) || (defined(_M_IX86) \ + && !defined(ETHR_PRE_PENTIUM4_COMPAT)) +#include <emmintrin.h> +#include <mmintrin.h> +#pragma intrinsic(_mm_mfence) +#define ETHR_MEMORY_BARRIER _mm_mfence() +#pragma intrinsic(_mm_sfence) +#define ETHR_WRITE_MEMORY_BARRIER _mm_sfence() +#pragma intrinsic(_mm_lfence) +#define ETHR_READ_MEMORY_BARRIER _mm_lfence() +#define ETHR_READ_DEPEND_MEMORY_BARRIER ETHR_COMPILER_BARRIER + +#else + +#define ETHR_MEMORY_BARRIER \ +do { \ + volatile long x___ = 0; \ + _InterlockedCompareExchange(&x___, (long) 1, (long) 0); \ +} while (0) + #endif -#ifdef ETHR_HAVE_INTERLOCKEDCOMPAREEXCHANGE_REL + +#endif /* ETHR_WIN_ATOMIC_COMMON__ */ + +#if ETHR_INCLUDE_ATOMIC_IMPL__ == 4 + +#define ETHR_HAVE_NATIVE_ATOMIC32 1 + +/* + * All used operations available as 32-bit intrinsics + */ + +#pragma intrinsic(_InterlockedDecrement) +#pragma intrinsic(_InterlockedIncrement) +#pragma intrinsic(_InterlockedExchangeAdd) +#pragma intrinsic(_InterlockedExchange) +#pragma intrinsic(_InterlockedAnd) +#pragma intrinsic(_InterlockedOr) +#ifdef ETHR_HAVE_INTERLOCKED_ACQUIRE_RELEASE_BARRIERS +#pragma intrinsic(_InterlockedExchangeAdd_acq) +#pragma intrinsic(_InterlockedIncrement_acq) +#pragma intrinsic(_InterlockedDecrement_rel) +#pragma intrinsic(_InterlockedCompareExchange_acq) #pragma intrinsic(_InterlockedCompareExchange_rel) #endif +#define ETHR_ILCKD__(X) _Interlocked ## X +#ifdef ETHR_HAVE_INTERLOCKED_ACQUIRE_RELEASE_BARRIERS +#define ETHR_ILCKD_ACQ__(X) _Interlocked ## X ## _acq +#define ETHR_ILCKD_REL__(X) _Interlocked ## X ## _rel +#else +#define ETHR_ILCKD_ACQ__(X) _Interlocked ## X +#define ETHR_ILCKD_REL__(X) _Interlocked ## X +#endif + +#define ETHR_NATMC_FUNC__(X) ethr_native_atomic32_ ## X +#define ETHR_ATMC_T__ ethr_native_atomic32_t +#define ETHR_AINT_T__ ethr_sint32_t + +#elif ETHR_INCLUDE_ATOMIC_IMPL__ == 8 + +#define ETHR_HAVE_NATIVE_ATOMIC64 1 + +/* + * _InterlockedCompareExchange64() is required. The other may not + * be available, but if so, we can generate them. + */ +#pragma intrinsic(_InterlockedCompareExchange64) + +#if ETHR_READ_AND_SET_WITHOUT_INTERLOCKED_OP__ +#define ETHR_OWN_ILCKD_INIT_VAL__(PTR) *(PTR) +#else +#define ETHR_OWN_ILCKD_INIT_VAL__(PTR) (__int64) 0 +#endif + +#define ETHR_OWN_ILCKD_BODY_IMPL__(FUNC, PTR, NEW, ACT, EXP, OPS, RET) \ +{ \ + __int64 NEW, ACT, EXP; \ + ACT = ETHR_OWN_ILCKD_INIT_VAL__(PTR); \ + do { \ + EXP = ACT; \ + { OPS; } \ + ACT = _InterlockedCompareExchange64(PTR, NEW, EXP); \ + } while (ACT != EXP); \ + return RET; \ +} + +#define ETHR_OWN_ILCKD_1_IMPL__(FUNC, NEW, ACT, EXP, OPS, RET) \ +static __forceinline __int64 \ +FUNC(__int64 volatile *ptr) \ +ETHR_OWN_ILCKD_BODY_IMPL__(FUNC, ptr, NEW, ACT, EXP, OPS, RET) + +#define ETHR_OWN_ILCKD_2_IMPL__(FUNC, NEW, ACT, EXP, OPS, ARG, RET) \ +static __forceinline __int64 \ +FUNC(__int64 volatile *ptr, __int64 ARG) \ +ETHR_OWN_ILCKD_BODY_IMPL__(FUNC, ptr, NEW, ACT, EXP, OPS, RET) + + +#ifdef ETHR_HAVE__INTERLOCKEDDECREMENT64 +#pragma intrinsic(_InterlockedDecrement64) +#else +ETHR_OWN_ILCKD_1_IMPL__(_InterlockedDecrement64, new, act, exp, + new = act - 1, new) +#endif +#ifdef ETHR_HAVE__INTERLOCKEDINCREMENT64 +#pragma intrinsic(_InterlockedIncrement64) +#else +ETHR_OWN_ILCKD_1_IMPL__(_InterlockedIncrement64, new, act, exp, + new = act + 1, new) +#endif +#ifdef ETHR_HAVE__INTERLOCKEDEXCHANGEADD64 +#pragma intrinsic(_InterlockedExchangeAdd64) +#else +ETHR_OWN_ILCKD_2_IMPL__(_InterlockedExchangeAdd64, new, act, exp, + new = act + arg, arg, act) +#endif +#ifdef ETHR_HAVE__INTERLOCKEDEXCHANGE64 +#pragma intrinsic(_InterlockedExchange64) +#else +ETHR_OWN_ILCKD_2_IMPL__(_InterlockedExchange64, new, act, exp, + new = arg, arg, act) +#endif +#ifdef ETHR_HAVE__INTERLOCKEDAND64 +#pragma intrinsic(_InterlockedAnd64) +#else +ETHR_OWN_ILCKD_2_IMPL__(_InterlockedAnd64, new, act, exp, + new = act & arg, arg, act) +#endif +#ifdef ETHR_HAVE__INTERLOCKEDOR64 +#pragma intrinsic(_InterlockedOr64) +#else +ETHR_OWN_ILCKD_2_IMPL__(_InterlockedOr64, new, act, exp, + new = act | arg, arg, act) +#endif +#ifdef ETHR_HAVE_INTERLOCKED_ACQUIRE_RELEASE_BARRIERS +#pragma intrinsic(_InterlockedExchangeAdd64_acq) +#pragma intrinsic(_InterlockedIncrement64_acq) +#pragma intrinsic(_InterlockedDecrement64_rel) +#pragma intrinsic(_InterlockedCompareExchange64_acq) +#pragma intrinsic(_InterlockedCompareExchange64_rel) +#endif + +#define ETHR_ILCKD__(X) _Interlocked ## X ## 64 +#ifdef ETHR_HAVE_INTERLOCKED_ACQUIRE_RELEASE_BARRIERS +#define ETHR_ILCKD_ACQ__(X) _Interlocked ## X ## 64_acq +#define ETHR_ILCKD_REL__(X) _Interlocked ## X ## 64_rel +#else +#define ETHR_ILCKD_ACQ__(X) _Interlocked ## X ## 64 +#define ETHR_ILCKD_REL__(X) _Interlocked ## X ## 64 +#endif + +#define ETHR_NATMC_FUNC__(X) ethr_native_atomic64_ ## X +#define ETHR_ATMC_T__ ethr_native_atomic64_t +#define ETHR_AINT_T__ ethr_sint64_t + +#else +#error "Unsupported integer size" +#endif typedef struct { - volatile LONG value; -} ethr_native_atomic_t; + volatile ETHR_AINT_T__ value; +} ETHR_ATMC_T__; -#define ETHR_MEMORY_BARRIER \ -do { \ - volatile LONG x___ = 0; \ - _InterlockedCompareExchange(&x___, (LONG) 1, (LONG) 0); \ -} while (0) +#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_ATOMIC_IMPL__) -#if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_AUX_IMPL__) +static ETHR_INLINE ETHR_AINT_T__ * +ETHR_NATMC_FUNC__(addr)(ETHR_ATMC_T__ *var) +{ + return (ETHR_AINT_T__ *) &var->value; +} static ETHR_INLINE void -ethr_native_atomic_init(ethr_native_atomic_t *var, long i) +ETHR_NATMC_FUNC__(init)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ i) { - var->value = (LONG) i; +#if ETHR_READ_AND_SET_WITHOUT_INTERLOCKED_OP__ + var->value = i; +#else + (void) ETHR_ILCKD__(Exchange)(&var->value, i); +#endif } static ETHR_INLINE void -ethr_native_atomic_set(ethr_native_atomic_t *var, long i) +ETHR_NATMC_FUNC__(set)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ i) { -#if ETHR_IMMED_ATOMIC_SET_GET_SAFE__ - var->value = (LONG) i; +#if ETHR_READ_AND_SET_WITHOUT_INTERLOCKED_OP__ + var->value = i; #else - (void) InterlockedExchange(&var->value, (LONG) i); + (void) ETHR_ILCKD__(Exchange)(&var->value, i); #endif } -static ETHR_INLINE long -ethr_native_atomic_read(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(read)(ETHR_ATMC_T__ *var) { -#if ETHR_IMMED_ATOMIC_SET_GET_SAFE__ +#if ETHR_READ_AND_SET_WITHOUT_INTERLOCKED_OP__ return var->value; #else - return InterlockedExchangeAdd(&var->value, (LONG) 0); + return ETHR_ILCKD__(ExchangeAdd)(&var->value, (ETHR_AINT_T__) 0); #endif } static ETHR_INLINE void -ethr_native_atomic_add(ethr_native_atomic_t *var, long incr) +ETHR_NATMC_FUNC__(add)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ incr) { - (void) InterlockedExchangeAdd(&var->value, (LONG) incr); + (void) ETHR_ILCKD__(ExchangeAdd)(&var->value, incr); } -static ETHR_INLINE long -ethr_native_atomic_add_return(ethr_native_atomic_t *var, long i) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(add_return)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ i) { - LONG tmp = InterlockedExchangeAdd(&var->value, (LONG) i); - return tmp + i; + return ETHR_ILCKD__(ExchangeAdd)(&var->value, i) + i; } static ETHR_INLINE void -ethr_native_atomic_inc(ethr_native_atomic_t *var) +ETHR_NATMC_FUNC__(inc)(ETHR_ATMC_T__ *var) { - (void) InterlockedIncrement(&var->value); + (void) ETHR_ILCKD__(Increment)(&var->value); } static ETHR_INLINE void -ethr_native_atomic_dec(ethr_native_atomic_t *var) +ETHR_NATMC_FUNC__(dec)(ETHR_ATMC_T__ *var) { - (void) InterlockedDecrement(&var->value); + (void) ETHR_ILCKD__(Decrement)(&var->value); } -static ETHR_INLINE long -ethr_native_atomic_inc_return(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(inc_return)(ETHR_ATMC_T__ *var) { - return (long) InterlockedIncrement(&var->value); + return ETHR_ILCKD__(Increment)(&var->value); } -static ETHR_INLINE long -ethr_native_atomic_dec_return(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(dec_return)(ETHR_ATMC_T__ *var) { - return (long) InterlockedDecrement(&var->value); + return ETHR_ILCKD__(Decrement)(&var->value); } -static ETHR_INLINE long -ethr_native_atomic_and_retold(ethr_native_atomic_t *var, long mask) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(and_retold)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ mask) { - return (long) _InterlockedAnd(&var->value, mask); + return ETHR_ILCKD__(And)(&var->value, mask); } -static ETHR_INLINE long -ethr_native_atomic_or_retold(ethr_native_atomic_t *var, long mask) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(or_retold)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ mask) { - return (long) _InterlockedOr(&var->value, mask); + return ETHR_ILCKD__(Or)(&var->value, mask); } - -static ETHR_INLINE long -ethr_native_atomic_cmpxchg(ethr_native_atomic_t *var, long new, long old) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg)(ETHR_ATMC_T__ *var, + ETHR_AINT_T__ new, + ETHR_AINT_T__ old) { - return (long) _InterlockedCompareExchange(&var->value, (LONG) new, (LONG) old); + return ETHR_ILCKD__(CompareExchange)(&var->value, new, old); } -static ETHR_INLINE long -ethr_native_atomic_xchg(ethr_native_atomic_t *var, long new) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(xchg)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ new) { - return (long) InterlockedExchange(&var->value, (LONG) new); + return ETHR_ILCKD__(Exchange)(&var->value, new); } /* * Atomic ops with at least specified barriers. */ -static ETHR_INLINE long -ethr_native_atomic_read_acqb(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(read_acqb)(ETHR_ATMC_T__ *var) { -#ifdef ETHR_HAVE_INTERLOCKEDEXCHANGEADDACQUIRE - return (long) InterlockedExchangeAddAcquire(&var->value, (LONG) 0); +#if ETHR_READ_ACQB_AND_SET_RELB_COMPILER_BARRIER_ONLY__ + ETHR_AINT_T__ val = var->value; + ETHR_COMPILER_BARRIER; + return val; #else - return (long) InterlockedExchangeAdd(&var->value, (LONG) 0); + return ETHR_ILCKD_ACQ__(ExchangeAdd)(&var->value, (ETHR_AINT_T__) 0); #endif } -static ETHR_INLINE long -ethr_native_atomic_inc_return_acqb(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(inc_return_acqb)(ETHR_ATMC_T__ *var) { -#ifdef ETHR_HAVE_INTERLOCKEDINCREMENTACQUIRE - return (long) InterlockedIncrementAcquire(&var->value); -#else - return (long) InterlockedIncrement(&var->value); -#endif + return ETHR_ILCKD_ACQ__(Increment)(&var->value); } static ETHR_INLINE void -ethr_native_atomic_set_relb(ethr_native_atomic_t *var, long i) +ETHR_NATMC_FUNC__(set_relb)(ETHR_ATMC_T__ *var, ETHR_AINT_T__ i) { - (void) InterlockedExchange(&var->value, (LONG) i); +#if ETHR_READ_ACQB_AND_SET_RELB_COMPILER_BARRIER_ONLY__ + ETHR_COMPILER_BARRIER; + var->value = i; +#else + (void) ETHR_ILCKD_REL__(Exchange)(&var->value, i); +#endif } static ETHR_INLINE void -ethr_native_atomic_dec_relb(ethr_native_atomic_t *var) +ETHR_NATMC_FUNC__(dec_relb)(ETHR_ATMC_T__ *var) { -#ifdef ETHR_HAVE_INTERLOCKEDDECREMENTRELEASE - (void) InterlockedDecrementRelease(&var->value); -#else - (void) InterlockedDecrement(&var->value); -#endif + (void) ETHR_ILCKD_REL__(Decrement)(&var->value); } -static ETHR_INLINE long -ethr_native_atomic_dec_return_relb(ethr_native_atomic_t *var) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(dec_return_relb)(ETHR_ATMC_T__ *var) { -#ifdef ETHR_HAVE_INTERLOCKEDDECREMENTRELEASE - return (long) InterlockedDecrementRelease(&var->value); -#else - return (long) InterlockedDecrement(&var->value); -#endif + return ETHR_ILCKD_REL__(Decrement)(&var->value); } -static ETHR_INLINE long -ethr_native_atomic_cmpxchg_acqb(ethr_native_atomic_t *var, long new, long old) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg_acqb)(ETHR_ATMC_T__ *var, + ETHR_AINT_T__ new, + ETHR_AINT_T__ old) { -#ifdef ETHR_HAVE_INTERLOCKEDCOMPAREEXCHANGE_ACQ - return (long) _InterlockedCompareExchange_acq(&var->value, (LONG) new, (LONG) old); -#else - return (long) _InterlockedCompareExchange(&var->value, (LONG) new, (LONG) old); -#endif + return ETHR_ILCKD_ACQ__(CompareExchange)(&var->value, new, old); } -static ETHR_INLINE long -ethr_native_atomic_cmpxchg_relb(ethr_native_atomic_t *var, long new, long old) +static ETHR_INLINE ETHR_AINT_T__ +ETHR_NATMC_FUNC__(cmpxchg_relb)(ETHR_ATMC_T__ *var, + ETHR_AINT_T__ new, + ETHR_AINT_T__ old) { - -#ifdef ETHR_HAVE_INTERLOCKEDCOMPAREEXCHANGE_REL - return (long) _InterlockedCompareExchange_rel(&var->value, (LONG) new, (LONG) old); -#else - return (long) _InterlockedCompareExchange(&var->value, (LONG) new, (LONG) old); -#endif + return ETHR_ILCKD_REL__(CompareExchange)(&var->value, new, old); } -#endif +#endif /* ETHR_TRY_INLINE_FUNCS */ -#endif +#undef ETHR_ILCKD__ +#undef ETHR_ILCKD_ACQ__ +#undef ETHR_ILCKD_REL__ +#undef ETHR_NATMC_FUNC__ +#undef ETHR_ATMC_T__ +#undef ETHR_AINT_T__ +#undef ETHR_READ_AND_SET_WITHOUT_INTERLOCKED_OP__ +#undef ETHR_READ_ACQB_AND_SET_RELB_COMPILER_BARRIER_ONLY__ + +#endif /* _MSC_VER */ + +#endif /* ETHR_INCLUDE_ATOMIC_IMPL__ */ diff --git a/erts/include/internal/win/ethr_event.h b/erts/include/internal/win/ethr_event.h index af57c20f91..598816b2c6 100644 --- a/erts/include/internal/win/ethr_event.h +++ b/erts/include/internal/win/ethr_event.h @@ -21,22 +21,24 @@ * Author: Rickard Green */ -#define ETHR_EVENT_OFF_WAITER__ ((LONG) -1) -#define ETHR_EVENT_OFF__ ((LONG) 1) -#define ETHR_EVENT_ON__ ((LONG) 0) +#define ETHR_EVENT_OFF_WAITER__ ((long) -1) +#define ETHR_EVENT_OFF__ ((long) 1) +#define ETHR_EVENT_ON__ ((long) 0) typedef struct { - volatile LONG state; + volatile long state; HANDLE handle; } ethr_event; #if defined(ETHR_TRY_INLINE_FUNCS) || defined(ETHR_EVENT_IMPL__) +#pragma intrinsic(_InterlockedExchange) + static ETHR_INLINE void ETHR_INLINE_FUNC_NAME_(ethr_event_set)(ethr_event *e) { - /* InterlockedExchange() imply a full memory barrier which is important */ - LONG state = InterlockedExchange(&e->state, ETHR_EVENT_ON__); + /* _InterlockedExchange() imply a full memory barrier which is important */ + long state = _InterlockedExchange(&e->state, ETHR_EVENT_ON__); if (state == ETHR_EVENT_OFF_WAITER__) { if (!SetEvent(e->handle)) ETHR_FATAL_ERROR__(ethr_win_get_errno__()); @@ -46,7 +48,7 @@ ETHR_INLINE_FUNC_NAME_(ethr_event_set)(ethr_event *e) static ETHR_INLINE void ETHR_INLINE_FUNC_NAME_(ethr_event_reset)(ethr_event *e) { - /* InterlockedExchange() imply a full memory barrier which is important */ + /* _InterlockedExchange() imply a full memory barrier which is important */ InterlockedExchange(&e->state, ETHR_EVENT_OFF__); } diff --git a/erts/include/internal/win/ethread.h b/erts/include/internal/win/ethread.h index b52710f6a3..c01b17cf14 100644 --- a/erts/include/internal/win/ethread.h +++ b/erts/include/internal/win/ethread.h @@ -25,7 +25,11 @@ #ifndef ETHREAD_WIN_H__ #define ETHREAD_WIN_H__ +#define ETHR_ATOMIC_WANT_32BIT_IMPL__ #include "ethr_atomic.h" -#define ETHR_HAVE_NATIVE_ATOMICS 1 +#if ETHR_SIZEOF_PTR == 8 +# define ETHR_ATOMIC_WANT_64BIT_IMPL__ +# include "ethr_atomic.h" +#endif #endif diff --git a/erts/lib_src/Makefile.in b/erts/lib_src/Makefile.in index 0d3181cace..757b3b24e2 100644 --- a/erts/lib_src/Makefile.in +++ b/erts/lib_src/Makefile.in @@ -283,6 +283,7 @@ endif ETHR_THR_LIB_BASE_DIR=@ETHR_THR_LIB_BASE_DIR@ ifneq ($(strip $(ETHR_LIB_NAME)),) ETHREAD_LIB_SRC=common/ethr_aux.c \ + common/ethr_atomics.c \ common/ethr_mutex.c \ common/ethr_cbf.c \ $(ETHR_THR_LIB_BASE_DIR)/ethread.c \ @@ -381,6 +382,11 @@ $(ERTS_LIB): $(ERTS_LIB_OBJS) # Object files # +ifeq ($(TYPE)-@GCC@,debug-yes) +$(r_OBJ_DIR)/ethr_aux.o: common/ethr_aux.c + $(CC) $(THR_DEFS) $(CFLAGS) -Wno-unused-function $(INCLUDES) -c $< -o $@ +endif + $(r_OBJ_DIR)/%.o: common/%.c $(CC) $(THR_DEFS) $(CFLAGS) $(INCLUDES) -c $< -o $@ @@ -445,6 +451,7 @@ INTERNAL_RELEASE_INCLUDES= \ $(ERTS_INCL_INT)/ethread.h \ $(ERTS_INCL_INT)/ethr_mutex.h \ $(ERTS_INCL_INT)/ethr_optimized_fallbacks.h \ + $(ERTS_INCL_INT)/ethr_atomics.h \ $(ERTS_INCL_INT)/$(TARGET)/ethread.mk \ $(ERTS_INCL_INT)/$(TARGET)/erts_internal.mk \ $(ERTS_INCL_INT)/$(TARGET)/ethread_header_config.h \ diff --git a/erts/lib_src/common/ethr_atomics.c b/erts/lib_src/common/ethr_atomics.c new file mode 100644 index 0000000000..94557d904a --- /dev/null +++ b/erts/lib_src/common/ethr_atomics.c @@ -0,0 +1,402 @@ +/* + * %CopyrightBegin% + * + * Copyright Ericsson AB 2010. All Rights Reserved. + * + * The contents of this file are subject to the Erlang Public License, + * Version 1.1, (the "License"); you may not use this file except in + * compliance with the License. You should have received a copy of the + * Erlang Public License along with this software. If not, it can be + * retrieved online at http://www.erlang.org/. + * + * Software distributed under the License is distributed on an "AS IS" + * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + * the License for the specific language governing rights and limitations + * under the License. + * + * %CopyrightEnd% + */ + +/* + * Description: The ethread atomic API + * Author: Rickard Green + */ + +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif + +#define ETHR_INLINE_FUNC_NAME_(X) X ## __ +#define ETHR_ATOMIC_IMPL__ + +#include "ethread.h" +#include "ethr_internal.h" + +#ifndef ETHR_HAVE_NATIVE_ATOMICS +ethr_atomic_protection_t ethr_atomic_protection__[1 << ETHR_ATOMIC_ADDR_BITS]; +#endif + +int +ethr_init_atomics(void) +{ +#ifndef ETHR_HAVE_NATIVE_ATOMICS + { + int i; + for (i = 0; i < (1 << ETHR_ATOMIC_ADDR_BITS); i++) { + int res = ethr_spinlock_init(ðr_atomic_protection__[i].u.lck); + if (res != 0) + return res; + } + } +#endif + return 0; +} + +/* + * --- Pointer size atomics --------------------------------------------------- + */ + +ethr_sint_t * +ethr_atomic_addr(ethr_atomic_t *var) +{ + ETHR_ASSERT(var); + return ethr_atomic_addr__(var); +} + +void +ethr_atomic_init(ethr_atomic_t *var, ethr_sint_t i) +{ + ETHR_ASSERT(var); + ethr_atomic_init__(var, i); +} + +void +ethr_atomic_set(ethr_atomic_t *var, ethr_sint_t i) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + ethr_atomic_set__(var, i); +} + +ethr_sint_t +ethr_atomic_read(ethr_atomic_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic_read__(var); +} + +ethr_sint_t +ethr_atomic_add_read(ethr_atomic_t *var, ethr_sint_t incr) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic_add_read__(var, incr); +} + +ethr_sint_t +ethr_atomic_inc_read(ethr_atomic_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic_inc_read__(var); +} + +ethr_sint_t +ethr_atomic_dec_read(ethr_atomic_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic_dec_read__(var); +} + +void +ethr_atomic_add(ethr_atomic_t *var, ethr_sint_t incr) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + ethr_atomic_add__(var, incr); +} + +void +ethr_atomic_inc(ethr_atomic_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + ethr_atomic_inc__(var); +} + +void +ethr_atomic_dec(ethr_atomic_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + ethr_atomic_dec__(var); +} + +ethr_sint_t +ethr_atomic_read_band(ethr_atomic_t *var, ethr_sint_t mask) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic_read_band__(var, mask); +} + +ethr_sint_t +ethr_atomic_read_bor(ethr_atomic_t *var, ethr_sint_t mask) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic_read_bor__(var, mask); +} + +ethr_sint_t +ethr_atomic_xchg(ethr_atomic_t *var, ethr_sint_t new) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic_xchg__(var, new); +} + +ethr_sint_t +ethr_atomic_cmpxchg(ethr_atomic_t *var, ethr_sint_t new, ethr_sint_t expected) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic_cmpxchg__(var, new, expected); +} + +ethr_sint_t +ethr_atomic_read_acqb(ethr_atomic_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic_read_acqb__(var); +} + +ethr_sint_t +ethr_atomic_inc_read_acqb(ethr_atomic_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic_inc_read_acqb__(var); +} + +void +ethr_atomic_set_relb(ethr_atomic_t *var, ethr_sint_t i) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + ethr_atomic_set_relb__(var, i); +} + +void +ethr_atomic_dec_relb(ethr_atomic_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + ethr_atomic_dec_relb__(var); +} + +ethr_sint_t +ethr_atomic_dec_read_relb(ethr_atomic_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic_dec_read_relb__(var); +} + +ethr_sint_t +ethr_atomic_cmpxchg_acqb(ethr_atomic_t *var, ethr_sint_t new, ethr_sint_t exp) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic_cmpxchg_acqb__(var, new, exp); +} + +ethr_sint_t +ethr_atomic_cmpxchg_relb(ethr_atomic_t *var, ethr_sint_t new, ethr_sint_t exp) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic_cmpxchg_relb__(var, new, exp); +} + + +/* + * --- 32-bit atomics --------------------------------------------------------- + */ + +ethr_sint32_t * +ethr_atomic32_addr(ethr_atomic32_t *var) +{ + ETHR_ASSERT(var); + return ethr_atomic32_addr__(var); +} + +void +ethr_atomic32_init(ethr_atomic32_t *var, ethr_sint32_t i) +{ + ETHR_ASSERT(var); + ethr_atomic32_init__(var, i); +} + +void +ethr_atomic32_set(ethr_atomic32_t *var, ethr_sint32_t i) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + ethr_atomic32_set__(var, i); +} + +ethr_sint32_t +ethr_atomic32_read(ethr_atomic32_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic32_read__(var); +} + + +ethr_sint32_t +ethr_atomic32_add_read(ethr_atomic32_t *var, ethr_sint32_t incr) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic32_add_read__(var, incr); +} + +ethr_sint32_t +ethr_atomic32_inc_read(ethr_atomic32_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic32_inc_read__(var); +} + +ethr_sint32_t +ethr_atomic32_dec_read(ethr_atomic32_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic32_dec_read__(var); +} + +void +ethr_atomic32_add(ethr_atomic32_t *var, ethr_sint32_t incr) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + ethr_atomic32_add__(var, incr); +} + +void +ethr_atomic32_inc(ethr_atomic32_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + ethr_atomic32_inc__(var); +} + +void +ethr_atomic32_dec(ethr_atomic32_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + ethr_atomic32_dec__(var); +} + +ethr_sint32_t +ethr_atomic32_read_band(ethr_atomic32_t *var, ethr_sint32_t mask) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic32_read_band__(var, mask); +} + +ethr_sint32_t +ethr_atomic32_read_bor(ethr_atomic32_t *var, ethr_sint32_t mask) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic32_read_bor__(var, mask); +} + +ethr_sint32_t +ethr_atomic32_xchg(ethr_atomic32_t *var, ethr_sint32_t new) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic32_xchg__(var, new); +} + +ethr_sint32_t +ethr_atomic32_cmpxchg(ethr_atomic32_t *var, + ethr_sint32_t new, + ethr_sint32_t expected) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic32_cmpxchg__(var, new, expected); +} + +ethr_sint32_t +ethr_atomic32_read_acqb(ethr_atomic32_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic32_read_acqb__(var); +} + +ethr_sint32_t +ethr_atomic32_inc_read_acqb(ethr_atomic32_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic32_inc_read_acqb__(var); +} + +void +ethr_atomic32_set_relb(ethr_atomic32_t *var, ethr_sint32_t i) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + ethr_atomic32_set_relb__(var, i); +} + +void +ethr_atomic32_dec_relb(ethr_atomic32_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + ethr_atomic32_dec_relb__(var); +} + +ethr_sint32_t +ethr_atomic32_dec_read_relb(ethr_atomic32_t *var) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic32_dec_read_relb__(var); +} + +ethr_sint32_t +ethr_atomic32_cmpxchg_acqb(ethr_atomic32_t *var, + ethr_sint32_t new, + ethr_sint32_t exp) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic32_cmpxchg_acqb__(var, new, exp); +} + +ethr_sint32_t +ethr_atomic32_cmpxchg_relb(ethr_atomic32_t *var, + ethr_sint32_t new, + ethr_sint32_t exp) +{ + ETHR_ASSERT(!ethr_not_inited__); + ETHR_ASSERT(var); + return ethr_atomic32_cmpxchg_relb__(var, new, exp); +} + diff --git a/erts/lib_src/common/ethr_aux.c b/erts/lib_src/common/ethr_aux.c index 4db4cffd3a..2c3e25a805 100644 --- a/erts/lib_src/common/ethr_aux.c +++ b/erts/lib_src/common/ethr_aux.c @@ -31,7 +31,10 @@ #define ETHR_INLINE_FUNC_NAME_(X) X ## __ #define ETHR_AUX_IMPL__ - +#define ETHR_ATOMIC_IMPL__ /* Needed in order to pull in + native atomic implementations + for optimized fallbacks of + spinlocks and rwspinlocks */ #include "ethread.h" #include "ethr_internal.h" #include <string.h> @@ -51,10 +54,6 @@ int ethr_not_inited__ = 1; ethr_memory_allocators ethr_mem__ = ETHR_MEM_ALLOCS_DEF_INITER__; -#ifndef ETHR_HAVE_OPTIMIZED_ATOMIC_OPS -ethr_atomic_protection_t ethr_atomic_protection__[1 << ETHR_ATOMIC_ADDR_BITS]; -#endif - void *(*ethr_thr_prepare_func__)(void) = NULL; void (*ethr_thr_parent_func__)(void *) = NULL; void (*ethr_thr_child_func__)(void *) = NULL; @@ -138,16 +137,9 @@ ethr_init_common__(ethr_init_data *id) #endif ethr_max_stack_size__ = ETHR_B2KW(ethr_max_stack_size__); -#ifndef ETHR_HAVE_OPTIMIZED_ATOMIC_OPS - { - int i; - for (i = 0; i < (1 << ETHR_ATOMIC_ADDR_BITS); i++) { - res = ethr_spinlock_init(ðr_atomic_protection__[i].u.lck); - if (res != 0) - return res; - } - } -#endif + res = ethr_init_atomics(); + if (res != 0) + return res; res = ethr_mutex_lib_init(erts_get_cpu_configured(ethr_cpu_info__)); if (res != 0) @@ -279,14 +271,6 @@ typedef union { static ethr_spinlock_t ts_ev_alloc_lock; static ethr_ts_event *free_ts_ev; -#if SIZEOF_VOID_P == SIZEOF_INT -typedef unsigned int EthrPtrSzUInt; -#elif SIZEOF_VOID_P == SIZEOF_LONG -typedef unsigned long EthrPtrSzUInt; -#else -#error No pointer sized integer type -#endif - static ethr_ts_event *ts_event_pool(int size, ethr_ts_event **endpp) { int i; @@ -295,16 +279,16 @@ static ethr_ts_event *ts_event_pool(int size, ethr_ts_event **endpp) + ETHR_CACHE_LINE_SIZE); if (!atsev) return NULL; - if ((((EthrPtrSzUInt) atsev) & ETHR_CACHE_LINE_MASK) == 0) + if ((((ethr_uint_t) atsev) & ETHR_CACHE_LINE_MASK) == 0) atsev = ((ethr_aligned_ts_event *) - ((((EthrPtrSzUInt) atsev) & ~ETHR_CACHE_LINE_MASK) + ((((ethr_uint_t) atsev) & ~ETHR_CACHE_LINE_MASK) + ETHR_CACHE_LINE_SIZE)); for (i = 1; i < size; i++) { atsev[i-1].ts_ev.next = &atsev[i].ts_ev; - ethr_atomic_init(&atsev[i-1].ts_ev.uaflgs, 0); + ethr_atomic32_init(&atsev[i-1].ts_ev.uaflgs, 0); atsev[i-1].ts_ev.iflgs = 0; } - ethr_atomic_init(&atsev[size-1].ts_ev.uaflgs, 0); + ethr_atomic32_init(&atsev[size-1].ts_ev.uaflgs, 0); atsev[size-1].ts_ev.iflgs = 0; atsev[size-1].ts_ev.next = NULL; if (endpp) @@ -466,170 +450,6 @@ int ethr_get_main_thr_status(int *on) return 0; } - -/* Atomics */ - -void -ethr_atomic_init(ethr_atomic_t *var, long i) -{ - ETHR_ASSERT(var); - ethr_atomic_init__(var, i); -} - -void -ethr_atomic_set(ethr_atomic_t *var, long i) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - ethr_atomic_set__(var, i); -} - -long -ethr_atomic_read(ethr_atomic_t *var) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - return ethr_atomic_read__(var); -} - - -long -ethr_atomic_add_read(ethr_atomic_t *var, long incr) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - return ethr_atomic_add_read__(var, incr); -} - -long -ethr_atomic_inc_read(ethr_atomic_t *var) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - return ethr_atomic_inc_read__(var); -} - -long -ethr_atomic_dec_read(ethr_atomic_t *var) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - return ethr_atomic_dec_read__(var); -} - -void -ethr_atomic_add(ethr_atomic_t *var, long incr) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - ethr_atomic_add__(var, incr); -} - -void -ethr_atomic_inc(ethr_atomic_t *var) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - ethr_atomic_inc__(var); -} - -void -ethr_atomic_dec(ethr_atomic_t *var) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - ethr_atomic_dec__(var); -} - -long -ethr_atomic_read_band(ethr_atomic_t *var, long mask) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - return ethr_atomic_read_band__(var, mask); -} - -long -ethr_atomic_read_bor(ethr_atomic_t *var, long mask) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - return ethr_atomic_read_bor__(var, mask); -} - -long -ethr_atomic_xchg(ethr_atomic_t *var, long new) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - return ethr_atomic_xchg__(var, new); -} - -long -ethr_atomic_cmpxchg(ethr_atomic_t *var, long new, long expected) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - return ethr_atomic_cmpxchg__(var, new, expected); -} - -long -ethr_atomic_read_acqb(ethr_atomic_t *var) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - return ethr_atomic_read_acqb__(var); -} - -long -ethr_atomic_inc_read_acqb(ethr_atomic_t *var) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - return ethr_atomic_inc_read_acqb__(var); -} - -void -ethr_atomic_set_relb(ethr_atomic_t *var, long i) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - ethr_atomic_set_relb__(var, i); -} - -void -ethr_atomic_dec_relb(ethr_atomic_t *var) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - ethr_atomic_dec_relb__(var); -} - -long -ethr_atomic_dec_read_relb(ethr_atomic_t *var) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - return ethr_atomic_dec_read_relb__(var); -} - -long -ethr_atomic_cmpxchg_acqb(ethr_atomic_t *var, long new, long exp) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - return ethr_atomic_cmpxchg_acqb__(var, new, exp); -} - -long -ethr_atomic_cmpxchg_relb(ethr_atomic_t *var, long new, long exp) -{ - ETHR_ASSERT(!ethr_not_inited__); - ETHR_ASSERT(var); - return ethr_atomic_cmpxchg_relb__(var, new, exp); -} - - /* Spinlocks and rwspinlocks */ int diff --git a/erts/lib_src/common/ethr_mutex.c b/erts/lib_src/common/ethr_mutex.c index 78323b62a3..2ddef32dfc 100644 --- a/erts/lib_src/common/ethr_mutex.c +++ b/erts/lib_src/common/ethr_mutex.c @@ -205,12 +205,17 @@ static void hard_debug_chk_q__(struct ethr_mutex_base_ *, int); #ifdef ETHR_USE_OWN_RWMTX_IMPL__ static void +rwmutex_transfer_read_lock(ethr_rwmutex *rwmtx, + ethr_sint32_t initial, + int q_locked); +static void rwmutex_unlock_wake(ethr_rwmutex *rwmtx, int have_w, - long initial); + ethr_sint32_t initial, + int transfer_read_lock); static int rwmutex_try_complete_runlock(ethr_rwmutex *rwmtx, - long initial, + ethr_sint32_t initial, ethr_ts_event *tse, int start_next_ix, int check_before_try, @@ -237,12 +242,12 @@ rwmutex_freqread_rdrs_add(ethr_rwmutex *rwmtx, int inc) { if (type == ETHR_RWMUTEX_TYPE_FREQUENT_READ || ix == 0) - ethr_atomic_add(&rwmtx->tdata.ra[ix].data.readers, inc); + ethr_atomic32_add(&rwmtx->tdata.ra[ix].data.readers, inc); else { ETHR_ASSERT(type == ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ); - ETHR_ASSERT(ethr_atomic_read(&rwmtx->tdata.ra[ix].data.readers) == 0); + ETHR_ASSERT(ethr_atomic32_read(&rwmtx->tdata.ra[ix].data.readers) == 0); ETHR_ASSERT(inc == 1); - ethr_atomic_set(&rwmtx->tdata.ra[ix].data.readers, (long) 1); + ethr_atomic32_set(&rwmtx->tdata.ra[ix].data.readers, (ethr_sint32_t) 1); } } @@ -253,18 +258,20 @@ rwmutex_freqread_rdrs_inc(ethr_rwmutex *rwmtx, ethr_ts_event *tse) if (rwmtx->type == ETHR_RWMUTEX_TYPE_FREQUENT_READ) { ix = tse->rgix; atomic_inc: - ethr_atomic_inc(&rwmtx->tdata.ra[ix].data.readers); + ethr_atomic32_inc(&rwmtx->tdata.ra[ix].data.readers); } else { ix = tse->mtix; if (ix == 0) goto atomic_inc; ETHR_ASSERT(rwmtx->type == ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ); - ETHR_ASSERT(ethr_atomic_read(&rwmtx->tdata.ra[ix].data.readers) == 0); - ethr_atomic_set(&rwmtx->tdata.ra[ix].data.readers, (long) 1); + ETHR_ASSERT(ethr_atomic32_read(&rwmtx->tdata.ra[ix].data.readers) == 0); + ethr_atomic32_set(&rwmtx->tdata.ra[ix].data.readers, (ethr_sint32_t) 1); } } +#if 0 /* Not used */ + static ETHR_INLINE void rwmutex_freqread_rdrs_dec(ethr_rwmutex *rwmtx, ethr_ts_event *tse) { @@ -272,69 +279,72 @@ rwmutex_freqread_rdrs_dec(ethr_rwmutex *rwmtx, ethr_ts_event *tse) if (rwmtx->type == ETHR_RWMUTEX_TYPE_FREQUENT_READ) { ix = tse->rgix; atomic_dec: - ethr_atomic_dec(&rwmtx->tdata.ra[ix].data.readers); + ethr_atomic32_dec(&rwmtx->tdata.ra[ix].data.readers); } else { ix = tse->mtix; if (ix == 0) goto atomic_dec; ETHR_ASSERT(rwmtx->type == ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ); - ETHR_ASSERT(ethr_atomic_read(&rwmtx->tdata.ra[ix].data.readers) == 1); - ethr_atomic_set(&rwmtx->tdata.ra[ix].data.readers, (long) 0); + ETHR_ASSERT(ethr_atomic32_read(&rwmtx->tdata.ra[ix].data.readers) == 1); + ethr_atomic32_set(&rwmtx->tdata.ra[ix].data.readers, (ethr_sint32_t) 0); } } -static ETHR_INLINE long +#endif + +static ETHR_INLINE ethr_sint32_t rwmutex_freqread_rdrs_dec_read(ethr_rwmutex *rwmtx, ethr_ts_event *tse) { int ix; if (rwmtx->type == ETHR_RWMUTEX_TYPE_FREQUENT_READ) { ix = tse->rgix; atomic_dec_read: - return ethr_atomic_dec_read(&rwmtx->tdata.ra[ix].data.readers); + return ethr_atomic32_dec_read(&rwmtx->tdata.ra[ix].data.readers); } else { ix = tse->mtix; if (ix == 0) goto atomic_dec_read; ETHR_ASSERT(rwmtx->type == ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ); - ETHR_ASSERT(ethr_atomic_read(&rwmtx->tdata.ra[ix].data.readers) == 1); - ethr_atomic_set(&rwmtx->tdata.ra[ix].data.readers, (long) 0); - return (long) 0; + ETHR_ASSERT(ethr_atomic32_read(&rwmtx->tdata.ra[ix].data.readers) == 1); + ethr_atomic32_set(&rwmtx->tdata.ra[ix].data.readers, (ethr_sint32_t) 0); + return (ethr_sint32_t) 0; } } -static ETHR_INLINE long +static ETHR_INLINE ethr_sint32_t rwmutex_freqread_rdrs_dec_read_relb(ethr_rwmutex *rwmtx, ethr_ts_event *tse) { int ix; if (rwmtx->type == ETHR_RWMUTEX_TYPE_FREQUENT_READ) { ix = tse->rgix; atomic_dec_read: - return ethr_atomic_dec_read_relb(&rwmtx->tdata.ra[ix].data.readers); + return ethr_atomic32_dec_read_relb(&rwmtx->tdata.ra[ix].data.readers); } else { ix = tse->mtix; if (ix == 0) goto atomic_dec_read; ETHR_ASSERT(rwmtx->type == ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ); - ETHR_ASSERT(ethr_atomic_read(&rwmtx->tdata.ra[ix].data.readers) == 1); - ethr_atomic_set_relb(&rwmtx->tdata.ra[ix].data.readers, (long) 0); - return (long) 0; + ETHR_ASSERT(ethr_atomic32_read(&rwmtx->tdata.ra[ix].data.readers) == 1); + ethr_atomic32_set_relb(&rwmtx->tdata.ra[ix].data.readers, + (ethr_sint32_t) 0); + return (ethr_sint32_t) 0; } } -static ETHR_INLINE long +static ETHR_INLINE ethr_sint32_t rwmutex_freqread_rdrs_read(ethr_rwmutex *rwmtx, int ix) { - long res = ethr_atomic_read(&rwmtx->tdata.ra[ix].data.readers); + ethr_sint32_t res = ethr_atomic32_read(&rwmtx->tdata.ra[ix].data.readers); #ifdef ETHR_DEBUG switch (rwmtx->type) { case ETHR_RWMUTEX_TYPE_FREQUENT_READ: ETHR_ASSERT(res >= 0); break; case ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ: - ETHR_ASSERT(res == 0 || res == 1); + ETHR_ASSERT(ix == 0 ? res >= 0 : (res == 0 || res == 1)); break; default: ETHR_ASSERT(0); @@ -393,18 +403,19 @@ static void event_wait(struct ethr_mutex_base_ *mtxb, ethr_ts_event *tse, int spincount, - long type, + ethr_sint32_t type, int is_rwmtx, int is_freq_read) { int locked = 0; - long act; + ethr_sint32_t act; int need_try_complete_runlock = 0; + int transfer_read_lock = 0; /* Need to enqueue and wait... */ tse->uflgs = type; - ethr_atomic_set(&tse->uaflgs, type); + ethr_atomic32_set(&tse->uaflgs, type); ETHR_MTX_Q_LOCK(&mtxb->qlck); locked = 1; @@ -413,7 +424,7 @@ event_wait(struct ethr_mutex_base_ *mtxb, hard_debug_chk_q__(mtxb, is_rwmtx); #endif - act = ethr_atomic_read(&mtxb->flgs); + act = ethr_atomic32_read(&mtxb->flgs); if (act & type) { @@ -443,9 +454,9 @@ event_wait(struct ethr_mutex_base_ *mtxb, /* Set wait bit */ while (1) { - long new, exp = act; - int freqread_tryrlock = 0; + ethr_sint32_t new, exp = act; need_try_complete_runlock = 0; + transfer_read_lock = 0; if (type == ETHR_RWMTX_W_WAIT_FLG__) { if (is_freq_read && act == ETHR_RWMTX_R_FLG__) @@ -465,19 +476,16 @@ event_wait(struct ethr_mutex_base_ *mtxb, new = act + 1; /* Try to get it */ } else { - if (act & ~ETHR_RWMTX_R_FLG__) - new = act | ETHR_RWMTX_R_WAIT_FLG__; - else { /* Try to get it */ - ethr_rwmutex *rwmtx = (ethr_rwmutex *) mtxb; - rwmutex_freqread_rdrs_inc(rwmtx, tse); - ETHR_MEMORY_BARRIER; - new = act | ETHR_RWMTX_R_FLG__; - freqread_tryrlock = 1; + new = act | ETHR_RWMTX_R_WAIT_FLG__; + if ((act & (ETHR_RWMTX_W_FLG__ + | ETHR_RWMTX_W_WAIT_FLG__)) == 0) { + /* Transfer read lock to this thread. */ + transfer_read_lock = 1; } } } - act = ethr_atomic_cmpxchg_acqb(&mtxb->flgs, new, exp); + act = ethr_atomic32_cmpxchg_acqb(&mtxb->flgs, new, exp); if (exp == act) { if (new & type) { act = new; @@ -488,24 +496,6 @@ event_wait(struct ethr_mutex_base_ *mtxb, goto done; } } - - if (freqread_tryrlock) { - ethr_rwmutex *rwmtx = (ethr_rwmutex *) mtxb; - - /* We didn't set ETHR_RWMTX_R_FLG__, however someone - else might have */ - if (act == ETHR_RWMTX_R_FLG__) - goto done; /* Got it by help from someone else */ - - ETHR_ASSERT((act & ETHR_RWMTX_WAIT_FLGS__) == 0); - /* - * We know that no waiter flags have been set, i.e., - * we cannot get into a situation where we need to wake - * someone up here. Just restore the readers counter - * and do it over again... - */ - rwmutex_freqread_rdrs_dec(rwmtx, tse); - } } /* Enqueue */ @@ -535,26 +525,42 @@ event_wait(struct ethr_mutex_base_ *mtxb, /* Wait */ locked = 0; - ETHR_MTX_Q_UNLOCK(&mtxb->qlck); - if (need_try_complete_runlock) { + ETHR_ASSERT(!(transfer_read_lock && need_try_complete_runlock)); + + if (transfer_read_lock) { ETHR_ASSERT(((ethr_rwmutex *) mtxb)->type != ETHR_RWMUTEX_TYPE_NORMAL); /* - * We were the only one in queue when we enqueued, and it - * was seemingly read locked. We need to try to complete a - * runlock otherwise we might be hanging forever. If the - * runlock could be completed we will be dequeued and - * woken by ourselves. + * We are the only one in the queue and we are not write + * locked; rwmutex_transfer_read_lock() will: + * - transfer a read lock to us (since we're first in q) + * - unlock the Q-lock */ - rwmutex_try_complete_runlock((ethr_rwmutex *) mtxb, - act, tse, 0, 1, 0); + rwmutex_transfer_read_lock(((ethr_rwmutex *) mtxb), act, 1); + } + else { + ETHR_MTX_Q_UNLOCK(&mtxb->qlck); + + if (need_try_complete_runlock) { + ETHR_ASSERT(((ethr_rwmutex *) mtxb)->type + != ETHR_RWMUTEX_TYPE_NORMAL); + /* + * We were the only one in queue when we enqueued, and it + * was seemingly read locked. We need to try to complete a + * runlock otherwise we might be hanging forever. If the + * runlock could be completed we will be dequeued and + * woken by ourselves. + */ + rwmutex_try_complete_runlock((ethr_rwmutex *) mtxb, + act, tse, 0, 1, 0); + } } while (1) { ethr_event_reset(&tse->event); - act = ethr_atomic_read_acqb(&tse->uaflgs); + act = ethr_atomic32_read_acqb(&tse->uaflgs); if (!act) goto done; /* Got it */ @@ -562,7 +568,7 @@ event_wait(struct ethr_mutex_base_ *mtxb, ethr_event_swait(&tse->event, spincount); /* swait result: 0 || EINTR */ - act = ethr_atomic_read_acqb(&tse->uaflgs); + act = ethr_atomic32_read_acqb(&tse->uaflgs); if (!act) goto done; /* Got it */ } @@ -582,7 +588,7 @@ wake_writer(struct ethr_mutex_base_ *mtxb, int is_rwmtx) dequeue(&mtxb->q, tse, tse); ETHR_ASSERT(tse->uflgs == ETHR_RWMTX_W_WAIT_FLG__); - ETHR_ASSERT(ethr_atomic_read(&tse->uaflgs) == ETHR_RWMTX_W_WAIT_FLG__); + ETHR_ASSERT(ethr_atomic32_read(&tse->uaflgs) == ETHR_RWMTX_W_WAIT_FLG__); #ifdef ETHR_MTX_HARD_DEBUG_WSQ mtxb->ws--; #endif @@ -592,7 +598,7 @@ wake_writer(struct ethr_mutex_base_ *mtxb, int is_rwmtx) ETHR_MTX_Q_UNLOCK(&mtxb->qlck); - ethr_atomic_set(&tse->uaflgs, 0); + ethr_atomic32_set(&tse->uaflgs, 0); ethr_event_set(&tse->event); } @@ -644,17 +650,15 @@ int check_readers_array(ethr_rwmutex *rwmtx, static ETHR_INLINE void write_lock_wait(struct ethr_mutex_base_ *mtxb, - long initial, + ethr_sint32_t initial, int is_rwmtx, int is_freq_read) { - long act = initial; + ethr_sint32_t act = initial; int scnt, start_scnt; ethr_ts_event *tse = NULL; int until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; int res; - int freq_read_size = -1; - int freq_read_start_ix = -1; ETHR_ASSERT(!is_freq_read || is_rwmtx); @@ -666,44 +670,23 @@ write_lock_wait(struct ethr_mutex_base_ *mtxb, */ while (1) { - long exp; - while (act != 0) { if (is_freq_read && act == ETHR_RWMTX_R_FLG__) { ethr_rwmutex *rwmtx = (ethr_rwmutex *) mtxb; + scnt--; if (!tse) tse = ethr_get_ts_event(); - if (freq_read_size < 0) { - if (rwmtx->type == ETHR_RWMUTEX_TYPE_FREQUENT_READ) { - freq_read_size = reader_groups_array_size; - freq_read_start_ix = tse->rgix; - } - else { - freq_read_size = main_threads_array_size; - freq_read_start_ix = tse->mtix; - } - } - res = check_readers_array(rwmtx, - freq_read_start_ix, - freq_read_size); - scnt--; - if (res == 0) { - act = ethr_atomic_read(&mtxb->flgs); - if (act & ETHR_RWMTX_R_MASK__) { - res = rwmutex_try_complete_runlock(rwmtx, act, - tse, 0, 0, - 1); - if (res != EBUSY) - goto done; /* Got it */ - } - if (scnt <= 0) - goto chk_spin; - if (--until_yield == 0) { - until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; - ETHR_YIELD(); - } - continue; + res = rwmutex_try_complete_runlock(rwmtx, act, + tse, 0, 0, + 1); + if (res != EBUSY) + goto done; /* Got it */ + if (scnt <= 0) + goto chk_spin; + if (--until_yield == 0) { + until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; + ETHR_YIELD(); } } @@ -724,15 +707,13 @@ write_lock_wait(struct ethr_mutex_base_ *mtxb, until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; ETHR_YIELD(); } - act = ethr_atomic_read(&mtxb->flgs); + act = ethr_atomic32_read(&mtxb->flgs); scnt--; } - exp = act; - - act = ethr_atomic_cmpxchg_acqb(&mtxb->flgs, - ETHR_RWMTX_W_FLG__, - exp); + act = ethr_atomic32_cmpxchg_acqb(&mtxb->flgs, + ETHR_RWMTX_W_FLG__, + 0); if (act == 0) goto done; /* Got it */ } @@ -753,6 +734,7 @@ mtxb_init(struct ethr_mutex_base_ *mtxb, #ifdef ETHR_MTX_HARD_DEBUG_WSQ mtxb->ws = 0; #endif + ETHR_MTX_CHK_EXCL_INIT(mtxb); if (no_spin) { mtxb->main_scnt = 0; mtxb->aux_scnt = 0; @@ -775,16 +757,16 @@ mtxb_init(struct ethr_mutex_base_ *mtxb, } mtxb->q = NULL; - ethr_atomic_init(&mtxb->flgs, 0); + ethr_atomic32_init(&mtxb->flgs, 0); return ETHR_MTX_QLOCK_INIT(&mtxb->qlck); } static int mtxb_destroy(struct ethr_mutex_base_ *mtxb) { - long act; + ethr_sint32_t act; ETHR_MTX_Q_LOCK(&mtxb->qlck); - act = ethr_atomic_read(&mtxb->flgs); + act = ethr_atomic32_read(&mtxb->flgs); ETHR_MTX_Q_UNLOCK(&mtxb->qlck); if (act != 0) return EINVAL; @@ -850,13 +832,13 @@ ethr_mutex_destroy(ethr_mutex *mtx) } void -ethr_mutex_lock_wait__(ethr_mutex *mtx, long initial) +ethr_mutex_lock_wait__(ethr_mutex *mtx, ethr_sint32_t initial) { write_lock_wait(&mtx->mtxb, initial, 0, 0); } void -ethr_mutex_unlock_wake__(ethr_mutex *mtx, long initial) +ethr_mutex_unlock_wake__(ethr_mutex *mtx, ethr_sint32_t initial) { ethr_ts_event *tse; @@ -864,7 +846,7 @@ ethr_mutex_unlock_wake__(ethr_mutex *mtx, long initial) tse = mtx->mtxb.q; ETHR_ASSERT(tse); - ETHR_ASSERT(ethr_atomic_read(&mtx->mtxb.flgs) + ETHR_ASSERT(ethr_atomic32_read(&mtx->mtxb.flgs) == (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)); ETHR_ASSERT(initial & ETHR_RWMTX_W_WAIT_FLG__); ETHR_MTX_HARD_DEBUG_CHK_Q(mtx); @@ -874,7 +856,7 @@ ethr_mutex_unlock_wake__(ethr_mutex *mtx, long initial) * mtxb->flgs; otherwise, we need to clear the write wait bit... */ if (tse->next == mtx->mtxb.q) - ethr_atomic_set(&mtx->mtxb.flgs, ETHR_RWMTX_W_FLG__); + ethr_atomic32_set(&mtx->mtxb.flgs, ETHR_RWMTX_W_FLG__); wake_writer(&mtx->mtxb, 0); } @@ -884,7 +866,7 @@ ethr_mutex_unlock_wake__(ethr_mutex *mtx, long initial) static void enqueue_mtx(ethr_mutex *mtx, ethr_ts_event *tse_start, ethr_ts_event *tse_end) { - long act; + ethr_sint32_t act; /* * `ethr_cond_signal()' and `ethr_cond_broadcast()' end up here. If `mtx' @@ -913,7 +895,7 @@ enqueue_mtx(ethr_mutex *mtx, ethr_ts_event *tse_start, ethr_ts_event *tse_end) } #endif - act = ethr_atomic_read(&mtx->mtxb.flgs); + act = ethr_atomic32_read(&mtx->mtxb.flgs); ETHR_ASSERT(act == 0 || act == ETHR_RWMTX_W_FLG__ || act == (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)); @@ -921,10 +903,10 @@ enqueue_mtx(ethr_mutex *mtx, ethr_ts_event *tse_start, ethr_ts_event *tse_end) /* The normal sane case */ if (!(act & ETHR_RWMTX_W_WAIT_FLG__)) { ETHR_ASSERT(!mtx->mtxb.q); - act = ethr_atomic_cmpxchg(&mtx->mtxb.flgs, - (ETHR_RWMTX_W_FLG__ - | ETHR_RWMTX_W_WAIT_FLG__), - ETHR_RWMTX_W_FLG__); + act = ethr_atomic32_cmpxchg(&mtx->mtxb.flgs, + (ETHR_RWMTX_W_FLG__ + | ETHR_RWMTX_W_WAIT_FLG__), + ETHR_RWMTX_W_FLG__); if (act != ETHR_RWMTX_W_FLG__) { /* * Sigh... this wasn't so sane after all since, the mutex was @@ -956,14 +938,14 @@ enqueue_mtx(ethr_mutex *mtx, ethr_ts_event *tse_start, ethr_ts_event *tse_end) multi = tse_start != tse_end; while (1) { - long new, exp = act; + ethr_sint32_t new, exp = act; if (multi || (act & ETHR_RWMTX_W_FLG__)) new = ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__; else new = ETHR_RWMTX_W_FLG__; - act = ethr_atomic_cmpxchg(&mtx->mtxb.flgs, new, exp); + act = ethr_atomic32_cmpxchg(&mtx->mtxb.flgs, new, exp); if (exp == act) { ETHR_ASSERT(!mtx->mtxb.q); if (act & ETHR_RWMTX_W_FLG__) { @@ -991,7 +973,7 @@ enqueue_mtx(ethr_mutex *mtx, ethr_ts_event *tse_start, ethr_ts_event *tse_end) ETHR_MTX_HARD_DEBUG_CHK_Q(mtx); ETHR_MTX_Q_UNLOCK(&mtx->mtxb.qlck); - ethr_atomic_set(&tse_start->uaflgs, 0); + ethr_atomic32_set(&tse_start->uaflgs, 0); ethr_event_set(&tse_start->event); } break; @@ -1082,9 +1064,9 @@ ethr_cond_signal(ethr_cond *cnd) ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); ETHR_ASSERT(tse->uflgs == ETHR_RWMTX_W_WAIT_FLG__); - ETHR_ASSERT(ethr_atomic_read(&tse->uaflgs) == ETHR_CND_WAIT_FLG__); + ETHR_ASSERT(ethr_atomic32_read(&tse->uaflgs) == ETHR_CND_WAIT_FLG__); - ethr_atomic_set(&tse->uaflgs, ETHR_RWMTX_W_WAIT_FLG__); + ethr_atomic32_set(&tse->uaflgs, ETHR_RWMTX_W_WAIT_FLG__); dequeue(&cnd->q, tse, tse); @@ -1135,10 +1117,11 @@ ethr_cond_broadcast(ethr_cond *cnd) /* The normal case */ ETHR_ASSERT(tse_tmp->uflgs == ETHR_RWMTX_W_WAIT_FLG__); - ETHR_ASSERT(ethr_atomic_read(&tse_tmp->uaflgs) + ETHR_ASSERT(ethr_atomic32_read(&tse_tmp->uaflgs) == ETHR_CND_WAIT_FLG__); - ethr_atomic_set(&tse_tmp->uaflgs, ETHR_RWMTX_W_WAIT_FLG__); + ethr_atomic32_set(&tse_tmp->uaflgs, + ETHR_RWMTX_W_WAIT_FLG__); } else { /* Should be very unusual */ @@ -1191,7 +1174,7 @@ ethr_cond_wait(ethr_cond *cnd, ethr_mutex *mtx) tse->udata = (void *) mtx; tse->uflgs = ETHR_RWMTX_W_WAIT_FLG__; /* Prep for mutex lock op */ - ethr_atomic_set(&tse->uaflgs, ETHR_CND_WAIT_FLG__); + ethr_atomic32_set(&tse->uaflgs, ETHR_CND_WAIT_FLG__); ETHR_MTX_Q_LOCK(&cnd->qlck); @@ -1204,11 +1187,11 @@ ethr_cond_wait(ethr_cond *cnd, ethr_mutex *mtx) /* Wait */ woken = 0; while (1) { - long act; + ethr_sint32_t act; ethr_event_reset(&tse->event); - act = ethr_atomic_read_acqb(&tse->uaflgs); + act = ethr_atomic32_read_acqb(&tse->uaflgs); if (!act) break; /* Mtx locked */ @@ -1224,7 +1207,7 @@ ethr_cond_wait(ethr_cond *cnd, ethr_mutex *mtx) */ if (act == ETHR_CND_WAIT_FLG__) { ETHR_MTX_Q_LOCK(&cnd->qlck); - act = ethr_atomic_read(&tse->uaflgs); + act = ethr_atomic32_read(&tse->uaflgs); ETHR_ASSERT(act == ETHR_CND_WAIT_FLG__ || act == ETHR_RWMTX_W_WAIT_FLG__); /* @@ -1254,7 +1237,7 @@ ethr_cond_wait(ethr_cond *cnd, ethr_mutex *mtx) ETHR_MTX_HARD_DEBUG_LFS_RWLOCK(&mtx->mtxb); ETHR_MTX_HARD_DEBUG_FENCE_CHK(cnd); ETHR_MTX_HARD_DEBUG_FENCE_CHK(mtx); - + ETHR_MTX_CHK_EXCL_SET_EXCL(&mtx->mtxb); tse->udata = udata; ethr_leave_ts_event(tse); return 0; @@ -1426,7 +1409,7 @@ wake_readers(ethr_rwmutex *rwmtx, int rs) rwmtx->rq_end = NULL; ETHR_ASSERT(!rwmtx->mtxb.q - || (ethr_atomic_read(&rwmtx->mtxb.q->uaflgs) + || (ethr_atomic32_read(&rwmtx->mtxb.q->uaflgs) == ETHR_RWMTX_W_WAIT_FLG__)); ETHR_RWMTX_HARD_DEBUG_CHK_Q(rwmtx); @@ -1437,7 +1420,7 @@ wake_readers(ethr_rwmutex *rwmtx, int rs) #ifdef ETHR_DEBUG ETHR_ASSERT(tse->uflgs == ETHR_RWMTX_R_WAIT_FLG__); - ETHR_ASSERT(ethr_atomic_read(&tse->uaflgs) + ETHR_ASSERT(ethr_atomic32_read(&tse->uaflgs) == ETHR_RWMTX_R_WAIT_FLG__); drs++; #endif @@ -1445,7 +1428,7 @@ wake_readers(ethr_rwmutex *rwmtx, int rs) tse_next = tse->next; /* we aren't allowed to read tse->next after we have reset uaflgs */ - ethr_atomic_set(&tse->uaflgs, 0); + ethr_atomic32_set(&tse->uaflgs, 0); ethr_event_set(&tse->event); tse = tse_next; } @@ -1488,7 +1471,7 @@ int check_readers_array(ethr_rwmutex *rwmtx, ETHR_MEMORY_BARRIER; do { - long act = rwmutex_freqread_rdrs_read(rwmtx, ix); + ethr_sint32_t act = rwmutex_freqread_rdrs_read(rwmtx, ix); if (act != 0) return EBUSY; ix++; @@ -1499,55 +1482,101 @@ int check_readers_array(ethr_rwmutex *rwmtx, return 0; } -static ETHR_INLINE void +static void +rwmutex_freqread_rdrs_dec_chk_wakeup(ethr_rwmutex *rwmtx, + ethr_ts_event *tse, + ethr_sint32_t initial) +{ + ethr_sint32_t act = initial; + + if ((act & (ETHR_RWMTX_W_FLG__| + ETHR_RWMTX_R_ABRT_UNLCK_FLG__)) == 0) { + if ((act & ETHR_RWMTX_WAIT_FLGS__) == 0) { + if (act & ETHR_RWMTX_R_PEND_UNLCK_MASK__) { + /* + * We *need* to try to complete the runlock. + * A writer that just enqueued (not seen by us + * in flag field) may depend on someone else + * completing the runlock. We just took over + * that responsibilty since we modified reader + * groups. + */ + rwmutex_try_complete_runlock(rwmtx, act, tse, 1, 0, 0); + } + } + else if ((act & ETHR_RWMTX_WAIT_FLGS__) == ETHR_RWMTX_R_WAIT_FLG__) + rwmutex_transfer_read_lock(rwmtx, act, 0); + else if ((act & ETHR_RWMTX_WAIT_FLGS__) == ETHR_RWMTX_W_WAIT_FLG__) + rwmutex_try_complete_runlock(rwmtx, act, tse, 1, 0, 0); + else { + /* + * Don't know if we got readers or writers + * first in queue; need to peek + */ + ETHR_MTX_Q_LOCK(&rwmtx->mtxb.qlck); + if (!rwmtx->mtxb.q) + ETHR_MTX_Q_UNLOCK(&rwmtx->mtxb.qlck); + else if (is_w_waiter(rwmtx->mtxb.q)) { + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); + ETHR_MTX_Q_UNLOCK(&rwmtx->mtxb.qlck); + if ((act & ETHR_RWMTX_W_FLG__) == 0) + rwmutex_try_complete_runlock(rwmtx, act, tse, 1, 0, 0); + } + else { + /* + * rwmutex_transfer_read_lock() will + * unlock Q lock. + */ + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); + if (act & ETHR_RWMTX_W_FLG__) + ETHR_MTX_Q_UNLOCK(&rwmtx->mtxb.qlck); + else + rwmutex_transfer_read_lock(rwmtx, act, 1); + } + } + } +} + +static void rwmutex_freqread_restore_failed_tryrlock(ethr_rwmutex *rwmtx, ethr_ts_event *tse) { - long act; + ethr_sint32_t act; /* * Restore failed increment */ act = rwmutex_freqread_rdrs_dec_read(rwmtx, tse); - ETHR_WRITE_MEMORY_BARRIER; + ETHR_MEMORY_BARRIER; if (act == 0) { - -#ifndef ETHR_WRITE_MEMORY_BARRIER_IS_FULL - ETHR_READ_MEMORY_BARRIER; -#endif - - act = ethr_atomic_read(&rwmtx->mtxb.flgs); - - if ((act & ETHR_RWMTX_W_FLG__) == 0 - && act & (ETHR_RWMTX_WAIT_FLGS__|ETHR_RWMTX_R_PEND_UNLCK_MASK__)) { - /* - * We either got waiters, or someone else trying - * to read unlock which we might have to help. - */ - rwmutex_try_complete_runlock(rwmtx, act, tse, 1, 1, 0); - } + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); + rwmutex_freqread_rdrs_dec_chk_wakeup(rwmtx, tse, act); } } static int rwmutex_try_complete_runlock(ethr_rwmutex *rwmtx, - long initial, + ethr_sint32_t initial, ethr_ts_event *tse, int start_next_ix, int check_before_try, int try_write_lock) { ethr_ts_event *tse_tmp; - long act = initial; + ethr_sint32_t act = initial; int six, res, length; + ETHR_ASSERT((act & ETHR_RWMTX_W_FLG__) == 0); + + if (act & ETHR_RWMTX_R_ABRT_UNLCK_FLG__) + return try_write_lock ? EBUSY : 0; + tse_tmp = tse; if (!tse_tmp) tse_tmp = ethr_get_ts_event(); - if ((act & ETHR_RWMTX_WAIT_FLGS__) - && (act & ~ETHR_RWMTX_WAIT_FLGS__) == 0) + if ((act & ETHR_RWMTX_WAIT_FLGS__) && (act & ~ETHR_RWMTX_WAIT_FLGS__) == 0) goto check_waiters; if (rwmtx->type == ETHR_RWMUTEX_TYPE_FREQUENT_READ) { @@ -1569,24 +1598,33 @@ rwmutex_try_complete_runlock(ethr_rwmutex *rwmtx, if (check_before_try) { res = check_readers_array(rwmtx, six, length); + + ETHR_MEMORY_BARRIER; + if (res == EBUSY) return try_write_lock ? EBUSY : 0; } + restart: + while (1) { - long exp = act; - long new = act+1; + ethr_sint32_t exp = act; + ethr_sint32_t new = act+1; + + ETHR_ASSERT((act & ETHR_RWMTX_R_ABRT_UNLCK_FLG__) == 0); ETHR_ASSERT((act & ETHR_RWMTX_R_PEND_UNLCK_MASK__) < ETHR_RWMTX_R_PEND_UNLCK_MASK__); - act = ethr_atomic_cmpxchg(&rwmtx->mtxb.flgs, new, exp); + act = ethr_atomic32_cmpxchg(&rwmtx->mtxb.flgs, new, exp); if (exp == act) { act = new; break; } + if (!try_write_lock) { - if (act == ETHR_RWMTX_W_FLG__ || act == 0) + if (act == 0 || (act & (ETHR_RWMTX_W_FLG__ + | ETHR_RWMTX_R_ABRT_UNLCK_FLG__))) return 0; if ((act & ETHR_RWMTX_WAIT_FLGS__) == 0) { if ((act & ETHR_RWMTX_R_FLG__) == 0) @@ -1601,33 +1639,50 @@ rwmutex_try_complete_runlock(ethr_rwmutex *rwmtx, else { if (act == 0) goto tryrwlock; - if (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_WAIT_FLGS__)) + if (act & (ETHR_RWMTX_W_FLG__ + | ETHR_RWMTX_R_ABRT_UNLCK_FLG__)) return EBUSY; } } res = check_readers_array(rwmtx, six, length); - if (res == EBUSY) { - act = ethr_atomic_dec_read(&rwmtx->mtxb.flgs); - if (act & ETHR_RWMTX_R_MASK__) - return try_write_lock ? EBUSY : 0; - } - else { - while (1) { - long exp = act; - long new = act; - new &= ~ETHR_RWMTX_R_FLG__; - new--; - ETHR_ASSERT(act & ETHR_RWMTX_R_PEND_UNLCK_MASK__); + ETHR_MEMORY_BARRIER; - act = ethr_atomic_cmpxchg(&rwmtx->mtxb.flgs, new, exp); - if (exp == act) { - if (new & ETHR_RWMTX_R_PEND_UNLCK_MASK__) - return try_write_lock ? EBUSY : 0; - act = new; - break; + ETHR_ASSERT((act & ETHR_RWMTX_W_FLG__) == 0); + + while (1) { + int finished_abort = 0; + ethr_sint32_t exp = act; + ethr_sint32_t new = act; + + new--; + if (act & ETHR_RWMTX_R_ABRT_UNLCK_FLG__) { + if ((new & ETHR_RWMTX_R_PEND_UNLCK_MASK__) == 0) { + new &= ~ETHR_RWMTX_R_ABRT_UNLCK_FLG__; + finished_abort = 1; } + ETHR_ASSERT(act & ETHR_RWMTX_R_FLG__); + } + else if ((act & ETHR_RWMTX_R_FLG__) && res != EBUSY) { + new &= ~ETHR_RWMTX_R_FLG__; + } + + ETHR_ASSERT(act & ETHR_RWMTX_R_PEND_UNLCK_MASK__); + + act = ethr_atomic32_cmpxchg(&rwmtx->mtxb.flgs, new, exp); + if (exp == act) { + act = new; + if (act & ETHR_RWMTX_W_FLG__) + return try_write_lock ? EBUSY : 0; + if (finished_abort && (act & ETHR_RWMTX_WAIT_FLGS__)) + goto restart; + if (act & (ETHR_RWMTX_R_FLG__ + | ETHR_RWMTX_R_ABRT_UNLCK_FLG__ + | ETHR_RWMTX_R_PEND_UNLCK_MASK__)) + return try_write_lock ? EBUSY : 0; + /* Read unlock completed */ + break; } } @@ -1637,12 +1692,9 @@ rwmutex_try_complete_runlock(ethr_rwmutex *rwmtx, * to write lock it). */ - if (act & ETHR_RWMTX_W_FLG__) - return try_write_lock ? EBUSY : 0; - if (act & ETHR_RWMTX_WAIT_FLGS__) { check_waiters: - rwmutex_unlock_wake(rwmtx, 0, act); + rwmutex_unlock_wake(rwmtx, 0, act, 0); return try_write_lock ? EBUSY : 0; } @@ -1652,9 +1704,9 @@ rwmutex_try_complete_runlock(ethr_rwmutex *rwmtx, tryrwlock: /* Try to write lock it */ - act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, - ETHR_RWMTX_W_FLG__, - 0); + act = ethr_atomic32_cmpxchg_acqb(&rwmtx->mtxb.flgs, + ETHR_RWMTX_W_FLG__, + 0); return act == 0 ? 0 : EBUSY; } @@ -1663,24 +1715,23 @@ rwmutex_try_complete_runlock(ethr_rwmutex *rwmtx, static ETHR_INLINE void rwmutex_incdec_restore_failed_tryrlock(ethr_rwmutex *rwmtx) { - long act; + ethr_sint32_t act; /* * Restore failed increment */ - act = ethr_atomic_dec_read(&rwmtx->mtxb.flgs); + act = ethr_atomic32_dec_read(&rwmtx->mtxb.flgs); if ((act & ETHR_RWMTX_WAIT_FLGS__) && (act & ~ETHR_RWMTX_WAIT_FLGS__) == 0) { - rwmutex_unlock_wake(rwmtx, 0, act); + rwmutex_unlock_wake(rwmtx, 0, act, 0); } } #endif static void -rwmutex_normal_rlock_wait(ethr_rwmutex *rwmtx, - long initial) +rwmutex_normal_rlock_wait(ethr_rwmutex *rwmtx, ethr_sint32_t initial) { - long act = initial, exp; + ethr_sint32_t act = initial, exp; int scnt, start_scnt; ethr_ts_event *tse = NULL; int until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; @@ -1696,11 +1747,11 @@ rwmutex_normal_rlock_wait(ethr_rwmutex *rwmtx, #ifdef ETHR_RLOCK_WITH_INC_DEC rwmutex_incdec_restore_failed_tryrlock(rwmtx); - act = ethr_atomic_read(&rwmtx->mtxb.flgs); + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); #endif while (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)) { - if (scnt >= 0) { + if (scnt <= 0) { tse = ethr_get_ts_event(); if (update_spincount(&rwmtx->mtxb, tse, &start_scnt, &scnt)) { event_wait(&rwmtx->mtxb, tse, scnt, @@ -1713,17 +1764,17 @@ rwmutex_normal_rlock_wait(ethr_rwmutex *rwmtx, until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; ETHR_YIELD(); } - act = ethr_atomic_read(&rwmtx->mtxb.flgs); + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); scnt--; } exp = act; #ifdef ETHR_RLOCK_WITH_INC_DEC - act = ethr_atomic_inc_read(&rwmtx->mtxb.flgs); + act = ethr_atomic32_inc_read(&rwmtx->mtxb.flgs); if ((act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)) == 0) goto done; /* Got it */ #else - act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, exp+1, exp); + act = ethr_atomic32_cmpxchg_acqb(&rwmtx->mtxb.flgs, exp+1, exp); if (act == exp) goto done; /* Got it */ #endif @@ -1736,10 +1787,83 @@ rwmutex_normal_rlock_wait(ethr_rwmutex *rwmtx, static void rwmutex_freqread_rlock_wait(ethr_rwmutex *rwmtx, - ethr_ts_event *tse, - long initial) + ethr_ts_event *tse); + +static int +rwmutex_freqread_rlock(ethr_rwmutex *rwmtx, ethr_ts_event *tse, int trylock) { - long act = initial; + int res = 0; + ethr_sint32_t act; + + rwmutex_freqread_rdrs_inc(rwmtx, tse); + + ETHR_MEMORY_BARRIER; + + act = ethr_atomic32_read_acqb(&rwmtx->mtxb.flgs); + + if (act != ETHR_RWMTX_R_FLG__) { + int wake_other_readers; + + while (1) { + ethr_sint32_t exp, new; + + wake_other_readers = 0; + + if (act == 0) + new = act | ETHR_RWMTX_R_FLG__; + else if (act == ETHR_RWMTX_R_FLG__) + break; /* Got it */ + else if (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)) { + rwmutex_freqread_restore_failed_tryrlock(rwmtx, tse); + if (trylock) + res = EBUSY; + else + rwmutex_freqread_rlock_wait(rwmtx, tse); + break; + } + else if (act & ETHR_RWMTX_R_ABRT_UNLCK_FLG__) { + if ((act & ETHR_RWMTX_R_FLG__) == 0) + ETHR_FATAL_ERROR__(EFAULT); + /* + * An aborted runlock, not write locked, and no write + * waiters, i.e., we got it... + */ + if (act & ETHR_RWMTX_R_WAIT_FLG__) + wake_other_readers = 1; + break; + } + else { + new = act | ETHR_RWMTX_R_FLG__; + if (act & ETHR_RWMTX_R_PEND_UNLCK_MASK__) { + /* + * Someone is doing tryrwlock (no writer and no + * write waiters); we will try to abort that... + */ + new |= ETHR_RWMTX_R_ABRT_UNLCK_FLG__; + } + + if (act & ETHR_RWMTX_R_WAIT_FLG__) + wake_other_readers = 1; + } + + exp = act; + act = ethr_atomic32_cmpxchg_acqb(&rwmtx->mtxb.flgs, new, exp); + if (act == exp) + break; + } + + if (wake_other_readers) + rwmutex_transfer_read_lock(rwmtx, act, 0); + } + + return res; +} + +static void +rwmutex_freqread_rlock_wait(ethr_rwmutex *rwmtx, + ethr_ts_event *tse) +{ + ethr_sint32_t act; int scnt, start_scnt; int until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; @@ -1752,12 +1876,10 @@ rwmutex_freqread_rlock_wait(ethr_rwmutex *rwmtx, while (1) { - rwmutex_freqread_restore_failed_tryrlock(rwmtx, tse); + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); - act = ethr_atomic_read(&rwmtx->mtxb.flgs); - - while (act & ~(ETHR_RWMTX_R_FLG__|ETHR_RWMTX_R_WAIT_FLG__)) { - if (scnt >= 0) { + while (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)) { + if (scnt <= 0) { if (update_spincount(&rwmtx->mtxb, tse, &start_scnt, &scnt)) { event_wait(&rwmtx->mtxb, tse, scnt, ETHR_RWMTX_R_WAIT_FLG__, 1, 1); @@ -1769,74 +1891,65 @@ rwmutex_freqread_rlock_wait(ethr_rwmutex *rwmtx, until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; ETHR_YIELD(); } - act = ethr_atomic_read(&rwmtx->mtxb.flgs); + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); scnt--; } - rwmutex_freqread_rdrs_inc(rwmtx, tse); - - ETHR_MEMORY_BARRIER; - - act = ethr_atomic_read(&rwmtx->mtxb.flgs); - - if (act == ETHR_RWMTX_R_FLG__) - return; /* Got it */ - - while (1) { - long exp, new; - - if (act & ~(ETHR_RWMTX_R_FLG__|ETHR_RWMTX_R_WAIT_FLG__)) - break; /* Busy (need to restore inc) */ - - if (act & ETHR_RWMTX_R_FLG__) - return; /* Got it */ - - exp = act; - new = act | ETHR_RWMTX_R_FLG__; - act = ethr_atomic_cmpxchg(&rwmtx->mtxb.flgs, new, exp); - if (act == exp) - return; /* Got it */ - } + if (rwmutex_freqread_rlock(rwmtx, tse, 1) != EBUSY) + break; /* Got it */ } } static void -rwmutex_normal_rwlock_wait(ethr_rwmutex *rwmtx, long initial) +rwmutex_normal_rwlock_wait(ethr_rwmutex *rwmtx, ethr_sint32_t initial) { write_lock_wait(&rwmtx->mtxb, initial, 1, 0); } static void -rwmutex_freqread_rwlock_wait(ethr_rwmutex *rwmtx, long initial) +rwmutex_freqread_rwlock_wait(ethr_rwmutex *rwmtx, ethr_sint32_t initial) { write_lock_wait(&rwmtx->mtxb, initial, 1, 1); } static ETHR_INLINE void -rwlock_wake_set_flags(ethr_rwmutex *rwmtx, long new_initial, int act_initial) +rwlock_wake_set_flags(ethr_rwmutex *rwmtx, + ethr_sint32_t new_initial, + ethr_sint32_t act_initial) { - long act, act_mask; + ethr_sint32_t act, act_mask; + int chk_abrt_flg; + + ETHR_MEMORY_BARRIER; + if (rwmtx->type != ETHR_RWMUTEX_TYPE_NORMAL) { /* r pend unlock mask may vary and must be retained */ act_mask = ETHR_RWMTX_R_PEND_UNLCK_MASK__; + if (new_initial & ETHR_RWMTX_R_FLG__) + chk_abrt_flg = 1; + else + chk_abrt_flg = 0; } else { #ifdef ETHR_RLOCK_WITH_INC_DEC /* rs mask may vary and must be retained */ act_mask = ETHR_RWMTX_RS_MASK__; + chk_abrt_flg = 0; #else /* rs mask always zero */ ETHR_ASSERT((act_initial & ETHR_RWMTX_RS_MASK__) == 0); - ethr_atomic_set(&rwmtx->mtxb.flgs, new_initial); + ethr_atomic32_set(&rwmtx->mtxb.flgs, new_initial); return; #endif } act = act_initial; while (1) { - long exp = act; - long new = new_initial + (act & act_mask); - act = ethr_atomic_cmpxchg(&rwmtx->mtxb.flgs, new, exp); + ethr_sint32_t exp = act; + ethr_sint32_t new = new_initial + (act & act_mask); + if (chk_abrt_flg && (act & act_mask)) + new |= ETHR_RWMTX_R_ABRT_UNLCK_FLG__; + act = ethr_atomic32_cmpxchg(&rwmtx->mtxb.flgs, new, exp); if (act == exp) break; exp = act; @@ -1850,7 +1963,7 @@ dbg_unlock_wake(ethr_rwmutex *rwmtx, int have_w, ethr_ts_event *tse) { - long exp, act, imask; + ethr_sint32_t exp, act, imask; exp = have_w ? ETHR_RWMTX_W_FLG__ : 0; @@ -1872,7 +1985,7 @@ dbg_unlock_wake(ethr_rwmutex *rwmtx, if (rwmtx->rq_end) { exp |= ETHR_RWMTX_R_WAIT_FLG__; } - act = ethr_atomic_read(&rwmtx->mtxb.flgs); + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); ETHR_ASSERT((exp & ~imask) == (act & ~imask)); ETHR_RWMTX_HARD_DEBUG_CHK_Q(rwmtx); @@ -1883,7 +1996,15 @@ dbg_unlock_wake(ethr_rwmutex *rwmtx, exp |= ETHR_RWMTX_R_WAIT_FLG__; if (rwmtx->rq_end->next != rwmtx->mtxb.q) exp |= ETHR_RWMTX_W_WAIT_FLG__; - act = ethr_atomic_read(&rwmtx->mtxb.flgs); + else if (exp == ETHR_RWMTX_R_WAIT_FLG__) { + if (!have_w) { + if (rwmtx->type != ETHR_RWMUTEX_TYPE_NORMAL) + imask |= ETHR_RWMTX_R_FLG__; + else + imask |= ETHR_RWMTX_RS_MASK__; + } + } + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); ETHR_ASSERT((exp & ~imask) == (act & ~imask)); ETHR_RWMTX_HARD_DEBUG_CHK_Q(rwmtx); @@ -1894,41 +2015,85 @@ dbg_unlock_wake(ethr_rwmutex *rwmtx, #endif static void -rwmutex_unlock_wake(ethr_rwmutex *rwmtx, int have_w, long initial) +rwmutex_transfer_read_lock(ethr_rwmutex *rwmtx, + ethr_sint32_t initial, + int q_locked) { - long new, act = initial; - ethr_ts_event *tse; + ethr_sint32_t act = initial; - if ((act & ETHR_RWMTX_WAIT_FLGS__) == 0) { - if (!have_w) + if (!q_locked) { + ethr_ts_event *tse; + ETHR_ASSERT(initial & ETHR_RWMTX_R_WAIT_FLG__); + ETHR_ASSERT((initial & ETHR_RWMTX_W_FLG__) == 0); + ETHR_MTX_Q_LOCK(&rwmtx->mtxb.qlck); + + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); + tse = rwmtx->mtxb.q; + if ((act & ETHR_RWMTX_W_FLG__) || !tse || is_w_waiter(tse)) { + /* Someone else woke the readers up... */ + ETHR_MTX_Q_UNLOCK(&rwmtx->mtxb.qlck); return; - else { - while ((act & ETHR_RWMTX_WAIT_FLGS__) == 0) { - long exp = act; - new = exp & ~ETHR_RWMTX_W_FLG__; - act = ethr_atomic_cmpxchg(&rwmtx->mtxb.flgs, new, exp); - if (act == exp) - return; - } } } - ETHR_MTX_Q_LOCK(&rwmtx->mtxb.qlck); - tse = rwmtx->mtxb.q; + rwmutex_unlock_wake(rwmtx, 0, initial, 1); +} - if (!have_w) { - if (!tse) { +static void +rwmutex_unlock_wake(ethr_rwmutex *rwmtx, int have_w, ethr_sint32_t initial, + int transfer_read_lock) +{ + ethr_sint32_t new, act = initial; + ethr_ts_event *tse; + + if (transfer_read_lock) { + /* + * - Q already locked + * - Got R waiters first in Q + * - Not W locked + */ + tse = rwmtx->mtxb.q; + + ETHR_ASSERT(act & ETHR_RWMTX_R_WAIT_FLG__); + ETHR_ASSERT((act & (ETHR_RWMTX_W_FLG__)) == 0); + ETHR_ASSERT(tse && !is_w_waiter(tse)); + } + else { + + if ((act & ETHR_RWMTX_WAIT_FLGS__) == 0) { + if (!have_w) + return; + else { + while ((act & ETHR_RWMTX_WAIT_FLGS__) == 0) { + ethr_sint32_t exp = act; + new = exp & ~ETHR_RWMTX_W_FLG__; + act = ethr_atomic32_cmpxchg(&rwmtx->mtxb.flgs, new, exp); + if (act == exp) + return; + } + } + } + + ETHR_MTX_Q_LOCK(&rwmtx->mtxb.qlck); + tse = rwmtx->mtxb.q; + + if (!have_w) { + if (!tse) { #ifdef ETHR_DEBUG - act = ethr_atomic_read(&rwmtx->mtxb.flgs); - ETHR_ASSERT((act & ETHR_RWMTX_WAIT_FLGS__) == 0); + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); + ETHR_ASSERT((act & ETHR_RWMTX_WAIT_FLGS__) == 0); #endif - goto already_served; - } - act = ethr_atomic_read(&rwmtx->mtxb.flgs); - if (act & ~ETHR_RWMTX_WAIT_FLGS__) { - already_served: - ETHR_MTX_Q_UNLOCK(&rwmtx->mtxb.qlck); - return; + goto already_served; + } + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); + if (act == (ETHR_RWMTX_R_WAIT_FLG__|ETHR_RWMTX_R_FLG__)) { + ETHR_ASSERT(tse && !is_w_waiter(tse)); + } + else if (act & ~ETHR_RWMTX_WAIT_FLGS__) { + already_served: + ETHR_MTX_Q_UNLOCK(&rwmtx->mtxb.qlck); + return; + } } } @@ -1939,9 +2104,12 @@ rwmutex_unlock_wake(ethr_rwmutex *rwmtx, int have_w, long initial) if (is_w_waiter(tse)) { if (!have_w) { - act = ethr_atomic_read_bor(&rwmtx->mtxb.flgs, + act = ethr_atomic32_read_bor(&rwmtx->mtxb.flgs, ETHR_RWMTX_W_FLG__); - ETHR_ASSERT((act & ~ETHR_RWMTX_WAIT_FLGS__) == 0); + ETHR_ASSERT((act & ~(ETHR_RWMTX_WAIT_FLGS__ + | (rwmtx->type == ETHR_RWMUTEX_TYPE_NORMAL + ? 0 + : ETHR_RWMTX_R_PEND_UNLCK_MASK__))) == 0); ETHR_ASSERT(act & ETHR_RWMTX_W_WAIT_FLG__); act |= ETHR_RWMTX_W_FLG__; } @@ -1968,7 +2136,7 @@ rwmutex_unlock_wake(ethr_rwmutex *rwmtx, int have_w, long initial) if (rwmtx->type == ETHR_RWMUTEX_TYPE_NORMAL) { rs = rwmtx->tdata.rs; - new = (long) rs; + new = (ethr_sint32_t) rs; rwmtx->tdata.rs = 0; } else { @@ -1988,6 +2156,7 @@ rwmutex_unlock_wake(ethr_rwmutex *rwmtx, int have_w, long initial) rwmutex_freqread_rdrs_add(rwmtx, type, ix, wrs); } } + new = ETHR_RWMTX_R_FLG__; } @@ -1995,6 +2164,7 @@ rwmutex_unlock_wake(ethr_rwmutex *rwmtx, int have_w, long initial) new |= ETHR_RWMTX_W_WAIT_FLG__; rwlock_wake_set_flags(rwmtx, new, act); + wake_readers(rwmtx, rs); } } @@ -2022,16 +2192,16 @@ alloc_readers_array(int length, ethr_rwmutex_lived lived) if (!mem) return NULL; - if ((((unsigned long) mem) & ETHR_CACHE_LINE_MASK) == 0) { + if ((((ethr_uint_t) mem) & ETHR_CACHE_LINE_MASK) == 0) { ra = (ethr_rwmtx_readers_array__ *) mem; ra->data.byte_offset = 0; } else { ra = ((ethr_rwmtx_readers_array__ *) - ((((unsigned long) mem) & ~ETHR_CACHE_LINE_MASK) + ((((ethr_uint_t) mem) & ~ETHR_CACHE_LINE_MASK) + ETHR_CACHE_LINE_SIZE)); - ra->data.byte_offset = (int) ((unsigned long) ra - - (unsigned long) mem); + ra->data.byte_offset = (int) ((ethr_uint_t) ra + - (ethr_uint_t) mem); } ra->data.lived = lived; return ra; @@ -2105,7 +2275,7 @@ ethr_rwmutex_init_opt(ethr_rwmutex *rwmtx, ethr_rwmutex_opt *opt) rwmtx->tdata.ra = ra; for (ix = 0; ix < length; ix++) { - ethr_atomic_init(&rwmtx->tdata.ra[ix].data.readers, 0); + ethr_atomic32_init(&rwmtx->tdata.ra[ix].data.readers, 0); rwmtx->tdata.ra[ix].data.waiting_readers = 0; } break; @@ -2157,8 +2327,9 @@ ethr_rwmutex_destroy(ethr_rwmutex *rwmtx) return EINVAL; } #endif + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); if (rwmtx->type != ETHR_RWMUTEX_TYPE_NORMAL) { - long act = ethr_atomic_read(&rwmtx->mtxb.flgs); + ethr_sint32_t act = ethr_atomic32_read(&rwmtx->mtxb.flgs); if (act == ETHR_RWMTX_R_FLG__) rwmutex_try_complete_runlock(rwmtx, act, NULL, 0, 0, 0); } @@ -2179,7 +2350,7 @@ int ethr_rwmutex_tryrlock(ethr_rwmutex *rwmtx) { int res = 0; - long act; + ethr_sint32_t act; ETHR_ASSERT(!ethr_not_inited__); ETHR_ASSERT(rwmtx); @@ -2187,25 +2358,27 @@ ethr_rwmutex_tryrlock(ethr_rwmutex *rwmtx) ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + switch (rwmtx->type) { case ETHR_RWMUTEX_TYPE_NORMAL: { #ifdef ETHR_RLOCK_WITH_INC_DEC - act = ethr_atomic_read(&rwmtx->mtxb.flgs); + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); if (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)) res = EBUSY; else { - act = ethr_atomic_inc_read_acqb(&rwmtx->mtxb.flgs); + act = ethr_atomic32_inc_read_acqb(&rwmtx->mtxb.flgs); if (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)) { rwmutex_incdec_restore_failed_tryrlock(rwmtx); res = EBUSY; } } #else - long exp = 0; + ethr_sint32_t exp = 0; int tries = 0; while (1) { - act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, exp+1, exp); + act = ethr_atomic32_cmpxchg_acqb(&rwmtx->mtxb.flgs, exp+1, exp); if (act == exp) { res = 0; break; @@ -2225,49 +2398,30 @@ ethr_rwmutex_tryrlock(ethr_rwmutex *rwmtx) case ETHR_RWMUTEX_TYPE_FREQUENT_READ: case ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ: { ethr_ts_event *tse = ethr_get_ts_event(); - - rwmutex_freqread_rdrs_inc(rwmtx, tse); - - ETHR_MEMORY_BARRIER; - - act = ethr_atomic_read_acqb(&rwmtx->mtxb.flgs); - - if (act != ETHR_RWMTX_R_FLG__) { - while (1) { - long exp, new; - - if (act & ~(ETHR_RWMTX_R_FLG__|ETHR_RWMTX_R_WAIT_FLG__)) { - rwmutex_freqread_restore_failed_tryrlock(rwmtx, tse); - res = EBUSY; - break; - } - - if (act & ETHR_RWMTX_R_FLG__) - break; - - exp = act; - new = act | ETHR_RWMTX_R_FLG__; - act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, new, exp); - if (act == exp) - break; - } - } - + res = rwmutex_freqread_rlock(rwmtx, tse, 1); ethr_leave_ts_event(tse); break; } } +#ifdef ETHR_MTX_CHK_EXCL + if (res == 0) { + ETHR_MTX_CHK_EXCL_SET_NON_EXCL(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_IS_NOT_EXCL(&rwmtx->mtxb); + } +#endif + ETHR_MTX_HARD_DEBUG_LFS_TRYRLOCK(&rwmtx->mtxb, res); ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); return res; } void ethr_rwmutex_rlock(ethr_rwmutex *rwmtx) { - long act; + ethr_sint32_t act; ETHR_ASSERT(!ethr_not_inited__); ETHR_ASSERT(rwmtx); @@ -2275,20 +2429,21 @@ ethr_rwmutex_rlock(ethr_rwmutex *rwmtx) ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + switch (rwmtx->type) { case ETHR_RWMUTEX_TYPE_NORMAL: { #ifdef ETHR_RLOCK_WITH_INC_DEC - act = ethr_atomic_inc_read_acqb(&rwmtx->mtxb.flgs); + act = ethr_atomic32_inc_read_acqb(&rwmtx->mtxb.flgs); if (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)) rwmutex_normal_rlock_wait(rwmtx, act); #else - long exp = 0; + ethr_sint32_t exp = 0; while (1) { - act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, exp+1, exp); - if (act == exp) { + act = ethr_atomic32_cmpxchg_acqb(&rwmtx->mtxb.flgs, exp+1, exp); + if (act == exp) break; - } if (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_W_WAIT_FLG__)) { rwmutex_normal_rlock_wait(rwmtx, act); @@ -2303,38 +2458,15 @@ ethr_rwmutex_rlock(ethr_rwmutex *rwmtx) case ETHR_RWMUTEX_TYPE_FREQUENT_READ: case ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ: { ethr_ts_event *tse = ethr_get_ts_event(); - - rwmutex_freqread_rdrs_inc(rwmtx, tse); - - ETHR_MEMORY_BARRIER; - - act = ethr_atomic_read_acqb(&rwmtx->mtxb.flgs); - - if (act != ETHR_RWMTX_R_FLG__) { - while (1) { - long exp, new; - - if (act & ~(ETHR_RWMTX_R_FLG__|ETHR_RWMTX_R_WAIT_FLG__)) { - rwmutex_freqread_rlock_wait(rwmtx, tse, act); - break; - } - - if (act & ETHR_RWMTX_R_FLG__) - break; - - exp = act; - new = act | ETHR_RWMTX_R_FLG__; - act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, new, exp); - if (act == exp) - break; - } - } - + rwmutex_freqread_rlock(rwmtx, tse, 0); ethr_leave_ts_event(tse); break; } } + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + ETHR_MTX_CHK_EXCL_SET_NON_EXCL(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_IS_NOT_EXCL(&rwmtx->mtxb); ETHR_MTX_HARD_DEBUG_LFS_RLOCK(&rwmtx->mtxb); ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); } @@ -2342,8 +2474,10 @@ ethr_rwmutex_rlock(ethr_rwmutex *rwmtx) void ethr_rwmutex_runlock(ethr_rwmutex *rwmtx) { - long act; + ethr_sint32_t act; + ETHR_MTX_CHK_EXCL_IS_NOT_EXCL(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_UNSET_NON_EXCL(&rwmtx->mtxb); ETHR_ASSERT(!ethr_not_inited__); ETHR_ASSERT(rwmtx); ETHR_ASSERT(rwmtx->initialized == ETHR_RWMUTEX_INITIALIZED); @@ -2351,13 +2485,15 @@ ethr_rwmutex_runlock(ethr_rwmutex *rwmtx) ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); ETHR_MTX_HARD_DEBUG_LFS_RUNLOCK(&rwmtx->mtxb); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + switch (rwmtx->type) { case ETHR_RWMUTEX_TYPE_NORMAL: - act = ethr_atomic_dec_read_relb(&rwmtx->mtxb.flgs); + act = ethr_atomic32_dec_read_relb(&rwmtx->mtxb.flgs); if ((act & ETHR_RWMTX_WAIT_FLGS__) && (act & ~ETHR_RWMTX_WAIT_FLGS__) == 0) { ETHR_ASSERT((act & ETHR_RWMTX_W_FLG__) == 0); - rwmutex_unlock_wake(rwmtx, 0, act); + rwmutex_unlock_wake(rwmtx, 0, act, 0); } break; @@ -2369,21 +2505,12 @@ ethr_rwmutex_runlock(ethr_rwmutex *rwmtx) ETHR_ASSERT(act >= 0); - ETHR_WRITE_MEMORY_BARRIER; + ETHR_MEMORY_BARRIER; if (act == 0) { - -#ifndef ETHR_WRITE_MEMORY_BARRIER_IS_FULL - ETHR_READ_MEMORY_BARRIER; -#endif - act = ethr_atomic_read(&rwmtx->mtxb.flgs); - - if ((act & ETHR_RWMTX_W_FLG__) == 0 - && (act & (ETHR_RWMTX_WAIT_FLGS__ - | ETHR_RWMTX_R_PEND_UNLCK_MASK__))) { - rwmutex_try_complete_runlock(rwmtx, act, tse, 1, 0, 0); - } - + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); + if (act != ETHR_RWMTX_R_FLG__) + rwmutex_freqread_rdrs_dec_chk_wakeup(rwmtx, tse, act); } ethr_leave_ts_event(tse); @@ -2391,6 +2518,7 @@ ethr_rwmutex_runlock(ethr_rwmutex *rwmtx) } } + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); } @@ -2398,7 +2526,7 @@ int ethr_rwmutex_tryrwlock(ethr_rwmutex *rwmtx) { int res = 0; - long act; + ethr_sint32_t act; ETHR_ASSERT(!ethr_not_inited__); ETHR_ASSERT(rwmtx); @@ -2406,10 +2534,12 @@ ethr_rwmutex_tryrwlock(ethr_rwmutex *rwmtx) ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + switch (rwmtx->type) { case ETHR_RWMUTEX_TYPE_NORMAL: - act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, - ETHR_RWMTX_W_FLG__, 0); + act = ethr_atomic32_cmpxchg_acqb(&rwmtx->mtxb.flgs, + ETHR_RWMTX_W_FLG__, 0); if (act != 0) res = EBUSY; break; @@ -2418,29 +2548,36 @@ ethr_rwmutex_tryrwlock(ethr_rwmutex *rwmtx) case ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ: res = 0; - act = ethr_atomic_read(&rwmtx->mtxb.flgs); + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); do { - if (act & (ETHR_RWMTX_W_FLG__|ETHR_RWMTX_WAIT_FLGS__)) { - res = EBUSY; - break; - } - - if (act & ETHR_RWMTX_R_MASK__) { + if (act == 0) + act = ethr_atomic32_cmpxchg_acqb(&rwmtx->mtxb.flgs, + ETHR_RWMTX_W_FLG__, 0); + else if (act == ETHR_RWMTX_R_FLG__) { res = rwmutex_try_complete_runlock(rwmtx, act, NULL, 0, 1, 1); break; } - - act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, - ETHR_RWMTX_W_FLG__, 0); + else { + res = EBUSY; + break; + } } while (act != 0); break; } +#ifdef ETHR_MTX_CHK_EXCL + if (res == 0) { + ETHR_MTX_CHK_EXCL_SET_EXCL(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_IS_NOT_NON_EXCL(&rwmtx->mtxb); + } +#endif + + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); ETHR_MTX_HARD_DEBUG_LFS_TRYRWLOCK(&rwmtx->mtxb, res); ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); @@ -2450,17 +2587,19 @@ ethr_rwmutex_tryrwlock(ethr_rwmutex *rwmtx) void ethr_rwmutex_rwlock(ethr_rwmutex *rwmtx) { - long act; + ethr_sint32_t act; ETHR_ASSERT(!ethr_not_inited__); ETHR_ASSERT(rwmtx); ETHR_ASSERT(rwmtx->initialized == ETHR_RWMUTEX_INITIALIZED); ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + switch (rwmtx->type) { case ETHR_RWMUTEX_TYPE_NORMAL: - act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, - ETHR_RWMTX_W_FLG__, 0); + act = ethr_atomic32_cmpxchg_acqb(&rwmtx->mtxb.flgs, + ETHR_RWMTX_W_FLG__, 0); if (act != 0) rwmutex_normal_rwlock_wait(rwmtx, act); break; @@ -2468,7 +2607,7 @@ ethr_rwmutex_rwlock(ethr_rwmutex *rwmtx) case ETHR_RWMUTEX_TYPE_FREQUENT_READ: case ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ: - act = ethr_atomic_read(&rwmtx->mtxb.flgs); + act = ethr_atomic32_read(&rwmtx->mtxb.flgs); do { @@ -2477,23 +2616,26 @@ ethr_rwmutex_rwlock(ethr_rwmutex *rwmtx) break; } - act = ethr_atomic_cmpxchg_acqb(&rwmtx->mtxb.flgs, - ETHR_RWMTX_W_FLG__, 0); + act = ethr_atomic32_cmpxchg_acqb(&rwmtx->mtxb.flgs, + ETHR_RWMTX_W_FLG__, 0); } while (act != 0); break; } + ETHR_MTX_CHK_EXCL_SET_EXCL(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_IS_NOT_NON_EXCL(&rwmtx->mtxb); ETHR_MTX_HARD_DEBUG_LFS_RWLOCK(&rwmtx->mtxb); ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); } void ethr_rwmutex_rwunlock(ethr_rwmutex *rwmtx) { - long act; + ethr_sint32_t act; ETHR_ASSERT(!ethr_not_inited__); ETHR_ASSERT(rwmtx); ETHR_ASSERT(rwmtx->initialized == ETHR_RWMUTEX_INITIALIZED); @@ -2501,24 +2643,30 @@ ethr_rwmutex_rwunlock(ethr_rwmutex *rwmtx) ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); ETHR_MTX_HARD_DEBUG_LFS_RWUNLOCK(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_IS_NOT_NON_EXCL(&rwmtx->mtxb); + ETHR_MTX_CHK_EXCL_UNSET_EXCL(&rwmtx->mtxb); + + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); + switch (rwmtx->type) { case ETHR_RWMUTEX_TYPE_NORMAL: - act = ethr_atomic_cmpxchg_relb(&rwmtx->mtxb.flgs, - 0, ETHR_RWMTX_W_FLG__); + act = ethr_atomic32_cmpxchg_relb(&rwmtx->mtxb.flgs, + 0, ETHR_RWMTX_W_FLG__); if (act != ETHR_RWMTX_W_FLG__) - rwmutex_unlock_wake(rwmtx, 1, act); + rwmutex_unlock_wake(rwmtx, 1, act, 0); break; case ETHR_RWMUTEX_TYPE_FREQUENT_READ: case ETHR_RWMUTEX_TYPE_EXTREMELY_FREQUENT_READ: - act = ethr_atomic_cmpxchg_relb(&rwmtx->mtxb.flgs, 0, - ETHR_RWMTX_W_FLG__); + act = ethr_atomic32_cmpxchg_relb(&rwmtx->mtxb.flgs, 0, + ETHR_RWMTX_W_FLG__); if (act != ETHR_RWMTX_W_FLG__) - rwmutex_unlock_wake(rwmtx, 1, act); + rwmutex_unlock_wake(rwmtx, 1, act, 0); break; } ETHR_MTX_HARD_DEBUG_FENCE_CHK(rwmtx); + ETHR_MTX_DBG_CHK_UNUSED_FLG_BITS(rwmtx); } #else @@ -2636,7 +2784,7 @@ static void hard_debug_chk_q__(struct ethr_mutex_base_ *mtxb, int is_rwmtx) { int res; - long flgs = ethr_atomic_read(&mtxb->flgs); + ethr_sint32_t flgs = ethr_atomic32_read(&mtxb->flgs); ETHR_MTX_HARD_ASSERT(res == 0); @@ -2659,12 +2807,12 @@ hard_debug_chk_q__(struct ethr_mutex_base_ *mtxb, int is_rwmtx) tse = mtxb->q; do { - long type; + ethr_sint32_t type; ETHR_MTX_HARD_ASSERT(tse->next->prev == tse); ETHR_MTX_HARD_ASSERT(tse->prev->next == tse); - type = ethr_atomic_read(&tse->uaflgs); + type = ethr_atomic32_read(&tse->uaflgs); ETHR_MTX_HARD_ASSERT(type == tse->uflgs); switch (type) { case ETHR_RWMTX_W_WAIT_FLG__: diff --git a/erts/lib_src/pthread/ethr_event.c b/erts/lib_src/pthread/ethr_event.c index 6731c0eb46..9434d60d0a 100644 --- a/erts/lib_src/pthread/ethr_event.c +++ b/erts/lib_src/pthread/ethr_event.c @@ -24,6 +24,10 @@ #define ETHR_INLINE_FUNC_NAME_(X) X ## __ #define ETHR_EVENT_IMPL__ +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif + #include "ethread.h" #if defined(ETHR_LINUX_FUTEX_IMPL__) @@ -37,7 +41,7 @@ int ethr_event_init(ethr_event *e) { - ethr_atomic_init(&e->futex, ETHR_EVENT_OFF__); + ethr_atomic32_init(&e->futex, ETHR_EVENT_OFF__); return 0; } @@ -52,7 +56,7 @@ wait__(ethr_event *e, int spincount) { unsigned sc = spincount; int res; - long val; + ethr_sint32_t val; int until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; if (spincount < 0) @@ -60,7 +64,7 @@ wait__(ethr_event *e, int spincount) while (1) { while (1) { - val = ethr_atomic_read(&e->futex); + val = ethr_atomic32_read(&e->futex); if (val == ETHR_EVENT_ON__) return 0; if (sc == 0) @@ -76,16 +80,18 @@ wait__(ethr_event *e, int spincount) } if (val != ETHR_EVENT_OFF_WAITER__) { - val = ethr_atomic_cmpxchg(&e->futex, - ETHR_EVENT_OFF_WAITER__, - ETHR_EVENT_OFF__); + val = ethr_atomic32_cmpxchg(&e->futex, + ETHR_EVENT_OFF_WAITER__, + ETHR_EVENT_OFF__); if (val == ETHR_EVENT_ON__) return 0; ETHR_ASSERT(val == ETHR_EVENT_OFF__); } - res = ETHR_FUTEX__(&e->futex, ETHR_FUTEX_WAIT__, ETHR_EVENT_OFF_WAITER__); + res = ETHR_FUTEX__(&e->futex, + ETHR_FUTEX_WAIT__, + ETHR_EVENT_OFF_WAITER__); if (res == EINTR) break; if (res != 0 && res != EWOULDBLOCK) @@ -102,7 +108,7 @@ int ethr_event_init(ethr_event *e) { int res; - ethr_atomic_init(&e->state, ETHR_EVENT_OFF__); + ethr_atomic32_init(&e->state, ETHR_EVENT_OFF__); res = pthread_mutex_init(&e->mtx, NULL); if (res != 0) return res; @@ -131,7 +137,7 @@ static ETHR_INLINE int wait__(ethr_event *e, int spincount) { int sc = spincount; - long val; + ethr_sint32_t val; int res, ulres; int until_yield = ETHR_YIELD_AFTER_BUSY_LOOPS; @@ -139,7 +145,7 @@ wait__(ethr_event *e, int spincount) ETHR_FATAL_ERROR__(EINVAL); while (1) { - val = ethr_atomic_read(&e->state); + val = ethr_atomic32_read(&e->state); if (val == ETHR_EVENT_ON__) return 0; if (sc == 0) @@ -155,9 +161,9 @@ wait__(ethr_event *e, int spincount) } if (val != ETHR_EVENT_OFF_WAITER__) { - val = ethr_atomic_cmpxchg(&e->state, - ETHR_EVENT_OFF_WAITER__, - ETHR_EVENT_OFF__); + val = ethr_atomic32_cmpxchg(&e->state, + ETHR_EVENT_OFF_WAITER__, + ETHR_EVENT_OFF__); if (val == ETHR_EVENT_ON__) return 0; ETHR_ASSERT(val == ETHR_EVENT_OFF__); @@ -172,7 +178,7 @@ wait__(ethr_event *e, int spincount) while (1) { - val = ethr_atomic_read(&e->state); + val = ethr_atomic32_read(&e->state); if (val == ETHR_EVENT_ON__) break; diff --git a/erts/lib_src/pthread/ethread.c b/erts/lib_src/pthread/ethread.c index ea1d9d43f0..f047104103 100644 --- a/erts/lib_src/pthread/ethread.c +++ b/erts/lib_src/pthread/ethread.c @@ -72,7 +72,7 @@ static void thr_exit_cleanup(void) /* Argument passed to thr_wrapper() */ typedef struct { - ethr_atomic_t result; + ethr_atomic32_t result; ethr_ts_event *tse; void *(*thr_func)(void *); void *arg; @@ -81,14 +81,14 @@ typedef struct { static void *thr_wrapper(void *vtwd) { - long result; + ethr_sint32_t result; void *res; ethr_thr_wrap_data__ *twd = (ethr_thr_wrap_data__ *) vtwd; void *(*thr_func)(void *) = twd->thr_func; void *arg = twd->arg; ethr_ts_event *tsep = NULL; - result = (long) ethr_make_ts_event__(&tsep); + result = (ethr_sint32_t) ethr_make_ts_event__(&tsep); if (result == 0) { tsep->iflgs |= ETHR_TS_EV_ETHREAD; @@ -99,7 +99,7 @@ static void *thr_wrapper(void *vtwd) tsep = twd->tse; /* We aren't allowed to follow twd after result has been set! */ - ethr_atomic_set(&twd->result, result); + ethr_atomic32_set(&twd->result, result); ethr_event_set(&tsep->event); @@ -191,7 +191,7 @@ ethr_thr_create(ethr_tid *tid, void * (*func)(void *), void *arg, } #endif - ethr_atomic_init(&twd.result, -1); + ethr_atomic32_init(&twd.result, (ethr_sint32_t) -1); twd.tse = ethr_get_ts_event(); twd.thr_func = func; twd.arg = arg; @@ -252,10 +252,10 @@ ethr_thr_create(ethr_tid *tid, void * (*func)(void *), void *arg, /* Wait for child to initialize... */ while (1) { - long result; + ethr_sint32_t result; ethr_event_reset(&twd.tse->event); - result = ethr_atomic_read(&twd.result); + result = ethr_atomic32_read(&twd.result); if (result == 0) break; @@ -349,32 +349,6 @@ ethr_leave_ts_event(ethr_ts_event *tsep) } /* - * Current time - */ - -int -ethr_time_now(ethr_timeval *time) -{ - int res; - struct timeval tv; -#if ETHR_XCHK - if (ethr_not_inited__) { - ETHR_ASSERT(0); - return EACCES; - } - if (!time) { - ETHR_ASSERT(0); - return EINVAL; - } -#endif - - res = gettimeofday(&tv, NULL); - time->tv_sec = (long) tv.tv_sec; - time->tv_nsec = ((long) tv.tv_usec)*1000; - return res; -} - -/* * Thread specific data */ diff --git a/erts/lib_src/win/ethr_event.c b/erts/lib_src/win/ethr_event.c index ddb4780ff1..68f093f49c 100644 --- a/erts/lib_src/win/ethr_event.c +++ b/erts/lib_src/win/ethr_event.c @@ -28,6 +28,9 @@ /* --- Windows implementation of thread events ------------------------------ */ +#pragma intrinsic(_InterlockedExchangeAdd) +#pragma intrinsic(_InterlockedCompareExchange) + int ethr_event_init(ethr_event *e) { @@ -72,10 +75,10 @@ wait(ethr_event *e, int spincount) while (1) { long on; while (1) { -#if ETHR_IMMED_ATOMIC_SET_GET_SAFE__ +#if ETHR_READ_AND_SET_WITHOUT_INTERLOCKED_OP__ state = e->state; #else - state = InterlockedExchangeAdd(&e->state, (LONG) 0); + state = _InterlockedExchangeAdd(&e->state, (LONG) 0); #endif if (state == ETHR_EVENT_ON__) return 0; diff --git a/erts/lib_src/win/ethread.c b/erts/lib_src/win/ethread.c index 69523edf94..789a360b11 100644 --- a/erts/lib_src/win/ethread.c +++ b/erts/lib_src/win/ethread.c @@ -49,7 +49,7 @@ /* Argument passed to thr_wrapper() */ typedef struct { ethr_tid *tid; - ethr_atomic_t result; + ethr_atomic32_t result; ethr_ts_event *tse; void *(*thr_func)(void *); void *arg; @@ -93,20 +93,20 @@ static void thr_exit_cleanup(ethr_tid *tid, void *res) static unsigned __stdcall thr_wrapper(LPVOID vtwd) { ethr_tid my_tid; - long result; + ethr_sint32_t result; void *res; ethr_thr_wrap_data__ *twd = (ethr_thr_wrap_data__ *) vtwd; void *(*thr_func)(void *) = twd->thr_func; void *arg = twd->arg; ethr_ts_event *tsep = NULL; - result = (long) ethr_make_ts_event__(&tsep); + result = (ethr_sint32_t) ethr_make_ts_event__(&tsep); if (result == 0) { tsep->iflgs |= ETHR_TS_EV_ETHREAD; my_tid = *twd->tid; if (!TlsSetValue(own_tid_key, (LPVOID) &my_tid)) { - result = (long) ethr_win_get_errno__(); + result = (ethr_sint32_t) ethr_win_get_errno__(); ethr_free_ts_event__(tsep); } else { @@ -118,7 +118,7 @@ static unsigned __stdcall thr_wrapper(LPVOID vtwd) tsep = twd->tse; /* We aren't allowed to follow twd after result has been set! */ - ethr_atomic_set(&twd->result, result); + ethr_atomic32_set(&twd->result, result); ethr_event_set(&tsep->event); @@ -128,28 +128,6 @@ static unsigned __stdcall thr_wrapper(LPVOID vtwd) return 0; } -#ifdef __GNUC__ -#define LL_LITERAL(X) X##LL -#else -#define LL_LITERAL(X) X##i64 -#endif - -#define EPOCH_JULIAN_DIFF LL_LITERAL(11644473600) - -static ETHR_INLINE void -get_curr_time(long *sec, long *nsec) -{ - SYSTEMTIME t; - FILETIME ft; - LONGLONG lft; - - GetSystemTime(&t); - SystemTimeToFileTime(&t, &ft); - memcpy(&lft, &ft, sizeof(lft)); - *nsec = ((long) (lft % LL_LITERAL(10000000)))*100; - *sec = (long) ((lft / LL_LITERAL(10000000)) - EPOCH_JULIAN_DIFF); -} - /* internal exports */ int @@ -320,7 +298,7 @@ ethr_thr_create(ethr_tid *tid, void * (*func)(void *), void *arg, ETHR_PAGE_ALIGN(ETHR_KW2B(suggested_stack_size)); } - ethr_atomic_init(&twd.result, -1); + ethr_atomic32_init(&twd.result, -1); twd.tid = tid; twd.thr_func = func; @@ -352,11 +330,11 @@ ethr_thr_create(ethr_tid *tid, void * (*func)(void *), void *arg, /* Wait for child to initialize... */ while (1) { - long result; + ethr_sint32_t result; int err; ethr_event_reset(&twd.tse->event); - result = ethr_atomic_read(&twd.result); + result = ethr_atomic32_read(&twd.result); if (result == 0) break; @@ -517,23 +495,6 @@ ethr_equal_tids(ethr_tid tid1, ethr_tid tid2) return tid1.id == tid2.id && tid1.id != ETHR_INVALID_TID_ID; } -int -ethr_time_now(ethr_timeval *time) -{ -#if ETHR_XCHK - if (ethr_not_inited__) { - ETHR_ASSERT(0); - return EACCES; - } - if (!time) { - ETHR_ASSERT(0); - return EINVAL; - } -#endif - get_curr_time(&time->tv_sec, &time->tv_nsec); - return 0; -} - /* * Thread specific data */ diff --git a/erts/preloaded/ebin/erl_prim_loader.beam b/erts/preloaded/ebin/erl_prim_loader.beam Binary files differindex 222809e662..87ff5119fd 100644 --- a/erts/preloaded/ebin/erl_prim_loader.beam +++ b/erts/preloaded/ebin/erl_prim_loader.beam diff --git a/erts/preloaded/ebin/erlang.beam b/erts/preloaded/ebin/erlang.beam Binary files differindex 65c7369b76..6b0d96ff8e 100644 --- a/erts/preloaded/ebin/erlang.beam +++ b/erts/preloaded/ebin/erlang.beam diff --git a/erts/preloaded/ebin/init.beam b/erts/preloaded/ebin/init.beam Binary files differindex c8d3b78b35..8a7a9a1314 100644 --- a/erts/preloaded/ebin/init.beam +++ b/erts/preloaded/ebin/init.beam diff --git a/erts/preloaded/ebin/otp_ring0.beam b/erts/preloaded/ebin/otp_ring0.beam Binary files differindex abf17bcb0e..5d544ff4aa 100644 --- a/erts/preloaded/ebin/otp_ring0.beam +++ b/erts/preloaded/ebin/otp_ring0.beam diff --git a/erts/preloaded/ebin/prim_file.beam b/erts/preloaded/ebin/prim_file.beam Binary files differindex a3f300268f..3ed02ecd44 100644 --- a/erts/preloaded/ebin/prim_file.beam +++ b/erts/preloaded/ebin/prim_file.beam diff --git a/erts/preloaded/ebin/prim_inet.beam b/erts/preloaded/ebin/prim_inet.beam Binary files differindex 3f53f35273..79a8d22366 100644 --- a/erts/preloaded/ebin/prim_inet.beam +++ b/erts/preloaded/ebin/prim_inet.beam diff --git a/erts/preloaded/ebin/prim_zip.beam b/erts/preloaded/ebin/prim_zip.beam Binary files differindex 0fe38a1fb2..3cc8c6b8be 100644 --- a/erts/preloaded/ebin/prim_zip.beam +++ b/erts/preloaded/ebin/prim_zip.beam diff --git a/erts/preloaded/ebin/zlib.beam b/erts/preloaded/ebin/zlib.beam Binary files differindex 7108bf44d0..3f9e867542 100644 --- a/erts/preloaded/ebin/zlib.beam +++ b/erts/preloaded/ebin/zlib.beam diff --git a/erts/preloaded/src/prim_file.erl b/erts/preloaded/src/prim_file.erl index 7f24889bb2..10be852e92 100644 --- a/erts/preloaded/src/prim_file.erl +++ b/erts/preloaded/src/prim_file.erl @@ -109,6 +109,8 @@ -define(FILE_RESP_LDATA, 6). -define(FILE_RESP_N2DATA, 7). -define(FILE_RESP_EOF, 8). +-define(FILE_RESP_FNAME, 9). +-define(FILE_RESP_ALL_DATA, 10). %% Open modes for the driver's open function. -define(EFILE_MODE_READ, 1). @@ -153,7 +155,7 @@ %% Opens a file using the driver port Port. Returns {error, Reason} %% | {ok, FileDescriptor} open(Port, File, ModeList) when is_port(Port), - is_list(File), + (is_list(File) orelse is_binary(File)), is_list(ModeList) -> case open_mode(ModeList) of {Mode, _Portopts, _Setopts} -> @@ -165,10 +167,11 @@ open(_,_,_) -> {error, badarg}. %% Opens a file. Returns {error, Reason} | {ok, FileDescriptor}. -open(File, ModeList) when is_list(File), is_list(ModeList) -> +open(File, ModeList) when (is_list(File) orelse is_binary(File)), + is_list(ModeList) -> case open_mode(ModeList) of {Mode, Portopts, Setopts} -> - open_int({?FD_DRV, Portopts}, File, Mode, Setopts); + open_int({?FD_DRV, Portopts},File, Mode, Setopts); Reason -> {error, Reason} end; @@ -196,7 +199,7 @@ open_int({Driver, Portopts}, File, Mode, Setopts) -> end; open_int(Port, File, Mode, Setopts) -> M = Mode band ?EFILE_MODE_MASK, - case drv_command(Port, [<<?FILE_OPEN, M:32>>, File, 0]) of + case drv_command(Port, [<<?FILE_OPEN, M:32>>, pathname(File)]) of {ok, Number} -> open_int_setopts(Port, Number, Setopts); Error -> @@ -489,7 +492,7 @@ ipread_s32bu_p32bu(#file_descriptor{module = ?MODULE, data = {_, _}}, %% Returns {ok, Contents} | {error, Reason} -read_file(File) -> +read_file(File) when (is_list(File) orelse is_binary(File)) -> case drv_open(?FD_DRV, [binary]) of {ok, Port} -> Result = read_file(Port, File), @@ -497,11 +500,14 @@ read_file(File) -> Result; {error, _} = Error -> Error - end. + end; +read_file(_) -> + {error, badarg}. %% Takes a Port opened with open/1. -read_file(Port, File) when is_port(Port) -> - Cmd = [?FILE_READ_FILE | File], +read_file(Port, File) when is_port(Port), + (is_list(File) orelse is_binary(File))-> + Cmd = [?FILE_READ_FILE | pathname(File)], case drv_command(Port, Cmd) of {error, enomem} -> %% It could possibly help to do a @@ -512,12 +518,14 @@ read_file(Port, File) when is_port(Port) -> drv_command(Port, Cmd); Result -> Result - end. + end; +read_file(_,_) -> + {error, badarg}. %% Returns {error, Reason} | ok. -write_file(File, Bin) -> +write_file(File, Bin) when (is_list(File) orelse is_binary(File)) -> case open(File, [binary, write]) of {ok, Handle} -> Result = write(Handle, Bin), @@ -525,8 +533,10 @@ write_file(File, Bin) -> Result; Error -> Error - end. - + end; +write_file(_, _) -> + {error, badarg}. + %%%----------------------------------------------------------------- @@ -539,7 +549,7 @@ write_file(File, Bin) -> %% Returns {ok, Port}, the Port should be used as first argument in all %% the following functions. Returns {error, Reason} upon failure. start() -> - try erlang:open_port({spawn, atom_to_list(?DRV)}, []) of + try erlang:open_port({spawn, atom_to_list(?DRV)}, [binary]) of Port -> {ok, Port} catch @@ -596,7 +606,7 @@ get_cwd(_, _) -> {error, badarg}. get_cwd_int(Drive) -> - get_cwd_int({?DRV, []}, Drive). + get_cwd_int({?DRV, [binary]}, Drive). get_cwd_int(Port, Drive) -> drv_command(Port, <<?FILE_PWD, Drive>>). @@ -606,7 +616,7 @@ get_cwd_int(Port, Drive) -> %% set_cwd/{1,2} set_cwd(Dir) -> - set_cwd_int({?DRV, []}, Dir). + set_cwd_int({?DRV, [binary]}, Dir). set_cwd(Port, Dir) when is_port(Port) -> set_cwd_int(Port, Dir). @@ -632,89 +642,88 @@ set_cwd_int(Port, Dir0) -> end), %% Dir is now either a string or an EXIT tuple. %% An EXIT tuple will fail in the following catch. - drv_command(Port, [?FILE_CHDIR, Dir, 0]). + drv_command(Port, [?FILE_CHDIR, pathname(Dir)]). %% delete/{1,2} delete(File) -> - delete_int({?DRV, []}, File). + delete_int({?DRV, [binary]}, File). delete(Port, File) when is_port(Port) -> delete_int(Port, File). delete_int(Port, File) -> - drv_command(Port, [?FILE_DELETE, File, 0]). + drv_command(Port, [?FILE_DELETE, pathname(File)]). %% rename/{2,3} rename(From, To) -> - rename_int({?DRV, []}, From, To). + rename_int({?DRV, [binary]}, From, To). rename(Port, From, To) when is_port(Port) -> rename_int(Port, From, To). rename_int(Port, From, To) -> - drv_command(Port, [?FILE_RENAME, From, 0, To, 0]). + drv_command(Port, [?FILE_RENAME, pathname(From), pathname(To)]). %% make_dir/{1,2} make_dir(Dir) -> - make_dir_int({?DRV, []}, Dir). + make_dir_int({?DRV, [binary]}, Dir). make_dir(Port, Dir) when is_port(Port) -> make_dir_int(Port, Dir). make_dir_int(Port, Dir) -> - drv_command(Port, [?FILE_MKDIR, Dir, 0]). + drv_command(Port, [?FILE_MKDIR, pathname(Dir)]). %% del_dir/{1,2} del_dir(Dir) -> - del_dir_int({?DRV, []}, Dir). + del_dir_int({?DRV, [binary]}, Dir). del_dir(Port, Dir) when is_port(Port) -> del_dir_int(Port, Dir). del_dir_int(Port, Dir) -> - drv_command(Port, [?FILE_RMDIR, Dir, 0]). + drv_command(Port, [?FILE_RMDIR, pathname(Dir)]). %% read_file_info/{1,2} read_file_info(File) -> - read_file_info_int({?DRV, []}, File). + read_file_info_int({?DRV, [binary]}, File). read_file_info(Port, File) when is_port(Port) -> read_file_info_int(Port, File). read_file_info_int(Port, File) -> - drv_command(Port, [?FILE_FSTAT, File, 0]). + drv_command(Port, [?FILE_FSTAT, pathname(File)]). %% altname/{1,2} altname(File) -> - altname_int({?DRV, []}, File). + altname_int({?DRV, [binary]}, File). altname(Port, File) when is_port(Port) -> altname_int(Port, File). altname_int(Port, File) -> - drv_command(Port, [?FILE_ALTNAME, File, 0]). - + drv_command(Port, [?FILE_ALTNAME, pathname(File)]). %% write_file_info/{2,3} write_file_info(File, Info) -> - write_file_info_int({?DRV, []}, File, Info). + write_file_info_int({?DRV, [binary]}, File, Info). write_file_info(Port, File, Info) when is_port(Port) -> write_file_info_int(Port, File, Info). @@ -740,72 +749,72 @@ write_file_info_int(Port, date_to_bytes(Atime), date_to_bytes(Mtime), date_to_bytes(Ctime), - File, 0]). + pathname(File)]). %% make_link/{2,3} make_link(Old, New) -> - make_link_int({?DRV, []}, Old, New). + make_link_int({?DRV, [binary]}, Old, New). make_link(Port, Old, New) when is_port(Port) -> make_link_int(Port, Old, New). make_link_int(Port, Old, New) -> - drv_command(Port, [?FILE_LINK, Old, 0, New, 0]). + drv_command(Port, [?FILE_LINK, pathname(Old), pathname(New)]). %% make_symlink/{2,3} make_symlink(Old, New) -> - make_symlink_int({?DRV, []}, Old, New). + make_symlink_int({?DRV, [binary]}, Old, New). make_symlink(Port, Old, New) when is_port(Port) -> make_symlink_int(Port, Old, New). make_symlink_int(Port, Old, New) -> - drv_command(Port, [?FILE_SYMLINK, Old, 0, New, 0]). + drv_command(Port, [?FILE_SYMLINK, pathname(Old), pathname(New)]). %% read_link/{2,3} read_link(Link) -> - read_link_int({?DRV, []}, Link). + read_link_int({?DRV, [binary]}, Link). read_link(Port, Link) when is_port(Port) -> read_link_int(Port, Link). read_link_int(Port, Link) -> - drv_command(Port, [?FILE_READLINK, Link, 0]). + drv_command(Port, [?FILE_READLINK, pathname(Link)]). %% read_link_info/{2,3} read_link_info(Link) -> - read_link_info_int({?DRV, []}, Link). + read_link_info_int({?DRV, [binary]}, Link). read_link_info(Port, Link) when is_port(Port) -> read_link_info_int(Port, Link). read_link_info_int(Port, Link) -> - drv_command(Port, [?FILE_LSTAT, Link, 0]). + drv_command(Port, [?FILE_LSTAT, pathname(Link)]). %% list_dir/{1,2} list_dir(Dir) -> - list_dir_int({?DRV, []}, Dir). + list_dir_int({?DRV, [binary]}, Dir). list_dir(Port, Dir) when is_port(Port) -> list_dir_int(Port, Dir). list_dir_int(Port, Dir) -> - drv_command(Port, [?FILE_READDIR, Dir, 0], []). + drv_command(Port, [?FILE_READDIR, pathname(Dir)], []). @@ -1026,8 +1035,6 @@ lseek_position(_) -> translate_response(?FILE_RESP_OK, []) -> ok; -translate_response(?FILE_RESP_OK, Data) -> - {ok, Data}; translate_response(?FILE_RESP_ERROR, List) when is_list(List) -> {error, list_to_atom(List)}; translate_response(?FILE_RESP_NUMBER, List) -> @@ -1074,6 +1081,16 @@ translate_response(?FILE_RESP_N2DATA = X, L0) when is_list(L0) -> end; translate_response(?FILE_RESP_EOF, []) -> eof; +translate_response(?FILE_RESP_FNAME, []) -> + ok; +translate_response(?FILE_RESP_FNAME, Data) when is_binary(Data) -> + {ok, prim_file:internal_native2name(Data)}; +translate_response(?FILE_RESP_FNAME, Data) -> + {ok, Data}; + +translate_response(?FILE_RESP_ALL_DATA, Data) -> + {ok, Data}; + translate_response(X, Data) -> {error, {bad_response_from_port, [X | Data]}}. @@ -1209,3 +1226,9 @@ lists_split([Hd | Tl], N, Rev) -> reverse(X) -> lists:reverse(X, []). reverse(L, T) -> lists:reverse(L, T). + +% Will add zero termination too +% The 'EXIT' tuple from a bad argument will eventually generate an error +% in list_to_binary, which is caught and generates the {error,badarg} return +pathname(File) -> + (catch prim_file:internal_name2native(File)). diff --git a/erts/test/ethread_SUITE.erl b/erts/test/ethread_SUITE.erl index 0cc315e9be..69e5af802f 100644 --- a/erts/test/ethread_SUITE.erl +++ b/erts/test/ethread_SUITE.erl @@ -37,7 +37,6 @@ equal_tids/1, mutex/1, try_lock_mutex/1, - time_now/1, cond_wait/1, broadcast/1, detached_thread/1, @@ -55,7 +54,6 @@ tests() -> equal_tids, mutex, try_lock_mutex, - time_now, cond_wait, broadcast, detached_thread, @@ -104,17 +102,6 @@ try_lock_mutex(suite) -> try_lock_mutex(Config) -> run_case(Config, "try_lock_mutex", ""). -time_now(doc) -> - ["Tests ethr_time_now by comparing time values with Erlang."]; -time_now(suite) -> - []; -time_now(Config) -> - run_case(Config, "time_now", "", fun (P) -> - spawn_link(fun () -> - watchdog(P) - end) - end). - wd_dispatch(P) -> receive bye -> @@ -171,7 +158,15 @@ max_threads(doc) -> max_threads(suite) -> []; max_threads(Config) -> - run_case(Config, "max_threads", ""). + case {os:type(), os:version()} of + {{unix,darwin}, {9, _, _}} -> + %% For some reason pthread_create() crashes when more + %% threads cannot be created, instead of returning an + %% error code on our MacOS X Leopard machine... + {skipped, "MacOS X Leopard cannot cope with this test..."}; + _ -> + run_case(Config, "max_threads", "") + end. tsd(doc) -> ["Tests thread specific data."]; diff --git a/erts/test/ethread_SUITE_data/ethread_tests.c b/erts/test/ethread_SUITE_data/ethread_tests.c index 7fc71d8047..0b59ff5aa6 100644 --- a/erts/test/ethread_SUITE_data/ethread_tests.c +++ b/erts/test/ethread_SUITE_data/ethread_tests.c @@ -514,69 +514,6 @@ try_lock_mutex_test(void) } /* - * The time now test. - * - * Tests ethr_time_now by comparing time values with Erlang. - */ -#define TNT_MAX_TIME_DIFF 200000 -#define TNT_MAX_TIME_VALUES 52 - -static void -time_now_test(void) -{ - int scanf_res, time_now_res, i, no_values, max_abs_diff; - static ethr_timeval tv[TNT_MAX_TIME_VALUES]; - static int ms[TNT_MAX_TIME_VALUES]; - - i = 0; - do { - ASSERT(i < TNT_MAX_TIME_VALUES); - scanf_res = scanf("%d", &ms[i]); - time_now_res = ethr_time_now(&tv[i]); - ASSERT(scanf_res == 1); - ASSERT(time_now_res == 0); -#if 0 - print_line("Got %d; %ld:%ld", ms[i], tv[i].tv_sec, tv[i].tv_nsec); -#endif - i++; - } while (ms[i-1] >= 0); - - no_values = i-1; - - ASSERT(ms[0] == 0); - - print_line("TNT_MAX_TIME_DIFF = %d (us)", TNT_MAX_TIME_DIFF); - - max_abs_diff = 0; - - for (i = 1; i < no_values; i++) { - long diff; - long tn_us; - long e_us; - - tn_us = (tv[i].tv_sec - tv[0].tv_sec) * 1000000; - tn_us += (tv[i].tv_nsec - tv[0].tv_nsec)/1000; - - e_us = ms[i]*1000; - - diff = e_us - tn_us; - - print_line("Erlang time = %ld us; ethr_time_now = %ld us; diff %ld us", - e_us, tn_us, diff); - - if (max_abs_diff < abs((int) diff)) { - max_abs_diff = abs((int) diff); - } - - ASSERT(e_us - TNT_MAX_TIME_DIFF <= tn_us); - ASSERT(tn_us <= e_us + TNT_MAX_TIME_DIFF); - } - - print_line("Max absolute diff = %d us", max_abs_diff); - succeed("Max absolute diff = %d us", max_abs_diff); -} - -/* * The cond wait test case. * * Tests ethr_cond_wait with ethr_cond_signal and ethr_cond_broadcast. @@ -1538,8 +1475,6 @@ main(int argc, char *argv[]) mutex_test(); else if (strcmp(testcase, "try_lock_mutex") == 0) try_lock_mutex_test(); - else if (strcmp(testcase, "time_now") == 0) - time_now_test(); else if (strcmp(testcase, "cond_wait") == 0) cond_wait_test(); else if (strcmp(testcase, "broadcast") == 0) diff --git a/erts/test/z_SUITE.erl b/erts/test/z_SUITE.erl index 8faddeb0d3..9f13a7083d 100644 --- a/erts/test/z_SUITE.erl +++ b/erts/test/z_SUITE.erl @@ -253,6 +253,8 @@ core_file_search(#core_search_conf{search_dir = Base, core_cand(Conf, Core, Cores); "core." ++ _ -> core_cand(Conf, Core, Cores); + Bin when is_binary(Bin) -> %Icky filename; ignore + Cores; BName -> case lists:suffix(".core", BName) of true -> core_cand(Conf, Core, Cores); diff --git a/erts/vsn.mk b/erts/vsn.mk index a5dd62feb2..8a1590e74c 100644 --- a/erts/vsn.mk +++ b/erts/vsn.mk @@ -17,8 +17,8 @@ # %CopyrightEnd% # -VSN = 5.8.2 -SYSTEM_VSN = R14B01 +VSN = 5.8.3 +SYSTEM_VSN = R14B02 # Port number 4365 in 4.2 # Port number 4366 in 4.3 diff --git a/lib/asn1/doc/src/notes.xml b/lib/asn1/doc/src/notes.xml index 375e859d20..c93adeffe2 100644 --- a/lib/asn1/doc/src/notes.xml +++ b/lib/asn1/doc/src/notes.xml @@ -31,6 +31,31 @@ <p>This document describes the changes made to the asn1 application.</p> +<section><title>Asn1 1.6.15</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + The encoding of ExtensionAdditionGroup (for PER and UPER) + is corrected.</p> + <p> + Own Id: OTP-8866 Aux Id: OTP-8797, SEQ-11557 </p> + </item> + <item> + <p> + A race condition when several processes in parallel start + to do encode/decode using the driver could cause an error + log regarding crashing port owner process. This race is + now eliminated.</p> + <p> + Own Id: OTP-8948 Aux Id: seq11733 </p> + </item> + </list> + </section> + +</section> + <section><title>Asn1 1.6.14.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/asn1/src/asn1rt_driver_handler.erl b/lib/asn1/src/asn1rt_driver_handler.erl index c95b243ae0..cc2b501e16 100644 --- a/lib/asn1/src/asn1rt_driver_handler.erl +++ b/lib/asn1/src/asn1rt_driver_handler.erl @@ -71,7 +71,10 @@ load_driver(Reason) -> end. init(FromPid,FromRef) -> - register(asn1_driver_owner,self()), + case catch register(asn1_driver_owner,self()) of + true -> true; + _Other -> exit(normal) + end, Dir = filename:join([code:priv_dir(asn1),"lib"]), case catch erl_ddll:load_driver(Dir,asn1_erl_drv) of ok -> diff --git a/lib/asn1/vsn.mk b/lib/asn1/vsn.mk index 0399ff2732..e900a52286 100644 --- a/lib/asn1/vsn.mk +++ b/lib/asn1/vsn.mk @@ -1,2 +1,2 @@ #next version number to use is 1.6.15 | 1.7 | 2.0 -ASN1_VSN = 1.6.14.1 +ASN1_VSN = 1.6.15 diff --git a/lib/common_test/doc/src/notes.xml b/lib/common_test/doc/src/notes.xml index af9dbfa9ec..2fd5dcf4f1 100644 --- a/lib/common_test/doc/src/notes.xml +++ b/lib/common_test/doc/src/notes.xml @@ -32,6 +32,57 @@ <file>notes.xml</file> </header> +<section><title>Common_Test 1.5.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Updated ct:get_status documentation to describe + no_tests_running return value.</p> + <p> + Own Id: OTP-8895 Aux Id: seq11701 </p> + </item> + <item> + <p> + Fixed race condition test failures in the test suites + testing common test's parallel groups feature.</p> + <p> + Own Id: OTP-8921</p> + </item> + <item> + <p> + The include directive of testspecs now work when used on + a remote node.</p> + <p> + Own Id: OTP-8935 Aux Id: seq11731 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + ct:parse_table can now handle multiline sql rows</p> + <p> + Own Id: OTP-8907 Aux Id: seq11702 </p> + </item> + <item> + <p> + The run_test executable has been renamed to the less + generic ct_run to better work with other applications. + run_test will remain until R16B at which point it will be + removed.</p> + <p> + Own Id: OTP-8936</p> + </item> + </list> + </section> + +</section> + <section><title>Common_Test 1.5.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/common_test/test/ct_test_support.erl b/lib/common_test/test/ct_test_support.erl index 7bfb9ffb49..5e9792f02c 100644 --- a/lib/common_test/test/ct_test_support.erl +++ b/lib/common_test/test/ct_test_support.erl @@ -43,6 +43,14 @@ init_per_suite(Config) -> init_per_suite(Config, 50). init_per_suite(Config, Level) -> + case os:type() of + {win32, _} -> + %% Extend timeout for windows as starting node + %% can take a long time there + test_server:timetrap( 120000 * test_server:timetrap_scale_factor()); + _ -> + ok + end, case delete_old_logs(os:type(), Config) of {'EXIT',DelLogsReason} -> test_server:format(0, "Failed to delete old log directories: ~p~n", @@ -51,6 +59,8 @@ init_per_suite(Config, Level) -> ok end, [_,Host] = string:tokens(atom_to_list(node()), "@"), + + test_server:format(0, "Trying to start ~s~n", ["ct@"++Host]), case slave:start(Host, ct, []) of {error,Reason} -> test_server:fail(Reason); @@ -351,13 +361,33 @@ locate({parallel,TEvs}, Node, Evs, Config) -> case Evs of [{TEH,#event{name=tc_start, node=Node, - data={M,{init_per_group,GroupName,Props}}}}, - {TEH,#event{name=tc_done, - node=Node, - data={M,{init_per_group,GroupName,Props},R}}} | Es] -> + data={M,{init_per_group, + GroupName,Props}}}}|Es] -> + %% Use dropwhile here as a tc_done from a + %% previous testcase might sneak in here + EvsG = lists:dropwhile( + fun({EH,#event{name=tc_done, + node=EvNode, + data={EvM,{init_per_group, + EvGroupName, + EvProps},EvR}}}) + when TEH == EH, EvNode == Node, EvM == M, + EvGroupName == GroupName, + EvProps == Props, + EvR == R -> + false; + ({EH,#event{name=stop_logging, + node=EvNode,data=_}}) + when EH == TEH, EvNode == Node -> + exit({group_init_done_not_found, + GroupName,Props}); + (_) -> + true + end, Es), + test_server:format("Found ~p!", [InitStart]), test_server:format("Found ~p!", [InitDone]), - {TEs,Es}; + {TEs,EvsG}; _ -> nomatch end; diff --git a/lib/common_test/vsn.mk b/lib/common_test/vsn.mk index 413ef21df3..1a820848b5 100644 --- a/lib/common_test/vsn.mk +++ b/lib/common_test/vsn.mk @@ -1,3 +1,3 @@ -COMMON_TEST_VSN = 1.5.1 +COMMON_TEST_VSN = 1.5.2 diff --git a/lib/compiler/doc/src/notes.xml b/lib/compiler/doc/src/notes.xml index 00ea0da55c..9d89b17afb 100644 --- a/lib/compiler/doc/src/notes.xml +++ b/lib/compiler/doc/src/notes.xml @@ -31,6 +31,56 @@ <p>This document describes the changes made to the Compiler application.</p> +<section><title>Compiler 4.7.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Two compiler bugs (that would cause the compiler to + terminate) reported by Christopher Williams have been + fixed.</p> + <p> + Own Id: OTP-8949</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>The compiler would translate binary comprehensions + containing tail segments in a way that would would + confuse Dialyzer. For instance:</p> + <p><c>[42 || <<_:8/integer, _/bits>> <= + Bits]</c></p> + <p> + would produce a Dialyzer warning.</p> + <p> + Own Id: OTP-8864</p> + </item> + <item> + <p> + Code such as <c>foo(A) -> <<A:0>></c> + would crash the compiler.</p> + <p> + Own Id: OTP-8865</p> + </item> + <item> + <p> + The compiler could fail with an internal error when + variables were exported from a receive block but the + return value of the receive block were not used. (Thanks + to Jim Engquist for reporting this error.)</p> + <p> + Own Id: OTP-8888</p> + </item> + </list> + </section> + +</section> + <section><title>Compiler 4.7.1</title> <section><title>Improvements and New Features</title> diff --git a/lib/compiler/src/beam_utils.erl b/lib/compiler/src/beam_utils.erl index 761d4ffec0..45cdf8a659 100644 --- a/lib/compiler/src/beam_utils.erl +++ b/lib/compiler/src/beam_utils.erl @@ -407,16 +407,23 @@ check_liveness(R, [{bif,Op,{f,Fail},Ss,D}|Is], St0) -> Other -> Other end; -check_liveness(R, [{gc_bif,Op,{f,Fail},_,Ss,D}|Is], St0) -> - case check_liveness_fail(R, Op, Ss, Fail, St0) of - {killed,St} = Killed -> - case member(R, Ss) of - true -> {used,St}; - false when R =:= D -> Killed; - false -> check_liveness(R, Is, St) - end; - Other -> - Other +check_liveness(R, [{gc_bif,Op,{f,Fail},Live,Ss,D}|Is], St0) -> + case R of + {x,X} when X >= Live -> + {killed,St0}; + {x,_} -> + {used,St0}; + _ -> + case check_liveness_fail(R, Op, Ss, Fail, St0) of + {killed,St}=Killed -> + case member(R, Ss) of + true -> {used,St}; + false when R =:= D -> Killed; + false -> check_liveness(R, Is, St) + end; + Other -> + Other + end end; check_liveness(R, [{bs_add,{f,0},Ss,D}|Is], St) -> case member(R, Ss) of @@ -482,10 +489,13 @@ check_liveness(R, [{bs_context_to_binary,S}|Is], St) -> S -> {used,St}; _ -> check_liveness(R, Is, St) end; -check_liveness(R, [{loop_rec,{f,_},{x,0}}|Is], St) -> +check_liveness(R, [{loop_rec,{f,_},{x,0}}|_], St) -> case R of - {x,_} -> {killed,St}; - _ -> check_liveness(R, Is, St) + {x,_} -> + {killed,St}; + _ -> + %% y register. Rarely happens. Be very conversative. + {unknown,St} end; check_liveness(R, [{loop_rec_end,{f,Fail}}|_], St) -> check_liveness_at(R, Fail, St); diff --git a/lib/compiler/test/compilation_SUITE.erl b/lib/compiler/test/compilation_SUITE.erl index 9c06740816..935e384d2d 100644 --- a/lib/compiler/test/compilation_SUITE.erl +++ b/lib/compiler/test/compilation_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -46,7 +46,7 @@ all(suite) -> trycatch_4, opt_crash, otp_5404,otp_5436,otp_5481,otp_5553,otp_5632, otp_5714,otp_5872,otp_6121,otp_6121a,otp_6121b, - otp_7202,otp_7345,on_load,string_table + otp_7202,otp_7345,on_load,string_table,otp_8949_a,otp_8949_a ]. -define(comp(N), @@ -606,5 +606,45 @@ string_table(Config) when is_list(Config) -> ?line {"StrT", <<"stringabletringtable">>} = StringTableChunk, ok. +otp_8949_a(Config) when is_list(Config) -> + value = otp_8949_a(), + ok. + +-record(cs, {exs,keys = [],flags = 1}). +-record(exs, {children = []}). + +otp_8949_a() -> + case id([#cs{}]) of + [#cs{}=Cs] -> + SomeVar = id(value), + if + Cs#cs.flags band 1 =/= 0 -> + id(SomeVar); + (((Cs#cs.exs)#exs.children /= []) + and + (Cs#cs.flags band (1 bsl 0 bor (1 bsl 22)) == 0)); + Cs#cs.flags band (1 bsl 22) =/= 0 -> + ok + end + end. + +otp_8949_b(Config) when is_list(Config) -> + self() ! something, + ?line value = otp_8949_b([], false), + ?line {'EXIT',_} = (catch otp_8949_b([], true)), + ok. + +%% Would cause an endless loop in beam_utils. +otp_8949_b(A, B) -> + Var = id(value), + if + A == [], B == false -> + ok + end, + receive + something -> + id(Var) + end. + id(I) -> I. diff --git a/lib/compiler/vsn.mk b/lib/compiler/vsn.mk index 4658eccd19..d180ecd4e2 100644 --- a/lib/compiler/vsn.mk +++ b/lib/compiler/vsn.mk @@ -1 +1 @@ -COMPILER_VSN = 4.7.1 +COMPILER_VSN = 4.7.2 diff --git a/lib/crypto/doc/src/notes.xml b/lib/crypto/doc/src/notes.xml index 3c571eb2a3..54dd0cb01f 100644 --- a/lib/crypto/doc/src/notes.xml +++ b/lib/crypto/doc/src/notes.xml @@ -30,6 +30,21 @@ </header> <p>This document describes the changes made to the Crypto application.</p> +<section><title>Crypto 2.0.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + AES CTR encryption support in <c>crypto</c>.</p> + <p> + Own Id: OTP-8752 Aux Id: seq11642 </p> + </item> + </list> + </section> + +</section> + <section><title>Crypto 2.0.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/crypto/vsn.mk b/lib/crypto/vsn.mk index e3549f0c50..4b35c7c0b4 100644 --- a/lib/crypto/vsn.mk +++ b/lib/crypto/vsn.mk @@ -1 +1 @@ -CRYPTO_VSN = 2.0.1 +CRYPTO_VSN = 2.0.2 diff --git a/lib/debugger/doc/src/notes.xml b/lib/debugger/doc/src/notes.xml index c72a5271ba..2f8bdc36a1 100644 --- a/lib/debugger/doc/src/notes.xml +++ b/lib/debugger/doc/src/notes.xml @@ -32,6 +32,20 @@ <p>This document describes the changes made to the Debugger application.</p> +<section><title>Debugger 3.2.5</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Miscellaneous updates</p> + <p> + Own Id: OTP-8976</p> + </item> + </list> + </section> + +</section> + <section><title>Debugger 3.2.4</title> <section><title>Improvements and New Features</title> diff --git a/lib/debugger/test/int_eval_SUITE.erl b/lib/debugger/test/int_eval_SUITE.erl index 19b006e750..1628ca69b1 100644 --- a/lib/debugger/test/int_eval_SUITE.erl +++ b/lib/debugger/test/int_eval_SUITE.erl @@ -65,10 +65,7 @@ bifs_outside_erlang(Config) when is_list(Config) -> Self = self(), ok = io:format("Self: ~p", [Self]), Info = ets:info(Id), - {owner,Self} = lists:nth(2, Info), - %% Was - %% {owner,Self} = element(2, Info), - %% in R10B. + Self = proplists:get_value(owner, Info), ?IM:ets_delete(Id), ok end, diff --git a/lib/debugger/vsn.mk b/lib/debugger/vsn.mk index 654dc11e20..b9786b4a75 100644 --- a/lib/debugger/vsn.mk +++ b/lib/debugger/vsn.mk @@ -1 +1 @@ -DEBUGGER_VSN = 3.2.4 +DEBUGGER_VSN = 3.2.5 diff --git a/lib/dialyzer/RELEASE_NOTES b/lib/dialyzer/RELEASE_NOTES index 08f274a996..3fd5e9cc7d 100644 --- a/lib/dialyzer/RELEASE_NOTES +++ b/lib/dialyzer/RELEASE_NOTES @@ -3,8 +3,13 @@ (in reversed chronological order) ============================================================================== -Version 2.x.x (in Erlang/OTP R14B01) +Version 2.4.0 (in Erlang/OTP R14B01) ------------------------------------ + - Added ability to supply multiple PLTs for the analysis (option --plts). + Currently these PLTs must be independent (i.e., no module appears in more + than one PLT) and there must not include files with module name clashes. + - Strengthened and streamlined hard-coded type information for some BIFs + and key library functions. - Fixed pretty rare infinite loop when refining the types of an SCC whose functions all returned none() (thanks to Stavros Aronis). - Fixed pretty rare crash when taking the infimum of two tuple_sets. diff --git a/lib/dialyzer/doc/manual.txt b/lib/dialyzer/doc/manual.txt index 470ddd6c73..cc6f9130c7 100644 --- a/lib/dialyzer/doc/manual.txt +++ b/lib/dialyzer/doc/manual.txt @@ -123,9 +123,10 @@ The exit status of the command line version is: Usage: dialyzer [--help] [--version] [--shell] [--quiet] [--verbose] - [-pa dir]* [--plt plt] [-Ddefine]* [-I include_dir]* - [--output_plt file] [-Wwarn]* [--src] [--gui | --wx] - [files_or_dirs] [-r dirs] [--apps applications] [-o outfile] + [-pa dir]* [--plt plt] [--plts plt*] [-Ddefine]* + [-I include_dir]* [--output_plt file] [-Wwarn]* + [--src] [--gui | --wx] [files_or_dirs] [-r dirs] + [--apps applications] [-o outfile] [--build_plt] [--add_to_plt] [--remove_from_plt] [--check_plt] [--no_check_plt] [--plt_info] [--get_warnings] [--no_native] @@ -134,59 +135,75 @@ Options: files_or_dirs (for backwards compatibility also as: -c files_or_dirs) Use Dialyzer from the command line to detect defects in the specified files or directories containing .erl or .beam files, - depending on the type of the analysis + depending on the type of the analysis. -r dirs Same as the previous but the specified directories are searched recursively for subdirectories containing .erl or .beam files in - them, depending on the type of analysis + them, depending on the type of analysis. --apps applications - Option typically used when building or modifying PLT as in: + Option typically used when building or modifying a plt as in: dialyzer --build_plt --apps erts kernel stdlib mnesia ... to conveniently refer to library applications corresponding to the Erlang/OTP installation. However, the option is general and can also be used during analysis in order to refer to Erlang/OTP applications. In addition, file or directory names can also be included, as in: dialyzer --apps inets ssl ./ebin ../other_lib/ebin/my_module.beam + -o outfile (or --output outfile) + When using Dialyzer from the command line, send the analysis + results to the specified outfile rather than to stdout. --raw When using Dialyzer from the command line, output the raw analysis results (Erlang terms) instead of the formatted result. The raw format is easier to post-process (for instance, to filter - warnings or to output HTML pages) + warnings or to output HTML pages). --src - Override the default, which is to analyze BEAM bytecode, and - analyze starting from Erlang source code instead + Override the default, which is to analyze BEAM files, and + analyze starting from Erlang source code instead. -Dname (or -Dname=value) - When analyzing from source, pass the define to Dialyzer (**) + When analyzing from source, pass the define to Dialyzer. (**) -I include_dir - When analyzing from source, pass the include_dir to Dialyzer (**) + When analyzing from source, pass the include_dir to Dialyzer. (**) -pa dir Include dir in the path for Erlang (useful when analyzing files - that have '-include_lib()' directives) + that have '-include_lib()' directives). --output_plt file - Store the plt at the specified file after building it + Store the plt at the specified file after building it. --plt plt Use the specified plt as the initial plt (if the plt was built - during setup the files will be checked for consistency) + during setup the files will be checked for consistency). + --plts plt* + Merge the specified plts to create the initial plt -- requires + that the plts are disjoint (i.e., do not have any module + appearing in more than one plt). + The plts are created in the usual way: + dialyzer --build_plt --output_plt plt_1 files_to_include + ... + dialyzer --build_plt --output_plt plt_n files_to_include + and then can be used in either of the following ways: + dialyzer files_to_analyze --plts plt_1 ... plt_n + or: + dialyzer --plts plt_1 ... plt_n -- files_to_analyze + (Note the -- delimiter in the second case) -Wwarn A family of options which selectively turn on/off warnings - (for help on the names of warnings use dialyzer -Whelp) + (for help on the names of warnings use dialyzer -Whelp). --shell - Do not disable the Erlang shell while running the GUI + Do not disable the Erlang shell while running the GUI. --version (or -v) - Prints the Dialyzer version and some more information and exits + Print the Dialyzer version and some more information and exit. --help (or -h) - Prints this message and exits + Print this message and exit. --quiet (or -q) - Makes Dialyzer a bit more quiet + Make Dialyzer a bit more quiet. --verbose - Makes Dialyzer a bit more verbose + Make Dialyzer a bit more verbose. --build_plt The analysis starts from an empty plt and creates a new one from the files specified with -c and -r. Only works for beam files. Use --plt or --output_plt to override the default plt location. --add_to_plt The plt is extended to also include the files specified with -c and -r. - Use --plt to specify wich plt to start from, and --output_plt to + Use --plt to specify which plt to start from, and --output_plt to specify where to put the plt. Note that the analysis might include files from the plt if they depend on the new files. This option only works with beam files. @@ -195,23 +212,23 @@ Options: from the plt. Note that this may cause a re-analysis of the remaining dependent files. --check_plt - Checks the plt for consistency and rebuilds it if it is not up-to-date. + Check the plt for consistency and rebuild it if it is not up-to-date. --no_check_plt Skip the plt check when running Dialyzer. Useful when working with installed plts that never change. --plt_info - Makes Dialyzer print information about the plt and then quit. The plt - can be specified with --plt. + Make Dialyzer print information about the plt and then quit. The plt + can be specified with --plt(s). --get_warnings - Makes Dialyzer emit warnings even when manipulating the plt. Only - emits warnings for files that are actually analyzed. + Make Dialyzer emit warnings even when manipulating the plt. Warnings + are only emitted for files that are actually analyzed. --dump_callgraph file Dump the call graph into the specified file whose format is determined by the file name extension. Supported extensions are: raw, dot, and ps. If something else is used as file name extension, default format '.raw' will be used. --no_native (or -nn) - Bypass the native code compilation of some key files that dialyzer + Bypass the native code compilation of some key files that Dialyzer heuristically performs when dialyzing many files; this avoids the compilation time but it may result in (much) longer analysis time. --gui @@ -231,12 +248,17 @@ Warning options: Suppress warnings for unused functions. -Wno_improper_lists Suppress warnings for construction of improper lists. + -Wno_tuple_as_fun + Suppress warnings for using tuples instead of funs. -Wno_fun_app Suppress warnings for fun applications that will fail. -Wno_match Suppress warnings for patterns that are unused or cannot match. + -Wno_opaque + Suppress warnings for violations of opaqueness of data types. -Wunmatched_returns *** - Include warnings for function calls which ignore the return value(s). + Include warnings for function calls which ignore a structured return + value or do not match against one of many possible return value(s). -Werror_handling *** Include warnings for functions that only return by means of an exception. -Wrace_conditions *** @@ -257,7 +279,7 @@ The following options are also available but their use is not recommended: Warn when the -spec is different than the success typing. Note: - *** These are options that turn on warnings rather than turning them off. + *** Identifies options that turn on warnings rather than turning them off. ----------------------------------------------- @@ -294,6 +316,7 @@ Option :: {files, [Filename :: string()]} | {defines, [{Macro :: atom(), Value :: term()}]} | {from, src_code | byte_code} %% Defaults to byte_code | {init_plt, FileName :: string()} %% If changed from default + | {plts, [FileName :: string()]} %% If changed from default | {include_dirs, [DirName :: string()]} | {output_file, FileName :: string()} | {output_plt, FileName :: string()} diff --git a/lib/dialyzer/doc/src/dialyzer.xml b/lib/dialyzer/doc/src/dialyzer.xml index 29308885fd..01a7e478bc 100644 --- a/lib/dialyzer/doc/src/dialyzer.xml +++ b/lib/dialyzer/doc/src/dialyzer.xml @@ -64,81 +64,144 @@ ]]></code> <p>Usage:</p> <code type="none"><![CDATA[ - dialyzer [--help] [--version] [--shell] [--quiet] [--verbose] - [-pa dir]* [--plt plt] [-Ddefine]* [-I include_dir]* - [--output_plt file] [-Wwarn]* [--src] - [-c applications] [-r applications] [-o outfile] + dialyzer [--help] [--version] [--shell] [--quiet] [--verbose] + [-pa dir]* [--plt plt] [--plts plt*] [-Ddefine]* + [-I include_dir]* [--output_plt file] [-Wwarn]* + [--src] [--gui | --wx] [files_or_dirs] [-r dirs] + [--apps applications] [-o outfile] [--build_plt] [--add_to_plt] [--remove_from_plt] [--check_plt] [--no_check_plt] [--plt_info] [--get_warnings] + [--no_native] ]]></code> <p>Options:</p> <taglist> - <tag><c><![CDATA[-c applications]]></c>(or <c><![CDATA[--command-line applications]]></c>)</tag> - <item>use Dialyzer from the command line (no GUI) to detect defects in the - specified applications (directories or <c><![CDATA[.erl]]></c> or <c><![CDATA[.beam]]></c> files)</item> - <tag><c><![CDATA[-r applications]]></c></tag> - <item>same as <c><![CDATA[-c]]></c> only that directories are searched recursively for - subdirectories containing <c><![CDATA[.erl]]></c> or <c><![CDATA[.beam]]></c> files (depending on the - type of analysis)</item> - <tag><c><![CDATA[-o outfile]]></c>(or <c><![CDATA[--output outfile]]></c>)</tag> - <item>when using Dialyzer from the command line, send the analysis - results in the specified <c><![CDATA[outfile]]></c> rather than in stdout</item> - <tag><c><![CDATA[--src]]></c></tag> - <item>override the default, which is to analyze debug compiled BEAM - bytecode, and analyze starting from Erlang source code instead</item> + <tag><c><![CDATA[files_or_dirs]]></c> (for backwards compatibility also + as: <c><![CDATA[-c files_or_dirs]]></c></tag> + <item>Use Dialyzer from the command line to detect defects in the + specified files or directories containing <c><![CDATA[.erl]]></c> or + <c><![CDATA[.beam]]></c> files, depending on the type of the + analysis.</item> + <tag><c><![CDATA[-r dirs]]></c></tag> + <item>Same as the previous but the specified directories are searched + recursively for subdirectories containing <c><![CDATA[.erl]]></c> or + <c><![CDATA[.beam]]></c> files in them, depending on the type of + analysis.</item> + <tag><c><![CDATA[--apps applications]]></c></tag> + <item>Option typically used when building or modifying a plt as in: + <code type="none"><![CDATA[ + dialyzer --build_plt --apps erts kernel stdlib mnesia ... + ]]></code> + to conveniently refer to library applications corresponding to the + Erlang/OTP installation. However, the option is general and can also + be used during analysis in order to refer to Erlang/OTP applications. + In addition, file or directory names can also be included, as in: + <code type="none"><![CDATA[ + dialyzer --apps inets ssl ./ebin ../other_lib/ebin/my_module.beam + ]]></code></item> + <tag><c><![CDATA[-o outfile]]></c> (or + <c><![CDATA[--output outfile]]></c>)</tag> + <item>When using Dialyzer from the command line, send the analysis + results to the specified outfile rather than to stdout.</item> <tag><c><![CDATA[--raw]]></c></tag> <item>When using Dialyzer from the command line, output the raw analysis - results (Erlang terms) instead of the formatted result. - The raw format is easier to post-process (for instance, to filter - warnings or to output HTML pages).</item> - <tag><c><![CDATA[-Dname]]></c>(or <c><![CDATA[-Dname=value]]></c>)</tag> - <item>when analyzing from source, pass the define to Dialyzer (**)</item> + results (Erlang terms) instead of the formatted result. The raw format + is easier to post-process (for instance, to filter warnings or to + output HTML pages).</item> + <tag><c><![CDATA[--src]]></c></tag> + <item>Override the default, which is to analyze BEAM files, and + analyze starting from Erlang source code instead.</item> + <tag><c><![CDATA[-Dname]]></c> (or <c><![CDATA[-Dname=value]]></c>)</tag> + <item>When analyzing from source, pass the define to Dialyzer. (**)</item> <tag><c><![CDATA[-I include_dir]]></c></tag> - <item>when analyzing from source, pass the <c><![CDATA[include_dir]]></c> to Dialyzer (**)</item> + <item>When analyzing from source, pass the <c><![CDATA[include_dir]]></c> + to Dialyzer. (**)</item> <tag><c><![CDATA[-pa dir]]></c></tag> - <item>Include <c><![CDATA[dir]]></c> in the path for Erlang. Useful when analyzing files - that have <c><![CDATA[-include_lib()]]></c> directives.</item> + <item>Include <c><![CDATA[dir]]></c> in the path for Erlang (useful when + analyzing files that have <c><![CDATA['-include_lib()']]></c> + directives).</item> <tag><c><![CDATA[--output_plt file]]></c></tag> - <item>Store the PLT at the specified location after building it.</item> + <item>Store the plt at the specified file after building it.</item> <tag><c><![CDATA[--plt plt]]></c></tag> - <item>Use the specified PLT as the initial persistent lookup table.</item> + <item>Use the specified plt as the initial plt (if the plt was built + during setup the files will be checked for consistency).</item> + <tag><c><![CDATA[--plts plt*]]></c></tag> + <item>Merge the specified plts to create the initial plt -- requires + that the plts are disjoint (i.e., do not have any module + appearing in more than one plt). + The plts are created in the usual way: + <code type="none"><![CDATA[ + dialyzer --build_plt --output_plt plt_1 files_to_include + ... + dialyzer --build_plt --output_plt plt_n files_to_include + ]]></code> + and then can be used in either of the following ways: + <code type="none"><![CDATA[ + dialyzer files_to_analyze --plts plt_1 ... plt_n + ]]></code> + or: + <code type="none"><![CDATA[ + dialyzer --plts plt_1 ... plt_n -- files_to_analyze + ]]></code> + (Note the -- delimiter in the second case)</item> <tag><c><![CDATA[-Wwarn]]></c></tag> - <item>a family of options which selectively turn on/off warnings. - (for help on the names of warnings use <c><![CDATA[dialyzer -Whelp]]></c>)</item> + <item>A family of options which selectively turn on/off warnings + (for help on the names of warnings use + <c><![CDATA[dialyzer -Whelp]]></c>).</item> <tag><c><![CDATA[--shell]]></c></tag> - <item>do not disable the Erlang shell while running the GUI</item> - <tag><c><![CDATA[--version (or -v)]]></c></tag> - <item>prints the Dialyzer version and some more information and exits</item> - <tag><c><![CDATA[--help (or -h)]]></c></tag> - <item>prints this message and exits</item> - <tag><c><![CDATA[--quiet (or -q)]]></c></tag> - <item>makes Dialyzer a bit more quiet</item> + <item>Do not disable the Erlang shell while running the GUI.</item> + <tag><c><![CDATA[--version]]></c> (or <c><![CDATA[-v]]></c>)</tag> + <item>Print the Dialyzer version and some more information and + exit.</item> + <tag><c><![CDATA[--help]]></c> (or <c><![CDATA[-h]]></c>)</tag> + <item>Print this message and exit.</item> + <tag><c><![CDATA[--quiet]]></c> (or <c><![CDATA[-q]]></c>)</tag> + <item>Make Dialyzer a bit more quiet.</item> <tag><c><![CDATA[--verbose]]></c></tag> - <item>makes Dialyzer a bit more verbose</item> - <tag><c><![CDATA[--check_plt]]></c></tag> - <item>Only checks if the initial PLT is up to date and rebuilds it if this is not the case</item> - <tag><c><![CDATA[--no_check_plt (or -n)]]></c></tag> - <item>Skip the PLT integrity check when running Dialyzer. - Useful when working with installed PLTs that never change.</item> + <item>Make Dialyzer a bit more verbose.</item> <tag><c><![CDATA[--build_plt]]></c></tag> - <item>The analysis starts from an empty PLT and creates a new one from - the files specified with -c and -r. Only works for beam files. - Use --plt or --output_plt to override the default PLT location.</item> - <tag><c><![CDATA[--add_to_plt]]></c></tag> - <item> The PLT is extended to also include the files specified with - -c and -r. Use --plt to specify which PLT to start from, and --output_plt - to specify where to put the PLT. Note that the analysis might include - files from the PLT if they depend on the new files. - This option only works with beam files.</item> + <item>The analysis starts from an empty plt and creates a new one from + the files specified with <c><![CDATA[-c]]></c> and + <c><![CDATA[-r]]></c>. Only works for beam files. Use + <c><![CDATA[--plt]]></c> or <c><![CDATA[--output_plt]]></c> to + override the default plt location.</item> + <tag><c><![CDATA[--add_to_plt]]></c></tag> + <item>The plt is extended to also include the files specified with + <c><![CDATA[-c]]></c> and <c><![CDATA[-r]]></c>. Use + <c><![CDATA[--plt]]></c> to specify which plt to start from, + and <c><![CDATA[--output_plt]]></c> to specify where to put the plt. + Note that the analysis might include files from the plt if they depend + on the new files. This option only works with beam files.</item> <tag><c><![CDATA[--remove_from_plt]]></c></tag> - <item>The information from the files specified with -c and -r is removed - from the PLT. Note that this may cause a re-analysis of the remaining - dependent files.</item> + <item>The information from the files specified with + <c><![CDATA[-c]]></c> and <c><![CDATA[-r]]></c> is removed + from the plt. Note that this may cause a re-analysis of the remaining + dependent files.</item> + <tag><c><![CDATA[--check_plt]]></c></tag> + <item>Check the plt for consistency and rebuild it if it is not + up-to-date.</item> + <tag><c><![CDATA[--no_check_plt]]></c></tag> + <item>Skip the plt check when running Dialyzer. Useful when working with + installed plts that never change.</item> + <tag><c><![CDATA[--plt_info]]></c></tag> + <item>Make Dialyzer print information about the plt and then quit. The + plt can be specified with <c><![CDATA[--plt(s)]]></c>.</item> <tag><c><![CDATA[--get_warnings]]></c></tag> - <item>Makes Dialyzer emit warnings even when manipulating the PLT. Only - emits warnings for files that are actually analyzed. The default is to - not emit any warnings when manipulating the PLT. This option has no - effect when performing a normal analysis.</item> + <item>Make Dialyzer emit warnings even when manipulating the plt. + Warnings are only emitted for files that are actually analyzed.</item> + <tag><c><![CDATA[--dump_callgraph file]]></c></tag> + <item>Dump the call graph into the specified file whose format is + determined by the file name extension. Supported extensions are: raw, + dot, and ps. If something else is used as file name extension, default + format '.raw' will be used.</item> + <tag><c><![CDATA[--no_native]]></c> (or <c><![CDATA[-nn]]></c>)</tag> + <item>Bypass the native code compilation of some key files that Dialyzer + heuristically performs when dialyzing many files; this avoids the + compilation time but it may result in (much) longer analysis + time.</item> + <tag><c><![CDATA[--gui]]></c></tag> + <item>Use the gs-based GUI.</item> + <tag><c><![CDATA[--wx]]></c></tag> + <item>Use the wx-based GUI..</item> </taglist> <note> <p>* denotes that multiple occurrences of these options are possible.</p> @@ -148,11 +211,14 @@ <p>Warning options:</p> <taglist> <tag><c><![CDATA[-Wno_return]]></c></tag> - <item>Suppress warnings for functions of no return.</item> + <item>Suppress warnings for functions that will never return a + value.</item> <tag><c><![CDATA[-Wno_unused]]></c></tag> <item>Suppress warnings for unused functions.</item> <tag><c><![CDATA[-Wno_improper_lists]]></c></tag> <item>Suppress warnings for construction of improper lists.</item> + <tag><c><![CDATA[-Wno_tuple_as_fun]]></c></tag> + <item>Suppress warnings for using tuples instead of funs.</item> <tag><c><![CDATA[-Wno_fun_app]]></c></tag> <item>Suppress warnings for fun applications that will fail.</item> <tag><c><![CDATA[-Wno_match]]></c></tag> @@ -160,6 +226,10 @@ match.</item> <tag><c><![CDATA[-Wno_opaque]]></c></tag> <item>Suppress warnings for violations of opaqueness of data types.</item> + <tag><c><![CDATA[-Wunmatched_returns]]></c>***</tag> + <item>Include warnings for function calls which ignore a structured return + value or do not match against one of many possible return + value(s).</item> <tag><c><![CDATA[-Werror_handling]]></c>***</tag> <item>Include warnings for functions that only return by means of an exception.</item> @@ -168,20 +238,22 @@ <tag><c><![CDATA[-Wbehaviours]]></c>***</tag> <item>Include warnings about behaviour callbacks which drift from the published recommended interfaces.</item> - <tag><c><![CDATA[-Wunmatched_returns]]></c>***</tag> - <item>Include warnings for function calls which ignore a structured return - value or do not match against one of many possible return value(s).</item> <tag><c><![CDATA[-Wunderspecs]]></c>***</tag> <item>Warn about underspecified functions - (the -spec is strictly more allowing than the success typing)</item> + (the -spec is strictly more allowing than the success typing).</item> + </taglist> + <p>The following options are also available but their use is not + recommended: (they are mostly for Dialyzer developers and internal + debugging)</p> + <taglist> <tag><c><![CDATA[-Woverspecs]]></c>***</tag> <item>Warn about overspecified functions - (the -spec is strictly less allowing than the success typing)</item> + (the -spec is strictly less allowing than the success typing).</item> <tag><c><![CDATA[-Wspecdiffs]]></c>***</tag> - <item>Warn when the -spec is different than the success typing</item> + <item>Warn when the -spec is different than the success typing.</item> </taglist> <note> - <p>*** These are options that turn on warnings rather than + <p>*** Identifies options that turn on warnings rather than turning them off.</p> </note> </section> @@ -210,6 +282,7 @@ Option : {files, [Filename : string()]} | {defines, [{Macro: atom(), Value : term()}]} | {from, src_code | byte_code} %% Defaults to byte_code | {init_plt, FileName : string()} %% If changed from default + | {plts, [FileName :: string()]} %% If changed from default | {include_dirs, [DirName : string()]} | {output_file, FileName : string()} | {output_plt, FileName :: string()} diff --git a/lib/dialyzer/doc/src/notes.xml b/lib/dialyzer/doc/src/notes.xml index ac3857b9ef..3678291be7 100755 --- a/lib/dialyzer/doc/src/notes.xml +++ b/lib/dialyzer/doc/src/notes.xml @@ -31,6 +31,40 @@ <p>This document describes the changes made to the Dialyzer application.</p> +<section><title>Dialyzer 2.4.0</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> - Fixed pretty rare infinite loop when refining the + types of an SCC whose functions all returned none() + (thanks to Stavros Aronis). </p><p> - Fixed pretty rare + crash when taking the infimum of two tuple_sets. </p> + <p> + Own Id: OTP-8979</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> - Added ability to supply multiple PLTs for the + analysis (option --plts). Currently these PLTs must be + independent (i.e., no module appears in more than one + PLT) and there must not include files with module name + clashes.</p><p> - Strengthened and streamlined hard-coded + type information for some BIFs and key library + functions.</p> + <p> + Own Id: OTP-8962</p> + </item> + </list> + </section> + +</section> + <section><title>Dialyzer 2.3.1</title> <section><title>Improvements and New Features</title> diff --git a/lib/dialyzer/src/dialyzer.erl b/lib/dialyzer/src/dialyzer.erl index b4161ea194..471f9fccd2 100644 --- a/lib/dialyzer/src/dialyzer.erl +++ b/lib/dialyzer/src/dialyzer.erl @@ -106,27 +106,35 @@ cl_print_plt_info(Opts) -> end, doit(F). -print_plt_info(#options{init_plt = PLT, output_file = OutputFile}) -> +print_plt_info(#options{init_plts = PLTs, output_file = OutputFile}) -> + PLTInfo = get_plt_info(PLTs), + do_print_plt_info(PLTInfo, OutputFile). + +get_plt_info([PLT|PLTs]) -> String = case dialyzer_plt:included_files(PLT) of {ok, Files} -> - io_lib:format("The PLT ~s includes the following files:\n~p\n", + io_lib:format("The PLT ~s includes the following files:\n~p\n\n", [PLT, Files]); {error, read_error} -> - Msg = io_lib:format("Could not read the PLT file ~p\n", [PLT]), + Msg = io_lib:format("Could not read the PLT file ~p\n\n", [PLT]), throw({dialyzer_error, Msg}); {error, no_such_file} -> - Msg = io_lib:format("The PLT file ~p does not exist\n", [PLT]), + Msg = io_lib:format("The PLT file ~p does not exist\n\n", [PLT]), throw({dialyzer_error, Msg}) end, + String ++ get_plt_info(PLTs); +get_plt_info([]) -> "". + +do_print_plt_info(PLTInfo, OutputFile) -> case OutputFile =:= none of true -> - io:format("~s", [String]), + io:format("~s", [PLTInfo]), ?RET_NOTHING_SUSPICIOUS; false -> case file:open(OutputFile, [write]) of {ok, FileDesc} -> - io:format(FileDesc, "~s", [String]), + io:format(FileDesc, "~s", [PLTInfo]), ok = file:close(FileDesc), ?RET_NOTHING_SUSPICIOUS; {error, Reason} -> diff --git a/lib/dialyzer/src/dialyzer.hrl b/lib/dialyzer/src/dialyzer.hrl index 2da8ed2e5d..1d98574585 100644 --- a/lib/dialyzer/src/dialyzer.hrl +++ b/lib/dialyzer/src/dialyzer.hrl @@ -129,7 +129,7 @@ defines = [] :: [dial_define()], from = byte_code :: start_from(), get_warnings = maybe :: boolean() | 'maybe', - init_plt = none :: 'none' | file:filename(), + init_plts = [] :: [file:filename()], include_dirs = [] :: [file:filename()], output_plt = none :: 'none' | file:filename(), legal_warnings = ordsets:new() :: ordset(dial_warn_tag()), diff --git a/lib/dialyzer/src/dialyzer_cl.erl b/lib/dialyzer/src/dialyzer_cl.erl index 0250c47ad0..1987c1732c 100644 --- a/lib/dialyzer/src/dialyzer_cl.erl +++ b/lib/dialyzer/src/dialyzer_cl.erl @@ -81,11 +81,15 @@ build_plt(Opts) -> init_opts_for_build(Opts) -> case Opts#options.output_plt =:= none of true -> - case Opts#options.init_plt of - none -> Opts#options{init_plt = none, output_plt = get_default_plt()}; - Plt -> Opts#options{init_plt = none, output_plt = Plt} + case Opts#options.init_plts of + [] -> Opts#options{output_plt = get_default_output_plt()}; + [Plt] -> Opts#options{init_plts = [], output_plt = Plt}; + Plts -> + Msg = io_lib:format("Could not build multiple PLT files: ~s\n", + [format_plts(Plts)]), + error(Msg) end; - false -> Opts#options{init_plt = none} + false -> Opts#options{init_plts = []} end. %%-------------------------------------------------------------------- @@ -98,39 +102,58 @@ add_to_plt(Opts) -> init_opts_for_add(Opts) -> case Opts#options.output_plt =:= none of true -> - case Opts#options.init_plt of - none -> Opts#options{output_plt = get_default_plt(), - init_plt = get_default_plt()}; - Plt -> Opts#options{output_plt = Plt} + case Opts#options.init_plts of + [] -> Opts#options{output_plt = get_default_output_plt(), + init_plts = get_default_init_plt()}; + [Plt] -> Opts#options{output_plt = Plt}; + Plts -> + Msg = io_lib:format("Could not add to multiple PLT files: ~s\n", + [format_plts(Plts)]), + error(Msg) end; false -> - case Opts#options.init_plt =:= none of - true -> Opts#options{init_plt = get_default_plt()}; + case Opts#options.init_plts =:= [] of + true -> Opts#options{init_plts = get_default_init_plt()}; false -> Opts end end. %%-------------------------------------------------------------------- -check_plt(Opts) -> +check_plt(#options{init_plts = []} = Opts) -> Opts1 = init_opts_for_check(Opts), - report_check(Opts), - plt_common(Opts1, [], []). + report_check(Opts1), + plt_common(Opts1, [], []); +check_plt(#options{init_plts = Plts} = Opts) -> + check_plt_aux(Plts, Opts). + +check_plt_aux([_] = Plt, Opts) -> + Opts1 = Opts#options{init_plts = Plt}, + Opts2 = init_opts_for_check(Opts1), + report_check(Opts2), + plt_common(Opts2, [], []); +check_plt_aux([Plt|Plts], Opts) -> + Opts1 = Opts#options{init_plts = [Plt]}, + Opts2 = init_opts_for_check(Opts1), + report_check(Opts2), + plt_common(Opts2, [], []), + check_plt_aux(Plts, Opts). init_opts_for_check(Opts) -> - Plt = - case Opts#options.init_plt of - none -> get_default_plt(); - Plt0 -> Plt0 + InitPlt = + case Opts#options.init_plts of + []-> get_default_init_plt(); + Plt -> Plt end, + [OutputPlt] = InitPlt, Opts#options{files = [], files_rec = [], analysis_type = plt_check, defines = [], from = byte_code, - init_plt = Plt, + init_plts = InitPlt, include_dirs = [], - output_plt = Plt, + output_plt = OutputPlt, use_contracts = true }. @@ -144,21 +167,25 @@ remove_from_plt(Opts) -> init_opts_for_remove(Opts) -> case Opts#options.output_plt =:= none of true -> - case Opts#options.init_plt of - none -> Opts#options{output_plt = get_default_plt(), - init_plt = get_default_plt()}; - Plt -> Opts#options{output_plt = Plt} + case Opts#options.init_plts of + [] -> Opts#options{output_plt = get_default_output_plt(), + init_plts = get_default_init_plt()}; + [Plt] -> Opts#options{output_plt = Plt}; + Plts -> + Msg = io_lib:format("Could not remove from multiple PLT files: ~s\n", + [format_plts(Plts)]), + error(Msg) end; false -> - case Opts#options.init_plt =:= none of - true -> Opts#options{init_plt = get_default_plt()}; + case Opts#options.init_plts =:= [] of + true -> Opts#options{init_plts = get_default_init_plt()}; false -> Opts end end. %%-------------------------------------------------------------------- -plt_common(Opts, RemoveFiles, AddFiles) -> +plt_common(#options{init_plts = [InitPlt]} = Opts, RemoveFiles, AddFiles) -> case check_plt(Opts, RemoveFiles, AddFiles) of ok -> case Opts#options.report_mode of @@ -174,7 +201,7 @@ plt_common(Opts, RemoveFiles, AddFiles) -> report_failed_plt_check(Opts, DiffMd5), {AnalFiles, RemovedMods, ModDeps1} = expand_dependent_modules(Md5, DiffMd5, ModDeps), - Plt = clean_plt(Opts#options.init_plt, RemovedMods), + Plt = clean_plt(InitPlt, RemovedMods), case AnalFiles =:= [] of true -> %% Only removed stuff. Just write the PLT. @@ -186,19 +213,19 @@ plt_common(Opts, RemoveFiles, AddFiles) -> end; {error, no_such_file} -> Msg = io_lib:format("Could not find the PLT: ~s\n~s", - [Opts#options.init_plt, default_plt_error_msg()]), + [InitPlt, default_plt_error_msg()]), error(Msg); {error, not_valid} -> Msg = io_lib:format("The file: ~s is not a valid PLT file\n~s", - [Opts#options.init_plt, default_plt_error_msg()]), + [InitPlt, default_plt_error_msg()]), error(Msg); {error, read_error} -> Msg = io_lib:format("Could not read the PLT: ~s\n~s", - [Opts#options.init_plt, default_plt_error_msg()]), + [InitPlt, default_plt_error_msg()]), error(Msg); {error, {no_file_to_remove, F}} -> Msg = io_lib:format("Could not remove the file ~s from the PLT: ~s\n", - [F, Opts#options.init_plt]), + [F, InitPlt]), error(Msg) end. @@ -218,8 +245,7 @@ default_plt_error_msg() -> %%-------------------------------------------------------------------- -check_plt(Opts, RemoveFiles, AddFiles) -> - Plt = Opts#options.init_plt, +check_plt(#options{init_plts = [Plt]} = Opts, RemoveFiles, AddFiles) -> case dialyzer_plt:check_plt(Plt, RemoveFiles, AddFiles) of {old_version, _MD5} = OldVersion -> report_old_version(Opts), @@ -234,14 +260,14 @@ check_plt(Opts, RemoveFiles, AddFiles) -> %%-------------------------------------------------------------------- -report_check(#options{report_mode = ReportMode, init_plt = InitPlt}) -> +report_check(#options{report_mode = ReportMode, init_plts = [InitPlt]}) -> case ReportMode of quiet -> ok; _ -> io:format(" Checking whether the PLT ~s is up-to-date...", [InitPlt]) end. -report_old_version(#options{report_mode = ReportMode, init_plt = InitPlt}) -> +report_old_version(#options{report_mode = ReportMode, init_plts = [InitPlt]}) -> case ReportMode of quiet -> ok; _ -> @@ -264,7 +290,7 @@ report_failed_plt_check(#options{analysis_type = AnalType, report_analysis_start(#options{analysis_type = Type, report_mode = ReportMode, - init_plt = InitPlt, + init_plts = InitPlts, output_plt = OutputPlt}) -> case ReportMode of quiet -> ok; @@ -272,6 +298,7 @@ report_analysis_start(#options{analysis_type = Type, io:format(" "), case Type of plt_add -> + [InitPlt] = InitPlts, case InitPlt =:= OutputPlt of true -> io:format("Adding information to ~s...", [OutputPlt]); false -> io:format("Adding information from ~s to ~s...", @@ -282,6 +309,7 @@ report_analysis_start(#options{analysis_type = Type, plt_check -> io:format("Rebuilding the information in ~s...", [OutputPlt]); plt_remove -> + [InitPlt] = InitPlts, case InitPlt =:= OutputPlt of true -> io:format("Removing information from ~s...", [OutputPlt]); false -> io:format("Removing information from ~s to ~s...", @@ -320,16 +348,28 @@ report_md5_diff(List) -> %%-------------------------------------------------------------------- -get_default_plt() -> +get_default_init_plt() -> + [dialyzer_plt:get_default_plt()]. + +get_default_output_plt() -> dialyzer_plt:get_default_plt(). %%-------------------------------------------------------------------- +format_plts([Plt]) -> Plt; +format_plts([Plt|Plts]) -> + Plt ++ ", " ++ format_plts(Plts). + +%%-------------------------------------------------------------------- + do_analysis(Options) -> Files = get_files_from_opts(Options), - case Options#options.init_plt of - none -> do_analysis(Files, Options, dialyzer_plt:new(), none); - File -> do_analysis(Files, Options, dialyzer_plt:from_file(File), none) + case Options#options.init_plts of + [] -> do_analysis(Files, Options, dialyzer_plt:new(), none); + PltFiles -> + Plts = [dialyzer_plt:from_file(F) || F <- PltFiles], + Plt = dialyzer_plt:merge_plts_or_report_conflicts(PltFiles, Plts), + do_analysis(Files, Options, Plt, none) end. do_analysis(Files, Options, Plt, PltInfo) -> diff --git a/lib/dialyzer/src/dialyzer_cl_parse.erl b/lib/dialyzer/src/dialyzer_cl_parse.erl index 0160b84abc..5ca7599b35 100644 --- a/lib/dialyzer/src/dialyzer_cl_parse.erl +++ b/lib/dialyzer/src/dialyzer_cl_parse.erl @@ -138,11 +138,17 @@ cl(["-pa", Path|T]) -> true -> cl(T); {error, _} -> error("Bad directory for -pa: "++Path) end; -cl(["--plt", PLT|T]) -> - put(dialyzer_init_plt, PLT), - cl(T); cl(["--plt"]) -> error("No plt specified for --plt"); +cl(["--plt", PLT|T]) -> + put(dialyzer_init_plts, [PLT]), + cl(T); +cl(["--plts"]) -> + error("No plts specified for --plts"); +cl(["--plts"|T]) -> + {PLTs, NewT} = get_plts(T, []), + put(dialyzer_init_plts, PLTs), + cl(NewT); cl(["-q"|T]) -> put(dialyzer_options_report_mode, quiet), cl(T); @@ -284,7 +290,7 @@ common_options() -> [{defines, get(dialyzer_options_defines)}, {from, get(dialyzer_options_from)}, {include_dirs, get(dialyzer_include)}, - {init_plt, get(dialyzer_init_plt)}, + {plts, get(dialyzer_init_plts)}, {output_plt, get(dialyzer_output_plt)}, {report_mode, get(dialyzer_options_report_mode)}, {use_spec, get(dialyzer_options_use_contracts)}, @@ -309,6 +315,13 @@ get_lib_dir([], Acc) -> %%----------------------------------------------------------------------- +get_plts(["--"|T], Acc) -> {lists:reverse(Acc), T}; +get_plts(["-"++_Opt = H|T], Acc) -> {lists:reverse(Acc), [H|T]}; +get_plts([H|T], Acc) -> get_plts(T, [H|Acc]); +get_plts([], Acc) -> {lists:reverse(Acc), []}. + +%%----------------------------------------------------------------------- + help_warnings() -> S = warning_options_msg(), io:put_chars(S), @@ -316,9 +329,10 @@ help_warnings() -> help_message() -> S = "Usage: dialyzer [--help] [--version] [--shell] [--quiet] [--verbose] - [-pa dir]* [--plt plt] [-Ddefine]* [-I include_dir]* - [--output_plt file] [-Wwarn]* [--src] [--gui | --wx] - [files_or_dirs] [-r dirs] [--apps applications] [-o outfile] + [-pa dir]* [--plt plt] [--plts plt*] [-Ddefine]* + [-I include_dir]* [--output_plt file] [-Wwarn]* + [--src] [--gui | --wx] [files_or_dirs] [-r dirs] + [--apps applications] [-o outfile] [--build_plt] [--add_to_plt] [--remove_from_plt] [--check_plt] [--no_check_plt] [--plt_info] [--get_warnings] [--no_native] @@ -326,13 +340,13 @@ Options: files_or_dirs (for backwards compatibility also as: -c files_or_dirs) Use Dialyzer from the command line to detect defects in the specified files or directories containing .erl or .beam files, - depending on the type of the analysis + depending on the type of the analysis. -r dirs Same as the previous but the specified directories are searched recursively for subdirectories containing .erl or .beam files in - them, depending on the type of analysis + them, depending on the type of analysis. --apps applications - Option typically used when building or modifying a PLT as in: + Option typically used when building or modifying a plt as in: dialyzer --build_plt --apps erts kernel stdlib mnesia ... to conveniently refer to library applications corresponding to the Erlang/OTP installation. However, the option is general and can also @@ -341,73 +355,86 @@ Options: dialyzer --apps inets ssl ./ebin ../other_lib/ebin/my_module.beam -o outfile (or --output outfile) When using Dialyzer from the command line, send the analysis - results to the specified \"outfile\" rather than to stdout + results to the specified outfile rather than to stdout. --raw When using Dialyzer from the command line, output the raw analysis results (Erlang terms) instead of the formatted result. The raw format is easier to post-process (for instance, to filter - warnings or to output HTML pages) + warnings or to output HTML pages). --src Override the default, which is to analyze BEAM files, and - analyze starting from Erlang source code instead + analyze starting from Erlang source code instead. -Dname (or -Dname=value) - When analyzing from source, pass the define to Dialyzer (**) + When analyzing from source, pass the define to Dialyzer. (**) -I include_dir - When analyzing from source, pass the include_dir to Dialyzer (**) + When analyzing from source, pass the include_dir to Dialyzer. (**) -pa dir Include dir in the path for Erlang (useful when analyzing files - that have '-include_lib()' directives) + that have '-include_lib()' directives). --output_plt file - Store the plt at the specified file after building it + Store the plt at the specified file after building it. --plt plt Use the specified plt as the initial plt (if the plt was built - during setup the files will be checked for consistency) + during setup the files will be checked for consistency). + --plts plt* + Merge the specified plts to create the initial plt -- requires + that the plts are disjoint (i.e., do not have any module + appearing in more than one plt). + The plts are created in the usual way: + dialyzer --build_plt --output_plt plt_1 files_to_include + ... + dialyzer --build_plt --output_plt plt_n files_to_include + and then can be used in either of the following ways: + dialyzer files_to_analyze --plts plt_1 ... plt_n + or: + dialyzer --plts plt_1 ... plt_n -- files_to_analyze + (Note the -- delimiter in the second case) -Wwarn A family of options which selectively turn on/off warnings - (for help on the names of warnings use dialyzer -Whelp) + (for help on the names of warnings use dialyzer -Whelp). --shell - Do not disable the Erlang shell while running the GUI + Do not disable the Erlang shell while running the GUI. --version (or -v) - Prints the Dialyzer version and some more information and exits + Print the Dialyzer version and some more information and exit. --help (or -h) - Prints this message and exits + Print this message and exit. --quiet (or -q) - Makes Dialyzer a bit more quiet + Make Dialyzer a bit more quiet. --verbose - Makes Dialyzer a bit more verbose + Make Dialyzer a bit more verbose. --build_plt The analysis starts from an empty plt and creates a new one from the files specified with -c and -r. Only works for beam files. - Use --plt or --output_plt to override the default plt location. + Use --plt(s) or --output_plt to override the default plt location. --add_to_plt The plt is extended to also include the files specified with -c and -r. - Use --plt to specify wich plt to start from, and --output_plt to - specify where to put the plt. Note that the analysis might include - files from the plt if they depend on the new files. + Use --plt(s) to specify which plt to start from, and --output_plt to + specify where to put the plt. Note that the analysis might include + files from the plt if they depend on the new files. This option only works with beam files. --remove_from_plt The information from the files specified with -c and -r is removed from the plt. Note that this may cause a re-analysis of the remaining dependent files. --check_plt - Checks the plt for consistency and rebuilds it if it is not up-to-date. + Check the plt for consistency and rebuild it if it is not up-to-date. Actually, this option is of rare use as it is on by default. --no_check_plt (or -n) Skip the plt check when running Dialyzer. Useful when working with installed plts that never change. --plt_info - Makes Dialyzer print information about the plt and then quit. The plt - can be specified with --plt. + Make Dialyzer print information about the plt and then quit. The plt + can be specified with --plt(s). --get_warnings - Makes Dialyzer emit warnings even when manipulating the plt. Only - emits warnings for files that are actually analyzed. + Make Dialyzer emit warnings even when manipulating the plt. Warnings + are only emitted for files that are actually analyzed. --dump_callgraph file Dump the call graph into the specified file whose format is determined by the file name extension. Supported extensions are: raw, dot, and ps. If something else is used as file name extension, default format '.raw' will be used. --no_native (or -nn) - Bypass the native code compilation of some key files that dialyzer + Bypass the native code compilation of some key files that Dialyzer heuristically performs when dialyzing many files; this avoids the compilation time but it may result in (much) longer analysis time. --gui diff --git a/lib/dialyzer/src/dialyzer_gui.erl b/lib/dialyzer/src/dialyzer_gui.erl index f353638cdf..4436330f7f 100644 --- a/lib/dialyzer/src/dialyzer_gui.erl +++ b/lib/dialyzer/src/dialyzer_gui.erl @@ -88,8 +88,8 @@ -spec start(#options{}) -> ?RET_NOTHING_SUSPICIOUS. -start(DialyzerOptions = #options{from = From, init_plt = InitPltFile, - legal_warnings = LegalWarnings}) -> +start(#options{from = From, init_plts = InitPltFiles, + legal_warnings = LegalWarnings} = DialyzerOptions) -> process_flag(trap_exit, true), GS = gs:start(), @@ -336,9 +336,13 @@ start(DialyzerOptions = #options{from = From, init_plt = InitPltFile, gs:config(Packer, WH), {ok, CWD} = file:get_cwd(), - InitPlt = try dialyzer_plt:from_file(InitPltFile) - catch throw:{dialyzer_error, _} -> dialyzer_plt:new() - end, + InitPlt = + case InitPltFiles of + [] -> dialyzer_plt:new(); + _ -> + Plts = [dialyzer_plt:from_file(F) || F <- InitPltFiles], + dialyzer_plt:merge_plts_or_report_conflicts(InitPltFiles, Plts) + end, State = #gui_state{add_all = AddAll, add_file = AddFile, diff --git a/lib/dialyzer/src/dialyzer_gui_wx.erl b/lib/dialyzer/src/dialyzer_gui_wx.erl index 2e309d7ec1..e711c15ea7 100644 --- a/lib/dialyzer/src/dialyzer_gui_wx.erl +++ b/lib/dialyzer/src/dialyzer_gui_wx.erl @@ -88,7 +88,7 @@ start(DialyzerOptions) -> State = wx:batch(fun() -> create_window(Wx, DialyzerOptions) end), gui_loop(State). -create_window(Wx, DialyzerOptions) -> +create_window(Wx, #options{init_plts = InitPltFiles} = DialyzerOptions) -> {ok, Host} = inet:gethostname(), %%---------- initializing frame --------- @@ -258,11 +258,15 @@ create_window(Wx, DialyzerOptions) -> plt = PltMenu, options =OptionsMenu, help = HelpMenu}, - - InitPlt = try dialyzer_plt:from_file(DialyzerOptions#options.init_plt) - catch throw:{dialyzer_error, _} -> dialyzer_plt:new() - end, + InitPlt = + case InitPltFiles of + [] -> dialyzer_plt:new(); + _ -> + Plts = [dialyzer_plt:from_file(F) || F <- InitPltFiles], + dialyzer_plt:merge_plts_or_report_conflicts(InitPltFiles, Plts) + end, + #gui_state{add = AddButton, add_dir = AddDirButton, add_rec = AddRecButton, diff --git a/lib/dialyzer/src/dialyzer_options.erl b/lib/dialyzer/src/dialyzer_options.erl index 010625b7bd..2c0afa6e2b 100644 --- a/lib/dialyzer/src/dialyzer_options.erl +++ b/lib/dialyzer/src/dialyzer_options.erl @@ -53,14 +53,21 @@ build(Opts) -> InitPlt = dialyzer_plt:get_default_plt(), DefaultOpts = #options{}, DefaultOpts1 = DefaultOpts#options{legal_warnings = DefaultWarns1, - init_plt = InitPlt}, - try - NewOpts = build_options(Opts, DefaultOpts1), + init_plts = [InitPlt]}, + try + Opts1 = preprocess_opts(Opts), + NewOpts = build_options(Opts1, DefaultOpts1), postprocess_opts(NewOpts) catch throw:{dialyzer_options_error, Msg} -> {error, Msg} end. +preprocess_opts([]) -> []; +preprocess_opts([{init_plt, File}|Opts]) -> + [{plts, [File]}|preprocess_opts(Opts)]; +preprocess_opts([Opt|Opts]) -> + [Opt|preprocess_opts(Opts)]. + postprocess_opts(Opts = #options{}) -> Opts1 = check_output_plt(Opts), adapt_get_warnings(Opts1). @@ -144,9 +151,9 @@ build_options([{OptionName, Value} = Term|Rest], Options) -> build_options(Rest, Options#options{from = Value}); get_warnings -> build_options(Rest, Options#options{get_warnings = Value}); - init_plt -> - assert_filenames([Term], [Value]), - build_options(Rest, Options#options{init_plt = Value}); + plts -> + assert_filenames(Term, Value), + build_options(Rest, Options#options{init_plts = Value}); include_dirs -> assert_filenames(Term, Value), OldVal = Options#options.include_dirs, diff --git a/lib/dialyzer/src/dialyzer_plt.erl b/lib/dialyzer/src/dialyzer_plt.erl index 08d0b318b5..a7ba270c41 100644 --- a/lib/dialyzer/src/dialyzer_plt.erl +++ b/lib/dialyzer/src/dialyzer_plt.erl @@ -51,6 +51,7 @@ lookup_contract/2, lookup_module/2, merge_plts/1, + merge_plts_or_report_conflicts/2, new/0, plt_and_info_from_file/1, get_specs/1, @@ -292,6 +293,38 @@ merge_plts(List) -> exported_types = sets_merge(ExpTypesList), contracts = table_merge(ContractsList)}. +-spec merge_disj_plts([plt()]) -> plt(). + +merge_disj_plts(List) -> + InfoList = [Info || #plt{info = Info} <- List], + TypesList = [Types || #plt{types = Types} <- List], + ExpTypesList = [ExpTypes || #plt{exported_types = ExpTypes} <- List], + ContractsList = [Contracts || #plt{contracts = Contracts} <- List], + #plt{info = table_disj_merge(InfoList), + types = table_disj_merge(TypesList), + exported_types = sets_disj_merge(ExpTypesList), + contracts = table_disj_merge(ContractsList)}. + +-spec merge_plts_or_report_conflicts([file:filename()], [plt()]) -> plt(). + +merge_plts_or_report_conflicts(PltFiles, Plts) -> + try + merge_disj_plts(Plts) + catch throw:{dialyzer_error, not_disjoint_plts} -> + IncFiles = lists:append([begin {ok, Fs} = included_files(F), Fs end + || F <- PltFiles]), + ConfFiles = find_duplicates(IncFiles), + Msg = io_lib:format("Could not merge PLTs since they are not disjoint\n" + "The following files are included in more than one " + "PLTs:\n~p\n", [ConfFiles]), + error(Msg) + end. + +find_duplicates(List) -> + ModList = [filename:basename(E) || E <- List], + SortedList = lists:usort(ModList), + lists:usort(ModList -- SortedList). + -spec to_file(file:filename(), plt(), mod_deps(), {[file_md5()], mod_deps()}) -> 'ok'. to_file(FileName, @@ -556,6 +589,25 @@ table_merge([Plt|Plts], Acc) -> NewAcc = dict:merge(fun(_Key, Val, Val) -> Val end, Plt, Acc), table_merge(Plts, NewAcc). +table_disj_merge([H|T]) -> + table_disj_merge(T, H). + +table_disj_merge([], Acc) -> + Acc; +table_disj_merge([Plt|Plts], Acc) -> + case table_is_disjoint(Plt, Acc) of + true -> + NewAcc = dict:merge(fun(_Key, _Val1, _Val2) -> gazonk end, + Plt, Acc), + table_disj_merge(Plts, NewAcc); + false -> throw({dialyzer_error, not_disjoint_plts}) + end. + +table_is_disjoint(T1, T2) -> + K1 = dict:fetch_keys(T1), + K2 = dict:fetch_keys(T2), + lists:all(fun(E) -> not lists:member(E, K2) end, K1). + sets_merge([H|T]) -> sets_merge(T, H). @@ -565,6 +617,19 @@ sets_merge([Plt|Plts], Acc) -> NewAcc = sets:union(Plt, Acc), sets_merge(Plts, NewAcc). +sets_disj_merge([H|T]) -> + sets_disj_merge(T, H). + +sets_disj_merge([], Acc) -> + Acc; +sets_disj_merge([Plt|Plts], Acc) -> + case sets:is_disjoint(Plt, Acc) of + true -> + NewAcc = sets:union(Plt, Acc), + sets_disj_merge(Plts, NewAcc); + false -> throw({dialyzer_error, not_disjoint_plts}) + end. + %%--------------------------------------------------------------------------- %% Debug utilities. diff --git a/lib/dialyzer/vsn.mk b/lib/dialyzer/vsn.mk index d3574e0a71..b2902e95ed 100644 --- a/lib/dialyzer/vsn.mk +++ b/lib/dialyzer/vsn.mk @@ -1 +1 @@ -DIALYZER_VSN = 2.3.1 +DIALYZER_VSN = 2.4.0 diff --git a/lib/docbuilder/doc/src/notes.xml b/lib/docbuilder/doc/src/notes.xml index 019cf1b083..4b8c04f323 100644 --- a/lib/docbuilder/doc/src/notes.xml +++ b/lib/docbuilder/doc/src/notes.xml @@ -31,6 +31,21 @@ <p>This document describes the changes made to the DocBuilder application.</p> +<section><title>Docbuilder 0.9.8.9</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Fix compatibility issues with docbuilder for R11 + documentation patches. </p> + <p> + Own Id: OTP-8946</p> + </item> + </list> + </section> + +</section> + <section><title>Docbuilder 0.9.8.8</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/docbuilder/src/docb_main.erl b/lib/docbuilder/src/docb_main.erl index 87a1401a02..4f5f035a65 100644 --- a/lib/docbuilder/src/docb_main.erl +++ b/lib/docbuilder/src/docb_main.erl @@ -34,14 +34,23 @@ %% Parses the source file File and transforms the result to html, %% latex and/or man page format. process(File, Opts) -> - - File1 = File ++ ".tmpconv", - os:cmd("sed -e 's/xi:include[ \t]*href/include file/g' -e 's/xmlns:xi=\"http:\\/\\/www.w3.org\\/2001\\/XInclude\"//g' < " ++ - File ++ ".xml > " ++ File1 ++ ".xml"), %LATH + + SrcType = docb_util:lookup_option(src_type, Opts), + + File1 = + case SrcType of + ".xml" -> + FileTmp = File ++ ".tmpconv", + os:cmd("sed -e 's/xi:include[ \t]*href/include file/g' -e 's/xmlns:xi=\"http:\\/\\/www.w3.org\\/2001\\/XInclude\"//g' < " ++ + File ++ ".xml > " ++ FileTmp ++ ".xml"), + FileTmp; + ".sgml" -> + File + end, case parse1(File1, Opts) of errors -> - file:delete(File1 ++ ".xml"), + delete_tmp_file(SrcType, File1), errors; {ok, Tree} -> From = element(1, Tree), @@ -62,15 +71,21 @@ process(File, Opts) -> Result = [transform(From, To, Opts, File, Tree)||To <- Tos], case lists:member(transformation_error,Result) of true -> - file:delete(File1 ++ ".xml"), + delete_tmp_file(SrcType, File1), errors; _ -> - file:delete(File1 ++ ".xml"), + delete_tmp_file(SrcType, File1), ok end end. + +delete_tmp_file(".xml", File) -> + file:delete(File ++ ".xml"); +delete_tmp_file(_, _) -> + ok. + %%---------------------------------------------------------------------- %% parse(File, Opts) -> {ok, Tree} | errors diff --git a/lib/docbuilder/vsn.mk b/lib/docbuilder/vsn.mk index b23ee521c7..1209b80d94 100644 --- a/lib/docbuilder/vsn.mk +++ b/lib/docbuilder/vsn.mk @@ -1 +1 @@ -DOCB_VSN = 0.9.8.8 +DOCB_VSN = 0.9.8.9 diff --git a/lib/edoc/doc/src/notes.xml b/lib/edoc/doc/src/notes.xml index 83ad27ed31..afcccf22b5 100644 --- a/lib/edoc/doc/src/notes.xml +++ b/lib/edoc/doc/src/notes.xml @@ -31,6 +31,21 @@ <p>This document describes the changes made to the EDoc application.</p> +<section><title>Edoc 0.7.6.8</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Compiler warnings were eliminated.</p> + <p> + Own Id: OTP-8855</p> + </item> + </list> + </section> + +</section> + <section><title>Edoc 0.7.6.7</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/edoc/vsn.mk b/lib/edoc/vsn.mk index 75e9a5c971..e030174862 100644 --- a/lib/edoc/vsn.mk +++ b/lib/edoc/vsn.mk @@ -1 +1 @@ -EDOC_VSN = 0.7.6.7 +EDOC_VSN = 0.7.6.8 diff --git a/lib/erl_docgen/Makefile b/lib/erl_docgen/Makefile index c5bed632a5..93a6353cac 100644 --- a/lib/erl_docgen/Makefile +++ b/lib/erl_docgen/Makefile @@ -1,19 +1,20 @@ -# ``The contents of this file are subject to the Erlang Public License, +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 1996-2010. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in # compliance with the License. You should have received a copy of the # Erlang Public License along with this software. If not, it can be -# retrieved via the world wide web at http://www.erlang.org/. +# retrieved online at http://www.erlang.org/. # # Software distributed under the License is distributed on an "AS IS" # basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See # the License for the specific language governing rights and limitations # under the License. # -# The Initial Developer of the Original Code is Ericsson Utvecklings AB. -# Portions created by Ericsson are Copyright 1999, Ericsson Utvecklings -# AB. All Rights Reserved.'' -# -# $Id$ +# %CopyrightEnd% # include $(ERL_TOP)/make/target.mk include $(ERL_TOP)/make/$(TARGET)/otp.mk @@ -22,9 +23,11 @@ include $(ERL_TOP)/make/$(TARGET)/otp.mk # Macros # -SUB_DIRECTORIES = priv +SUB_DIRECTORIES = src priv #doc/src +include vsn.mk +VSN = $(ERL_DOCGEN_VSN) SPECIAL_TARGETS = diff --git a/lib/erl_docgen/doc/src/notes.xml b/lib/erl_docgen/doc/src/notes.xml index 5b5398fec6..c7a7926c40 100644 --- a/lib/erl_docgen/doc/src/notes.xml +++ b/lib/erl_docgen/doc/src/notes.xml @@ -29,7 +29,36 @@ <file>notes.xml</file> </header> <p>This document describes the changes made to the erl_docgen application.</p> - <section><title>Erl_Docgen 0.2.2</title> + <section><title>Erl_Docgen 0.2.3</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix format_man_pages so it handles all man sections + and remove warnings/errors in various man pages. </p> + <p> + Own Id: OTP-8600</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Support for using Dialyzer specifications and types + has been added. This is an experimental release; changes + are expected before the new functionality is used when + building the OTP documentation. </p> + <p> + Own Id: OTP-8720</p> + </item> + </list> + </section> + +</section> + +<section><title>Erl_Docgen 0.2.2</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/erl_docgen/ebin/.gitignore b/lib/erl_docgen/ebin/.gitignore new file mode 100644 index 0000000000..e69de29bb2 --- /dev/null +++ b/lib/erl_docgen/ebin/.gitignore diff --git a/lib/erl_docgen/priv/bin/specs_gen.escript b/lib/erl_docgen/priv/bin/specs_gen.escript new file mode 100644 index 0000000000..840fed6dd5 --- /dev/null +++ b/lib/erl_docgen/priv/bin/specs_gen.escript @@ -0,0 +1,129 @@ +#!/usr/bin/env escript +%% -*- erlang -*- +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% + +%%% <script> [-I<dir>]... [-o<dir>] [-module Module] [File] +%%% +%%% Use EDoc and the layout module 'otp_specs' to create an XML file +%%% containing Dialyzer types and specifications (-type, -spec). +%%% +%%% Options: +%%% +%%% "-o<dir>" The output directory for the created file. +%%% Default is ".". +%%% "-I<dir>" Directory to be searched when including a file. +%%% "-module Module" +%%% Module name to use when there is no File argument. +%%% A temporary file will be created. +%%% Exactly one of -module Module and File must be given. +%%% +%%% The name of the generated file is "specs_<module>.xml". Its exact +%%% format is not further described here. + +main(Args) -> + case catch parse(Args, [], ".", no_module) of + {ok, FileSpec, InclFs, Dir} -> + call_edoc(FileSpec, InclFs, Dir); + {error, Msg} -> + io:format("~s\n", [Msg]), + usage() + end. + +parse(["-o"++Dir | Opts], InclFs, _, Module) -> + parse(Opts, InclFs, Dir, Module); +parse(["-I"++I | Opts], InclFs, Dir, Module) -> + parse(Opts, [I | InclFs], Dir, Module); +parse(["-module", Module | Opts], InclFs, Dir, _) -> + parse(Opts, InclFs, Dir, Module); +parse([File], InclFs, Dir, no_module) -> + {ok, {file, File}, lists:reverse(InclFs), Dir}; +parse([_], _, _, _) -> + {error, io_lib:format("Cannot have both -module option and file", [])}; +parse([], _, _, no_module) -> + {error, io_lib:format("Missing -module option or file", [])}; +parse([], InclFs, Dir, Module) -> + {ok, {module, Module}, lists:reverse(InclFs), Dir}; +parse(Args, _, _, _) -> + {error, io_lib:format("Bad arguments: ~p", [Args])}. + +usage() -> + io:format("usage: ~s [-I<include_dir>]... [-o<out_dir>] " + "[-module <module>] [file]\n", [escript:script_name()]), + halt(1). + +call_edoc(FileSpec, InclFs, Dir) -> + Incl = [{includes, InclFs}], + Pre = [{preprocess, true}], + Choice = [{dialyzer_specs, all}], + DirOpt = [{dir, Dir}], + Pretty = [{pretty_print, erl_pp}], + Layout = [{layout, otp_specs}, + {file_suffix, ".specs"}, + {stylesheet, ""}], + Warn = [{report_missing_type, false}, + {report_type_mismatch, false}], + OptionList = (DirOpt ++ Choice ++ Pre ++ Warn ++ Pretty ++ Layout ++ Incl), + {File, TmpFile} = case FileSpec of + {file, File0} -> + {File0, false}; + {module, Module} -> + {create_tmp_file(Dir, Module), true} + end, + try edoc:files([File], OptionList) of + ok -> + clean_up(Dir, File, TmpFile), + rename(Dir, File) + catch + _:_ -> + io:format("EDoc could not process file '~s'\n", [File]), + clean_up(Dir, File, TmpFile), + halt(3) + end. + +rename(Dir, F) -> + Mod = filename:basename(F, ".erl"), + Old = filename:join(Dir, Mod ++ ".specs"), + New = filename:join(Dir, "specs_" ++ Mod ++ ".xml"), + case file:rename(Old, New) of + ok -> + ok; + {error, R} -> + R1 = file:format_error(R), + io:format("could not rename file '~s': ~s\n", [New, R1]), + halt(2) + end. + +clean_up(Dir, File, TmpFile) -> + [file:delete(File) || TmpFile], + _ = [file:delete(filename:join(Dir, F)) || + F <- ["packages-frame.html", + "overview-summary.html", + "modules-frame.html", + "index.html", "erlang.png", "edoc-info"]], + ok. + +create_tmp_file(Dir, Module) -> + TmpFile = filename:join(Dir, Module++".erl"), + case file:write_file(TmpFile, "-module(" ++ Module ++ ").\n") of + ok -> + TmpFile; + {error, R} -> + R1 = file:format_error(R), + io:format("could not write file '~s': ~s\n", [TmpFile, R1]), + halt(2) + end. diff --git a/lib/erl_docgen/priv/bin/xref_mod_app.escript b/lib/erl_docgen/priv/bin/xref_mod_app.escript new file mode 100755 index 0000000000..fcc3a96ada --- /dev/null +++ b/lib/erl_docgen/priv/bin/xref_mod_app.escript @@ -0,0 +1,107 @@ +#!/usr/bin/env escript +%% -*- erlang -*- +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% + +%%% Find all applications and all modules given a root directory. +%%% Output an XML file that can be used for finding which application +%%% a given module belongs to. +%%% +%%% Options: +%%% +%%% "-topdir <D>" Applications are found under D/lib/. +%%% The default value is $ERL_TOP. +%%% +%%% "-outfile <F>" Output is written onto F. +%%% The default value is "mod2app.xml". +%%% +%%% The output file has the following format: +%%% +%%% <?xml version="1.0"?> +%%% <mod2app> +%%% <module name="ModName1">AppName1</module> +%%% ... +%%% <mod2app> +%%% +%%% meaning that module ModName1 resides in application AppName1. + +main(Args) -> + case catch parse(Args, os:getenv("ERL_TOP"), "mod2app.xml") of + {ok, TopDir, OutFile} -> + case modapp(TopDir) of + [] -> + io:format("no applications found\n"), + halt(3); + MA -> + Layout = layout(MA), + XML = xmerl:export_simple(Layout, xmerl_xml), + write_file(XML, OutFile) + end; + {error, Msg} -> + io:format("~s\n", [Msg]), + usage() + end. + +parse(["-topdir", TopDir | Opts], _, OutFile) -> + parse(Opts, TopDir, OutFile); +parse(["-outfile", OutFile | Opts], TopDir, _) -> + parse(Opts, TopDir, OutFile); +parse([], TopDir, OutFile) -> + {ok, TopDir, OutFile}; +parse([Opt | _], _, _) -> + {error, io_lib:format("Bad option: ~p", [Opt])}. + +usage() -> + io:format("usage: ~s [-topdir <dir>] [-outfile <file>]\n", + [escript:script_name()]), + halt(1). + +modapp(TopDir) -> + AppDirs = filelib:wildcard(filename:join([TopDir,"lib","*"])), + AM = [appmods(D) || D <- AppDirs], + lists:keysort(1, [{M,A} || {A,Ms} <- AM, M <- Ms]). + +%% It's OK if too much data is generated as long as all applications +%% and all modules are mentioned. +appmods(D) -> + ErlFiles = filelib:wildcard(filename:join([D,"src","*.erl"])), + AppV = filename:basename(D), + App = case string:rstr(AppV, "-") of + 0 -> AppV; + P -> string:sub_string(AppV, 1, P-1) + end, + {App, [filename:basename(EF, ".erl") || EF <- ErlFiles]}. + +-include_lib("xmerl/include/xmerl.hrl"). + +-define(IND(N), lists:duplicate(N, $\s)). +-define(NL, "\n"). + +layout(MAL) -> + ML = lists:append([[?IND(2),{module,[{name,M}],[A]},?NL] || {M,A} <- MAL]), + [?NL,{mod2app,[?NL|ML]},?NL]. + +write_file(Text, File) -> + case file:open(File, [write]) of + {ok, FD} -> + io:put_chars(FD, Text), + ok = file:close(FD); + {error, R} -> + R1 = file:format_error(R), + io:format("could not write file '~s': ~s\n", [File, R1]), + halt(2) + end. diff --git a/lib/erl_docgen/priv/docbuilder_dtd/common.refs.dtd b/lib/erl_docgen/priv/docbuilder_dtd/common.refs.dtd index 7b9974fbda..c1237766e1 100644 --- a/lib/erl_docgen/priv/docbuilder_dtd/common.refs.dtd +++ b/lib/erl_docgen/priv/docbuilder_dtd/common.refs.dtd @@ -26,15 +26,18 @@ <!ELEMENT description (%block;|quote|br|marker|warning|note)* > <!ELEMENT funcs (func)+ > -<!ELEMENT func (name+,fsummary,type?,desc?) > +<!ELEMENT func (name+,type_desc+,fsummary,type?,desc?) > <!-- ELEMENT name is defined in each ref dtd --> <!ELEMENT fsummary (#PCDATA|c|em)* > <!ELEMENT type (v,d?)+ > <!ELEMENT v (#PCDATA) > <!ELEMENT d (#PCDATA|c|em)* > -<!ELEMENT desc (%block;|quote|br|marker|warning|note)* > +<!ELEMENT desc (%block;|quote|br|marker|warning|note|anno)* > <!ELEMENT authors (aname,email)+ > <!ELEMENT aname (#PCDATA) > <!ELEMENT email (#PCDATA) > <!ELEMENT section (marker*,title,(%block;|quote|br|marker| warning|note)*) > +<!ELEMENT datatypes (datatype)+ > +<!ELEMENT datatype (name+,desc?) > +<!ELEMENT type_desc (#PCDATA) > diff --git a/lib/erl_docgen/priv/docbuilder_dtd/erlref.dtd b/lib/erl_docgen/priv/docbuilder_dtd/erlref.dtd index 21656a1446..9905086ff4 100644 --- a/lib/erl_docgen/priv/docbuilder_dtd/erlref.dtd +++ b/lib/erl_docgen/priv/docbuilder_dtd/erlref.dtd @@ -22,7 +22,7 @@ %common.refs; <!ELEMENT erlref (header,module,modulesummary,description, - (section|funcs)*,authors?) > + (section|funcs|datatypes)*,authors?) > <!ELEMENT module (#PCDATA) > <!ELEMENT modulesummary (#PCDATA) > diff --git a/lib/erl_docgen/priv/dtd_man_entities/xhtml-lat1.ent b/lib/erl_docgen/priv/dtd_man_entities/xhtml-lat1.ent index 3df9970a43..7a07e2c406 100644 --- a/lib/erl_docgen/priv/dtd_man_entities/xhtml-lat1.ent +++ b/lib/erl_docgen/priv/dtd_man_entities/xhtml-lat1.ent @@ -21,26 +21,26 @@ <!ENTITY sect "§"> <!-- section sign, U+00A7 ISOnum --> <!ENTITY uml "¨"> <!-- diaeresis = spacing diaeresis, U+00A8 ISOdia --> -<!ENTITY copy "©"> <!-- copyright sign, U+00A9 ISOnum --> +<!ENTITY copy "(C)"> <!-- copyright sign, U+00A9 ISOnum --> <!ENTITY ordf "ª"> <!-- feminine ordinal indicator, U+00AA ISOnum --> -<!ENTITY laquo "«"> <!-- left-pointing double angle quotation mark - = left pointing guillemet, U+00AB ISOnum --> +<!ENTITY laquo """> <!-- left-pointing double angle quotation mark + = left pointing guillemetn = " in man pages, U+00AB ISOnum --> <!ENTITY not "¬"> <!-- not sign = discretionary hyphen, U+00AC ISOnum --> -<!ENTITY shy "­"> <!-- soft hyphen = discretionary hyphen, +<!ENTITY shy ""> <!-- soft hyphen = discretionary hyphen, U+00AD ISOnum --> -<!ENTITY reg "®"> <!-- registered sign = registered trade mark sign, +<!ENTITY reg "(R)"> <!-- registered sign = registered trade mark sign, U+00AE ISOnum --> <!ENTITY macr "¯"> <!-- macron = spacing macron = overline = APL overbar, U+00AF ISOdia --> <!ENTITY deg "°"> <!-- degree sign, U+00B0 ISOnum --> -<!ENTITY plusmn "±"> <!-- plus-minus sign = plus-or-minus sign, +<!ENTITY plusmn "+/-"> <!-- plus-minus sign = plus-or-minus sign, U+00B1 ISOnum --> <!ENTITY sup2 "²"> <!-- superscript two = superscript digit two = squared, U+00B2 ISOnum --> <!ENTITY sup3 "³"> <!-- superscript three = superscript digit three = cubed, U+00B3 ISOnum --> -<!ENTITY acute "´"> <!-- acute accent = spacing acute, +<!ENTITY acute "'"> <!-- acute accent = spacing acute, U+00B4 ISOdia --> <!ENTITY micro "µ"> <!-- micro sign, U+00B5 ISOnum --> <!ENTITY para "¶"> <!-- pilcrow sign = paragraph sign, @@ -62,134 +62,134 @@ = fraction three quarters, U+00BE ISOnum --> <!ENTITY iquest "¿"> <!-- inverted question mark = turned question mark, U+00BF ISOnum --> -<!ENTITY Agrave "À"> <!-- latin capital letter A with grave +<!ENTITY Agrave "A"> <!-- latin capital letter A with grave = latin capital letter A grave, U+00C0 ISOlat1 --> -<!ENTITY Aacute "Á"> <!-- latin capital letter A with acute, +<!ENTITY Aacute "A"> <!-- latin capital letter A with acute, U+00C1 ISOlat1 --> -<!ENTITY Acirc "Â"> <!-- latin capital letter A with circumflex, +<!ENTITY Acirc "A"> <!-- latin capital letter A with circumflex, U+00C2 ISOlat1 --> -<!ENTITY Atilde "Ã"> <!-- latin capital letter A with tilde, +<!ENTITY Atilde "A"> <!-- latin capital letter A with tilde, U+00C3 ISOlat1 --> -<!ENTITY Auml "Ä"> <!-- latin capital letter A with diaeresis, +<!ENTITY Auml "A"> <!-- latin capital letter A with diaeresis, U+00C4 ISOlat1 --> -<!ENTITY Aring "Å"> <!-- latin capital letter A with ring above +<!ENTITY Aring "A"> <!-- latin capital letter A with ring above = latin capital letter A ring, U+00C5 ISOlat1 --> -<!ENTITY AElig "Æ"> <!-- latin capital letter AE +<!ENTITY AElig "AE"> <!-- latin capital letter AE = latin capital ligature AE, U+00C6 ISOlat1 --> -<!ENTITY Ccedil "Ç"> <!-- latin capital letter C with cedilla, +<!ENTITY Ccedil "C"> <!-- latin capital letter C with cedilla, U+00C7 ISOlat1 --> -<!ENTITY Egrave "È"> <!-- latin capital letter E with grave, +<!ENTITY Egrave "E"> <!-- latin capital letter E with grave, U+00C8 ISOlat1 --> -<!ENTITY Eacute "É"> <!-- latin capital letter E with acute, +<!ENTITY Eacute "E"> <!-- latin capital letter E with acute, U+00C9 ISOlat1 --> -<!ENTITY Ecirc "Ê"> <!-- latin capital letter E with circumflex, +<!ENTITY Ecirc "E"> <!-- latin capital letter E with circumflex, U+00CA ISOlat1 --> -<!ENTITY Euml "Ë"> <!-- latin capital letter E with diaeresis, +<!ENTITY Euml "E"> <!-- latin capital letter E with diaeresis, U+00CB ISOlat1 --> -<!ENTITY Igrave "Ì"> <!-- latin capital letter I with grave, +<!ENTITY Igrave "I"> <!-- latin capital letter I with grave, U+00CC ISOlat1 --> -<!ENTITY Iacute "Í"> <!-- latin capital letter I with acute, +<!ENTITY Iacute "I"> <!-- latin capital letter I with acute, U+00CD ISOlat1 --> -<!ENTITY Icirc "Î"> <!-- latin capital letter I with circumflex, +<!ENTITY Icirc "I"> <!-- latin capital letter I with circumflex, U+00CE ISOlat1 --> -<!ENTITY Iuml "Ï"> <!-- latin capital letter I with diaeresis, +<!ENTITY Iuml "I"> <!-- latin capital letter I with diaeresis, U+00CF ISOlat1 --> <!ENTITY ETH "Ð"> <!-- latin capital letter ETH, U+00D0 ISOlat1 --> -<!ENTITY Ntilde "Ñ"> <!-- latin capital letter N with tilde, +<!ENTITY Ntilde "N"> <!-- latin capital letter N with tilde, U+00D1 ISOlat1 --> -<!ENTITY Ograve "Ò"> <!-- latin capital letter O with grave, +<!ENTITY Ograve "O"> <!-- latin capital letter O with grave, U+00D2 ISOlat1 --> -<!ENTITY Oacute "Ó"> <!-- latin capital letter O with acute, +<!ENTITY Oacute "O"> <!-- latin capital letter O with acute, U+00D3 ISOlat1 --> -<!ENTITY Ocirc "Ô"> <!-- latin capital letter O with circumflex, +<!ENTITY Ocirc "O"> <!-- latin capital letter O with circumflex, U+00D4 ISOlat1 --> -<!ENTITY Otilde "Õ"> <!-- latin capital letter O with tilde, +<!ENTITY Otilde "O"> <!-- latin capital letter O with tilde, U+00D5 ISOlat1 --> -<!ENTITY Ouml "Ö"> <!-- latin capital letter O with diaeresis, +<!ENTITY Ouml "O"> <!-- latin capital letter O with diaeresis, U+00D6 ISOlat1 --> -<!ENTITY times "×"> <!-- multiplication sign, U+00D7 ISOnum --> +<!ENTITY times "x"> <!-- multiplication sign, U+00D7 ISOnum --> <!ENTITY Oslash "Ø"> <!-- latin capital letter O with stroke = latin capital letter O slash, U+00D8 ISOlat1 --> -<!ENTITY Ugrave "Ù"> <!-- latin capital letter U with grave, +<!ENTITY Ugrave "U"> <!-- latin capital letter U with grave, U+00D9 ISOlat1 --> -<!ENTITY Uacute "Ú"> <!-- latin capital letter U with acute, +<!ENTITY Uacute "U"> <!-- latin capital letter U with acute, U+00DA ISOlat1 --> -<!ENTITY Ucirc "Û"> <!-- latin capital letter U with circumflex, +<!ENTITY Ucirc "U"> <!-- latin capital letter U with circumflex, U+00DB ISOlat1 --> -<!ENTITY Uuml "Ü"> <!-- latin capital letter U with diaeresis, +<!ENTITY Uuml "U"> <!-- latin capital letter U with diaeresis, U+00DC ISOlat1 --> -<!ENTITY Yacute "Ý"> <!-- latin capital letter Y with acute, +<!ENTITY Yacute "Y"> <!-- latin capital letter Y with acute, U+00DD ISOlat1 --> <!ENTITY THORN "Þ"> <!-- latin capital letter THORN, U+00DE ISOlat1 --> <!ENTITY szlig "ß"> <!-- latin small letter sharp s = ess-zed, U+00DF ISOlat1 --> -<!ENTITY agrave "à"> <!-- latin small letter a with grave +<!ENTITY agrave "a"> <!-- latin small letter a with grave = latin small letter a grave, U+00E0 ISOlat1 --> -<!ENTITY aacute "á"> <!-- latin small letter a with acute, +<!ENTITY aacute "a"> <!-- latin small letter a with acute, U+00E1 ISOlat1 --> -<!ENTITY acirc "â"> <!-- latin small letter a with circumflex, +<!ENTITY acirc "a"> <!-- latin small letter a with circumflex, U+00E2 ISOlat1 --> -<!ENTITY atilde "ã"> <!-- latin small letter a with tilde, +<!ENTITY atilde "a"> <!-- latin small letter a with tilde, U+00E3 ISOlat1 --> -<!ENTITY auml "ä"> <!-- latin small letter a with diaeresis, +<!ENTITY auml "a"> <!-- latin small letter a with diaeresis, U+00E4 ISOlat1 --> -<!ENTITY aring "å"> <!-- latin small letter a with ring above +<!ENTITY aring "a"> <!-- latin small letter a with ring above = latin small letter a ring, U+00E5 ISOlat1 --> -<!ENTITY aelig "æ"> <!-- latin small letter ae +<!ENTITY aelig "ae"> <!-- latin small letter ae = latin small ligature ae, U+00E6 ISOlat1 --> -<!ENTITY ccedil "ç"> <!-- latin small letter c with cedilla, +<!ENTITY ccedil "c"> <!-- latin small letter c with cedilla, U+00E7 ISOlat1 --> -<!ENTITY egrave "è"> <!-- latin small letter e with grave, +<!ENTITY egrave "e"> <!-- latin small letter e with grave, U+00E8 ISOlat1 --> -<!ENTITY eacute "é"> <!-- latin small letter e with acute, +<!ENTITY eacute "e"> <!-- latin small letter e with acute, U+00E9 ISOlat1 --> -<!ENTITY ecirc "ê"> <!-- latin small letter e with circumflex, +<!ENTITY ecirc "e"> <!-- latin small letter e with circumflex, U+00EA ISOlat1 --> -<!ENTITY euml "ë"> <!-- latin small letter e with diaeresis, +<!ENTITY euml "e"> <!-- latin small letter e with diaeresis, U+00EB ISOlat1 --> -<!ENTITY igrave "ì"> <!-- latin small letter i with grave, +<!ENTITY igrave "i"> <!-- latin small letter i with grave, U+00EC ISOlat1 --> -<!ENTITY iacute "í"> <!-- latin small letter i with acute, +<!ENTITY iacute "i"> <!-- latin small letter i with acute, U+00ED ISOlat1 --> -<!ENTITY icirc "î"> <!-- latin small letter i with circumflex, +<!ENTITY icirc "i"> <!-- latin small letter i with circumflex, U+00EE ISOlat1 --> -<!ENTITY iuml "ï"> <!-- latin small letter i with diaeresis, +<!ENTITY iuml "i"> <!-- latin small letter i with diaeresis, U+00EF ISOlat1 --> <!ENTITY eth "ð"> <!-- latin small letter eth, U+00F0 ISOlat1 --> -<!ENTITY ntilde "ñ"> <!-- latin small letter n with tilde, +<!ENTITY ntilde "n"> <!-- latin small letter n with tilde, U+00F1 ISOlat1 --> -<!ENTITY ograve "ò"> <!-- latin small letter o with grave, +<!ENTITY ograve "o"> <!-- latin small letter o with grave, U+00F2 ISOlat1 --> -<!ENTITY oacute "ó"> <!-- latin small letter o with acute, +<!ENTITY oacute "o"> <!-- latin small letter o with acute, U+00F3 ISOlat1 --> -<!ENTITY ocirc "ô"> <!-- latin small letter o with circumflex, +<!ENTITY ocirc "o"> <!-- latin small letter o with circumflex, U+00F4 ISOlat1 --> -<!ENTITY otilde "õ"> <!-- latin small letter o with tilde, +<!ENTITY otilde "o"> <!-- latin small letter o with tilde, U+00F5 ISOlat1 --> -<!ENTITY ouml "ö"> <!-- latin small letter o with diaeresis, +<!ENTITY ouml "o"> <!-- latin small letter o with diaeresis, U+00F6 ISOlat1 --> <!ENTITY divide "÷"> <!-- division sign, U+00F7 ISOnum --> -<!ENTITY oslash "ø"> <!-- latin small letter o with stroke, +<!ENTITY oslash "o"> <!-- latin small letter o with stroke, = latin small letter o slash, U+00F8 ISOlat1 --> -<!ENTITY ugrave "ù"> <!-- latin small letter u with grave, +<!ENTITY ugrave "u"> <!-- latin small letter u with grave, U+00F9 ISOlat1 --> -<!ENTITY uacute "ú"> <!-- latin small letter u with acute, +<!ENTITY uacute "u"> <!-- latin small letter u with acute, U+00FA ISOlat1 --> -<!ENTITY ucirc "û"> <!-- latin small letter u with circumflex, +<!ENTITY ucirc "u"> <!-- latin small letter u with circumflex, U+00FB ISOlat1 --> -<!ENTITY uuml "ü"> <!-- latin small letter u with diaeresis, +<!ENTITY uuml "u"> <!-- latin small letter u with diaeresis, U+00FC ISOlat1 --> -<!ENTITY yacute "ý"> <!-- latin small letter y with acute, +<!ENTITY yacute "y"> <!-- latin small letter y with acute, U+00FD ISOlat1 --> <!ENTITY thorn "þ"> <!-- latin small letter thorn with, U+00FE ISOlat1 --> -<!ENTITY yuml "ÿ"> <!-- latin small letter y with diaeresis, +<!ENTITY yuml "y"> <!-- latin small letter y with diaeresis, U+00FF ISOlat1 --> diff --git a/lib/erl_docgen/priv/xsl/db_html.xsl b/lib/erl_docgen/priv/xsl/db_html.xsl index bba0f97645..732560e303 100644 --- a/lib/erl_docgen/priv/xsl/db_html.xsl +++ b/lib/erl_docgen/priv/xsl/db_html.xsl @@ -1,5 +1,5 @@ <?xml version="1.0" encoding="utf-8"?> -<!-- +<!-- # # %CopyrightBegin% # @@ -17,15 +17,315 @@ # under the License. # # %CopyrightEnd% - + --> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" - xmlns:fn="http://www.w3.org/2005/02/xpath-functions"> + xmlns:fn="http://www.w3.org/2005/02/xpath-functions"> <xsl:include href="db_html_params.xsl"/> + <!-- Start of Dialyzer type/spec tags. + See also the template matching "name" and the template "menu.funcs" + --> + + <xsl:param name="specs_file" select="''"/> + <xsl:variable name="i" select="document($specs_file)"></xsl:variable> + + <xsl:param name="mod2app_file" select="''"/> + <xsl:variable name="m2a" select="document($mod2app_file)"></xsl:variable> + <xsl:key name="mod2app" match="module" use="@name"/> + + <xsl:template name="err"> + <xsl:param name="m"/> + <xsl:param name="n"/> + <xsl:param name="a"/> + <xsl:param name="s"/> + <xsl:message terminate="yes"> + Error <xsl:if test="$m != ''"><xsl:value-of select ="$m"/>:</xsl:if> + <xsl:value-of + select="$n"/>/<xsl:value-of + select="$a"/>: <xsl:value-of select="$s"/> + </xsl:message> + </xsl:template> + + <xsl:template name="spec_name"> + <xsl:variable name="curModule" select="ancestor::erlref/module"/> + <xsl:variable name="mod" select="@mod"/> + <xsl:variable name="name" select="@name"/> + <xsl:variable name="arity" select="@arity"/> + <xsl:variable name="clause" select="@clause"/> + <xsl:variable name="spec0" select= + "$i/specs/module[@name=$curModule]/spec + [name=$name and arity=$arity + and (string-length($mod) = 0 or module = $mod)]"/> + <xsl:variable name="spec" select="$spec0[string-length($clause) = 0 + or position() = $clause]"/> + <xsl:if test="count($spec) = 0"> + <xsl:call-template name="err"> + <xsl:with-param name="m" select="$mod"/> + <xsl:with-param name="n" select="$name"/> + <xsl:with-param name="a" select="$arity"/> + <xsl:with-param name="s">unknown spec</xsl:with-param> + </xsl:call-template> + </xsl:if> + + <xsl:variable name="arity_clause"> + <xsl:choose> + <xsl:when test="string-length(@clause) > 0"> + <xsl:value-of select="@arity"/>/<xsl:value-of select="@clause"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="@arity"/> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + + <xsl:choose> + <xsl:when test="ancestor::cref"> + <xsl:message terminate="yes"> + Error: did not expect a 'name' tag with name/arity attributes here! + </xsl:message> + </xsl:when> + <xsl:when test="ancestor::erlref"> + <a name="{$name}-{$arity_clause}"></a> + <xsl:choose> + <xsl:when test="string(@with_guards) = 'no'"> + <xsl:apply-templates select="$spec/contract/clause/head"/> + </xsl:when> + <xsl:otherwise> + <xsl:call-template name="contract"> + <xsl:with-param name="contract" select="$spec/contract"/> + </xsl:call-template> + </xsl:otherwise> + </xsl:choose> + </xsl:when> + </xsl:choose> + </xsl:template> + + <xsl:template name="contract"> + <xsl:param name="contract"/> + <xsl:call-template name="clause"> + <xsl:with-param name="clause" select="$contract/clause"/> + </xsl:call-template> + </xsl:template> + + <xsl:template name="clause"> + <xsl:param name="clause"/> + <xsl:variable name="type_desc" select="../type_desc"/> + <xsl:for-each select="$clause"> + <xsl:apply-templates select="head"/> + <xsl:if test="count(guard) > 0"> + <xsl:call-template name="guard"> + <xsl:with-param name="guard" select="guard"/> + <xsl:with-param name="type_desc" select="$type_desc"/> + </xsl:call-template> + </xsl:if> + </xsl:for-each> + </xsl:template> + + <xsl:template match="head"> + <span class="bold_code"> + <xsl:apply-templates/> + </span> + <br/> + </xsl:template> + + <xsl:template name="guard"> + <xsl:param name="guard"/> + <xsl:param name="type_desc"/> + <div class="REFBODY"><p>Types:</p> + <xsl:call-template name="subtype"> + <xsl:with-param name="subtype" select="$guard/subtype"/> + <xsl:with-param name="type_desc" select="$type_desc"/> + </xsl:call-template> + </div> + </xsl:template> + + <xsl:template name="subtype"> + <xsl:param name="subtype"/> + <xsl:param name="type_desc"/> + <xsl:for-each select="$subtype"> + <xsl:variable name="tname" select="typename"/> + <xsl:variable name="tdesc" select="$type_desc[@name = $tname]"/> + <div class="REFTYPES"> + <span class="bold_code"> + <xsl:apply-templates select="string"/> + </span> + </div> + <xsl:apply-templates select="$type_desc[@name = $tname]"/> + </xsl:for-each> + </xsl:template> + + <!-- Note: <type_desc> has not been implemented for data types. --> + + <!-- Similar to <d> --> + <xsl:template match="type_desc"> + <div class="REFBODY"> + <xsl:apply-templates/> + </div> + </xsl:template> + + <!-- This is for debugging. All modules! --> + <xsl:template match="all_etypes"> + <xsl:for-each select= "$i//type"> + <pre> + <span class="bold_code"> + <xsl:apply-templates select="typedecl"/> + </span><xsl:text> +</xsl:text> + </pre> + </xsl:for-each> + </xsl:template> + + <!-- Datatypes --> + <xsl:template match="datatypes"> + <h3> + <xsl:text>DATA TYPES</xsl:text> + </h3> + <xsl:apply-templates/> + </xsl:template> + + <!-- Datatype --> + <xsl:template match="datatype"> + <p><xsl:apply-templates select="name"/></p> + <xsl:apply-templates select="desc"/> + </xsl:template> + + <xsl:template match="typehead"> + <span class="bold_code"> + <xsl:apply-templates/> + </span><br/> + </xsl:template> + + <!-- local_defs --> + <xsl:template match="local_defs"> + <div class="REFBODY"> + <xsl:apply-templates> + </xsl:apply-templates> + </div> + </xsl:template> + + <xsl:template match="local_def"> + <div class="REFTYPES"> + <span class="bold_code"> + <xsl:apply-templates/> + </span> + </div> + </xsl:template> + + <xsl:template name="type_name"> + <xsl:variable name="curModule" select="ancestor::erlref/module"/> + <xsl:variable name="mod" select="@mod"/> + <xsl:variable name="name" select="@name"/> + <xsl:variable name="n_vars"> + <xsl:choose> + <xsl:when test="string-length(@n_vars) > 0"> + <xsl:value-of select="@n_vars"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="0"/> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + + <xsl:choose> + <xsl:when test="string-length($name) > 0"> + <xsl:variable name="type" select= + "$i/specs/module[@name=$curModule]/type + [name=$name and n_vars=$n_vars + and (string-length($mod) = 0 or module = $mod)]"/> + + <xsl:if test="count($type) != 1"> + <xsl:call-template name="err"> + <xsl:with-param name="m" select="$mod"/> + <xsl:with-param name="n" select="$name"/> + <xsl:with-param name="a" select="$n_vars"/> + <xsl:with-param name="s">unknown type</xsl:with-param> + </xsl:call-template> + </xsl:if> + <xsl:apply-templates select="$type/typedecl"/> + </xsl:when> + <xsl:otherwise> + <span class="bold_code"> + <xsl:value-of select="."/> + </span> + </xsl:otherwise> + </xsl:choose> + </xsl:template> + + <!-- Used both in <datatype> and in <func>! --> + <xsl:template match="anno"> + <xsl:variable name="curModule" select="ancestor::erlref/module"/> + <xsl:variable name="anno" select="normalize-space(text())"/> + <xsl:variable name="namespec" + select="ancestor::desc/preceding-sibling::name"/> + <xsl:if test="count($namespec) = 0 and string-length($specs_file) > 0"> + <xsl:call-template name="err"> + <xsl:with-param name="s">cannot find 'name' (<xsl:value-of select="$anno"/>) + </xsl:with-param> + </xsl:call-template> + </xsl:if> + + <xsl:variable name="mod" select="$namespec/@mod"/> + <xsl:variable name="name" select="$namespec/@name"/> + <xsl:variable name="arity" select="$namespec/@arity"/> + <xsl:variable name="clause" select="$namespec/@clause"/> + <xsl:variable name="tmp_n_vars" select="$namespec/@n_vars"/> + <xsl:variable name="n_vars"> + <xsl:choose> + <xsl:when test="string-length($tmp_n_vars) > 0"> + <xsl:value-of select="$tmp_n_vars"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="0"/> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + <xsl:variable name="spec0" select= + "$i/specs/module[@name=$curModule]/spec + [name=$name and arity=$arity + and (string-length($mod) = 0 or module = $mod)]"/> + <xsl:variable name="spec_annos" select= + "$spec0[string-length($clause) = 0 + or position() = $clause]/anno[.=$anno]"/> + <xsl:variable name="type_annos" select= + "$i/specs/module[@name=$curModule]/type + [name=$name and n_vars=$n_vars + and (string-length($mod) = 0 or module = $mod)]/anno[.=$anno]"/> + + <xsl:if test="count($spec_annos) = 0 + and count($type_annos) = 0 + and string-length($specs_file) > 0"> + <xsl:variable name="n"> + <xsl:choose> + <xsl:when test="string-length($arity) = 0"> + <xsl:value-of select="$n_vars"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="$arity"/> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + <xsl:call-template name="err"> + <xsl:with-param name="m" select="$mod"/> + <xsl:with-param name="n" select="$name"/> + <xsl:with-param name="a" select="$n"/> + <xsl:with-param name="s">unknown annotation <xsl:value-of select="$anno"/> + </xsl:with-param> + </xsl:call-template> + </xsl:if> + <xsl:value-of select="$anno"/> + </xsl:template> + + <!-- Used for indentation of formatted types and specs --> + <xsl:template match="nbsp"> + <xsl:text> </xsl:text> + </xsl:template> + + <!-- End of Dialyzer type/spec tags --> + <!-- Page layout --> <xsl:template name="pagelayout"> <xsl:param name="chapnum"/> @@ -36,19 +336,19 @@ <title>Erlang -- <xsl:value-of select="header/title"/></title> </head> <body bgcolor="white" text="#000000" link="#0000ff" vlink="#ff00ff" alink="#ff0000"> - + <div id="container"> <script id="js" type="text/javascript" language="JavaScript" src="{$topdocdir}/js/flipmenu/flipmenu.js"/> <script id="js2" type="text/javascript" src="{$topdocdir}/js/erlresolvelinks.js"></script> <script language="JavaScript" type="text/javascript"> <xsl:text disable-output-escaping="yes"><![CDATA[ - <!-- + <!-- function getWinHeight() { var myHeight = 0; if( typeof( window.innerHeight ) == 'number' ) { //Non-IE myHeight = window.innerHeight; - } else if( document.documentElement && ( document.documentElement.clientWidth || + } else if( document.documentElement && ( document.documentElement.clientWidth || document.documentElement.clientHeight ) ) { //IE 6+ in 'standards compliant mode' myHeight = document.documentElement.clientHeight; @@ -56,7 +356,7 @@ //IE 4 compatible myHeight = document.body.clientHeight; } - return myHeight; + return myHeight; } function setscrollpos() { @@ -64,16 +364,16 @@ document.getElementById("leftnav").scrollTop = objf.offsetTop - getWinHeight()/2; } - function addEvent(obj, evType, fn){ - if (obj.addEventListener){ - obj.addEventListener(evType, fn, true); - return true; - } else if (obj.attachEvent){ - var r = obj.attachEvent("on"+evType, fn); - return r; - } else { - return false; - } + function addEvent(obj, evType, fn){ + if (obj.addEventListener){ + obj.addEventListener(evType, fn, true); + return true; + } else if (obj.attachEvent){ + var r = obj.attachEvent("on"+evType, fn); + return r; + } else { + return false; + } } addEvent(window, 'load', setscrollpos); @@ -85,7 +385,7 @@ <xsl:with-param name="chapnum" select="$chapnum"/> <xsl:with-param name="curModule" select="$curModule"/> </xsl:call-template> - + <div id="content"> <div class="innertube"> @@ -124,17 +424,17 @@ <xsl:if test="$lname = 'releasenotes'"> <!-- .../part --> <xsl:call-template name="releasenotes.content" /> - </xsl:if> + </xsl:if> <xsl:if test="$lname = 'part'"> <!-- .../part --> <xsl:call-template name="part.content" /> - </xsl:if> + </xsl:if> <xsl:if test="$lname = 'chapter'"> <!-- .../part/chapter --> <xsl:call-template name="chapter.content"> <xsl:with-param name="chapnum" select="$chapnum"/> </xsl:call-template> - </xsl:if> + </xsl:if> <xsl:if test="$lname = 'application'"> <!-- .../application --> <xsl:call-template name="app.content" /> @@ -178,37 +478,37 @@ <small> <xsl:if test="boolean(/book/parts/part)"> <a href="users_guide.html">User's Guide</a><br/> - </xsl:if> + </xsl:if> <xsl:if test="boolean(/book/applications)"> <a href="index.html">Reference Manual</a><br/> - </xsl:if> + </xsl:if> <xsl:if test="boolean(/book/releasenotes)"> <a href="release_notes.html">Release Notes</a><br/> - </xsl:if> + </xsl:if> <a href="{$pdfdir}/{$appname}-{$appver}.pdf">PDF</a><br/> <a href="{$topdocdir}/index.html">Top</a> </small> </xsl:template> - + <xsl:template name="menu_middle"> <!-- small> <xsl:choose> <xsl:when test="ancestor::parts"> <a href="users_guide_bibliography.html">Bibliography</a><br/> <a href="users_guide_glossary.html">Glossary</a><br/> - </xsl:when> - <xsl:when test="ancestor::applications"> + </xsl:when> + <xsl:when test="ancestor::applications"> <a href="ref_man_bibliography.html">Bibliography</a><br/> <a href="ref_man_glossary.html">Glossary</a><br/> - </xsl:when> + </xsl:when> </xsl:choose> </small --> <br/> <a href="javascript:openAllFlips()">Expand All</a><br/> <a href="javascript:closeAllFlips()">Contract All</a> - </xsl:template> - + </xsl:template> + <!-- Book --> <xsl:template match="/book"> @@ -243,7 +543,7 @@ <!-- Chapter/Section --> <xsl:template match="chapter/section"> - <xsl:param name="chapnum"/> + <xsl:param name="chapnum"/> <h3> <a name="{generate-id(title)}"> <xsl:value-of select="$chapnum"/>.<xsl:number/>  @@ -302,7 +602,7 @@ <!-- Lists --> - + <xsl:template match="list"> <xsl:param name="chapnum"/> <ul> @@ -330,7 +630,7 @@ </xsl:apply-templates> </dl> </xsl:template> - + <xsl:template match="taglist/tag"> <xsl:param name="chapnum"/> <dt> @@ -377,7 +677,7 @@ </xsl:apply-templates> </p> </div> - </div> + </div> </xsl:template> <!-- Paragraph --> @@ -402,7 +702,7 @@ </xsl:template> <xsl:template match="em"> - <strong><xsl:apply-templates/></strong> + <strong><xsl:apply-templates/></strong> </xsl:template> <!-- Code --> @@ -507,7 +807,7 @@ <!-- Part --> <xsl:template match="part"> <!-- Generate Glossary for Users Guide --> - <!--xsl:call-template name="glossary"> + <!--xsl:call-template name="glossary"> <xsl:with-param name="type">users_guide</xsl:with-param> </xsl:call-template--> @@ -530,9 +830,9 @@ <center><h4>Version <xsl:value-of select="$appver"/></h4></center> <center><h4><xsl:value-of select="$gendate"/></h4></center> - + <xsl:apply-templates select="chapter"/> - + </xsl:template> <!-- Menu.ug --> @@ -565,10 +865,10 @@ </xsl:call-template> </ul> </div> - </div> + </div> </xsl:template> - - + + <xsl:template name="menu.chapter"> <xsl:param name="entries"/> <xsl:param name="chapnum"/> @@ -596,7 +896,7 @@ <a href="{$chapter_file}.html"> Top of chapter </a> - </li> + </li> <xsl:call-template name="menu.section"> <xsl:with-param name="entries" select="section[title]"/> @@ -623,7 +923,7 @@ <!-- Chapter (if top tag)--> <xsl:template match="/chapter"> - <xsl:document href="{substring-before(header/file, '.xml')}.html" method="html" encoding="UTF-8" indent="yes" + <xsl:document href="{substring-before(header/file, '.xml')}.html" method="html" encoding="UTF-8" indent="yes" doctype-public="-//W3C//DTD HTML 4.01 Transitional//EN"> <xsl:call-template name="pagelayout"> @@ -635,7 +935,7 @@ <!-- Chapter --> <xsl:template match="chapter"> - <xsl:document href="{substring-before(header/file, '.xml')}.html" method="html" encoding="UTF-8" indent="yes" + <xsl:document href="{substring-before(header/file, '.xml')}.html" method="html" encoding="UTF-8" indent="yes" doctype-public="-//W3C//DTD HTML 4.01 Transitional//EN"> <xsl:call-template name="pagelayout"> @@ -670,7 +970,7 @@ <xsl:template match="application"> <!-- Generate Glossary for Ref. Manual --> - <!--xsl:call-template name="glossary"> + <!--xsl:call-template name="glossary"> <xsl:with-param name="type">ref_man</xsl:with-param> </xsl:call-template--> @@ -678,7 +978,7 @@ <!--xsl:call-template name="bibliography"> <xsl:with-param name="type">ref_man</xsl:with-param> </xsl:call-template--> - + <xsl:document href="{$outdir}/index.html" method="html" encoding="UTF-8" indent="yes" doctype-public="-//W3C//DTD HTML 4.01 Transitional//EN"> @@ -695,9 +995,9 @@ <center><h4>Version <xsl:value-of select="$appver"/></h4></center> <center><h4><xsl:value-of select="$gendate"/></h4></center> - + <xsl:apply-templates select="erlref|cref|comref|fileref|appref"/> - + </xsl:template> <!-- Menu.ref --> @@ -730,16 +1030,16 @@ </xsl:call-template> </ul> </div> - </div> + </div> </xsl:template> - - + + <xsl:template name="menu.ref2"> <xsl:param name="entries"/> <!--xsl:param name="genFuncMenu"/--> <xsl:param name="curModule"/> <xsl:for-each select="$entries"> - + <xsl:variable name="cval"> <xsl:choose> <xsl:when test="local-name() = 'erlref'"> @@ -767,9 +1067,9 @@ <xsl:when test="local-name() = 'fileref'">false</xsl:when> <xsl:when test="descendant::funcs">true</xsl:when> <xsl:otherwise>false</xsl:otherwise> - </xsl:choose> + </xsl:choose> </xsl:variable> - + <xsl:variable name="expanded"> <xsl:choose> <xsl:when test="$curModule = $cval">true</xsl:when> @@ -796,7 +1096,7 @@ <a href="{$link_cval}.html"> Top of manual page </a> - </li> + </li> <xsl:call-template name="menu.funcs"> <xsl:with-param name="entries" select="funcs/func/name"/> @@ -823,7 +1123,7 @@ </xsl:otherwise> </xsl:choose> </xsl:otherwise> - </xsl:choose> + </xsl:choose> </xsl:for-each> </xsl:template> @@ -831,7 +1131,7 @@ <xsl:template name="menu.funcs"> <xsl:param name="entries"/> <xsl:param name="basename"/> - + <xsl:for-each select="$entries"> <xsl:choose> @@ -840,74 +1140,97 @@ <xsl:choose> <xsl:when test="string-length($fname) > 0"> <li title="{$fname}"> - <a href="{$basename}.html#{$fname}"> + <a href="{$basename}.html#{$fname}"> <xsl:value-of select="$fname"/>() </a> - </li> + </li> </xsl:when> <xsl:otherwise> <li title="{name/nametext}"> - <a href="{$basename}.html#{name/nametext}"> + <a href="{$basename}.html#{name/nametext}"> <xsl:value-of select="nametext"/>() - </a> - </li> + </a> + </li> </xsl:otherwise> - </xsl:choose> + </xsl:choose> </xsl:when> - + <xsl:when test="ancestor::erlref"> - + <xsl:variable name="tmpstring"> <xsl:value-of select="substring-before(substring-after(., '('), '->')"/> - </xsl:variable> - + </xsl:variable> + <xsl:variable name="ustring"> <xsl:choose> <xsl:when test="string-length($tmpstring) > 0"> <xsl:call-template name="remove-paren"> <xsl:with-param name="string" select="$tmpstring"/> - </xsl:call-template> + </xsl:call-template> </xsl:when> <xsl:otherwise> <xsl:call-template name="remove-paren"> <xsl:with-param name="string" select="substring-after(., '(')"/> - </xsl:call-template> + </xsl:call-template> </xsl:otherwise> </xsl:choose> - </xsl:variable> - + </xsl:variable> + <xsl:variable name="arity"> - <xsl:call-template name="calc-arity"> - <xsl:with-param name="string" select="substring-before($ustring, ')')"/> - <xsl:with-param name="no-of-pars" select="0"/> - </xsl:call-template> - </xsl:variable> - + <xsl:choose> + <xsl:when test="string-length(@arity) > 0"> + <!-- Dialyzer spec --> + <xsl:choose> + <xsl:when test="string-length(@clause) > 0"> + <xsl:value-of select="@arity"/>/<xsl:value-of select="@clause"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="@arity"/> + </xsl:otherwise> + </xsl:choose> + </xsl:when> + <xsl:otherwise> + <xsl:call-template name="calc-arity"> + <xsl:with-param name="string" select="substring-before($ustring, ')')"/> + <xsl:with-param name="no-of-pars" select="0"/> + </xsl:call-template> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + <xsl:variable name="fname"> - <xsl:variable name="fname1"> - <xsl:value-of select="substring-before(., '(')"/> - </xsl:variable> - <xsl:variable name="fname2"> - <xsl:value-of select="substring-after($fname1, 'erlang:')"/> - </xsl:variable> <xsl:choose> - <xsl:when test="string-length($fname2) > 0"> - <xsl:value-of select="$fname2"/> + <xsl:when test="string-length(@name) > 0"> + <!-- Dialyzer spec --> + <xsl:value-of select="@name"/> </xsl:when> <xsl:otherwise> - <xsl:value-of select="$fname1"/> + <xsl:variable name="fname1"> + <xsl:value-of select="substring-before(., '(')"/> + </xsl:variable> + <xsl:variable name="fname2"> + <xsl:value-of select="substring-after($fname1, 'erlang:')"/> + </xsl:variable> + <xsl:choose> + <xsl:when test="string-length($fname2) > 0"> + <xsl:value-of select="$fname2"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="$fname1"/> + </xsl:otherwise> + </xsl:choose> </xsl:otherwise> </xsl:choose> </xsl:variable> - + <li title="{$fname}-{$arity}"> - <a href="{$basename}.html#{$fname}-{$arity}"> + <a href="{$basename}.html#{$fname}-{$arity}"> <xsl:value-of select="$fname"/>/<xsl:value-of select="$arity"/> </a> - </li> + </li> </xsl:when> </xsl:choose> - + </xsl:for-each> </xsl:template> @@ -1148,7 +1471,7 @@ <!-- Func --> <xsl:template match="func"> <xsl:param name="partnum"/> - + <p><xsl:apply-templates select="name"/></p> <xsl:apply-templates select="fsummary|type|desc"> @@ -1159,33 +1482,48 @@ <xsl:template match="name"> + <xsl:choose> + <!-- @arity is mandatory when referring to a specification --> + <xsl:when test="string-length(@arity) > 0"> + <xsl:call-template name="spec_name"/> + </xsl:when> + <xsl:when test="ancestor::datatype"> + <xsl:call-template name="type_name"/> + </xsl:when> + <xsl:otherwise> + <xsl:call-template name="name"/> + </xsl:otherwise> + </xsl:choose> + </xsl:template> + + <xsl:template name="name"> <xsl:variable name="tmpstring"> <xsl:value-of select="substring-before(substring-after(., '('), '->')"/> - </xsl:variable> + </xsl:variable> <xsl:variable name="ustring"> <xsl:choose> <xsl:when test="string-length($tmpstring) > 0"> <xsl:call-template name="remove-paren"> <xsl:with-param name="string" select="$tmpstring"/> - </xsl:call-template> + </xsl:call-template> </xsl:when> <xsl:otherwise> <xsl:call-template name="remove-paren"> <xsl:with-param name="string" select="substring-after(., '(')"/> - </xsl:call-template> + </xsl:call-template> </xsl:otherwise> </xsl:choose> - </xsl:variable> - + </xsl:variable> + <xsl:variable name="arity"> <xsl:call-template name="calc-arity"> <xsl:with-param name="string" select="substring-before($ustring, ')')"/> - <xsl:with-param name="no-of-pars" select="0"/> + <xsl:with-param name="no-of-pars" select="0"/> </xsl:call-template> - </xsl:variable> - + </xsl:variable> + <xsl:choose> <xsl:when test="ancestor::cref"> <a name="{substring-before(nametext, '(')}"><span class="bold_code"><xsl:value-of select="ret"/><xsl:text> </xsl:text><xsl:value-of select="nametext"/></span></a><br/> @@ -1199,7 +1537,7 @@ <xsl:value-of select="substring-after($fname1, 'erlang:')"/> </xsl:variable> <xsl:choose> - <xsl:when test="string-length($fname2) > 0"> + <xsl:when test="string-length($fname2) > 0"> <xsl:value-of select="$fname2"/> </xsl:when> <xsl:otherwise> @@ -1213,21 +1551,20 @@ <span class="bold_code"><xsl:value-of select="."/></span> </xsl:otherwise> </xsl:choose> - - </xsl:template> + </xsl:template> <!-- Type --> <xsl:template match="type"> <xsl:param name="partnum"/> - <div class="REFBODY"><p>Types:</p> + <div class="REFBODY"><p>Types:</p> <xsl:apply-templates> <xsl:with-param name="partnum" select="$partnum"/> </xsl:apply-templates> </div> - + </xsl:template> @@ -1286,16 +1623,37 @@ <xsl:variable name="modulepart"><xsl:value-of select="substring-before($filepart, ':')"/></xsl:variable> <xsl:choose> <xsl:when test="string-length($modulepart) > 0"> - <xsl:variable name="filepart1"><xsl:value-of select="substring-after($filepart, ':')"/></xsl:variable> + <xsl:variable name="filepart1"><xsl:value-of select="substring-after($filepart, ':')"/></xsl:variable> <span class="bold_code"><a href="javascript:erlhref('{$topdocdir}/../','{$modulepart}','{$filepart1}.html#{$linkpart}');"><xsl:apply-templates/></a></span> </xsl:when> <xsl:otherwise> <xsl:choose> + <!-- Dialyzer seealso (the application is unknown) --> + <xsl:when test="string-length($specs_file) > 0 + and count($i/specs/module[@name=$filepart]) = 0"> + <!-- Deemed to slow; use key() instead + <xsl:variable name="app" + select="$m2a/mod2app/module[@name=$filepart]"/> + --> + <xsl:variable name="reftext" select="text()"/> + <xsl:for-each select="$m2a"> + <xsl:variable name="app" select="key('mod2app', $filepart)"/> + <xsl:choose> + <xsl:when test="string-length($app) > 0"> + <span class="bold_code"><a href="javascript:erlhref('{$topdocdir}/../','{$app}','{$filepart}.html');"><xsl:value-of select="$reftext"/></a></span> + </xsl:when> + <xsl:otherwise> + <!-- Unknown application; no link --> + <xsl:value-of select="$reftext"/> + </xsl:otherwise> + </xsl:choose> + </xsl:for-each> + </xsl:when> <xsl:when test="string-length($linkpart) > 0"> <span class="bold_code"><a href="{$filepart}.html#{$linkpart}"><xsl:apply-templates/></a></span> </xsl:when> - <xsl:otherwise> - <span class="bold_code"><a href="{$filepart}.html"><xsl:apply-templates/></a></span> + <xsl:otherwise> + <span class="bold_code"><a href="{$filepart}.html"><xsl:apply-templates/></a></span> </xsl:otherwise> </xsl:choose> </xsl:otherwise> @@ -1308,16 +1666,16 @@ </xsl:when> <xsl:otherwise> <xsl:variable name="modulepart"><xsl:value-of select="substring-before(@marker, ':')"/></xsl:variable> - + <xsl:choose> <xsl:when test="string-length($modulepart) > 0"> - <xsl:variable name="filepart1"><xsl:value-of select="substring-after(@marker, ':')"/></xsl:variable> + <xsl:variable name="filepart1"><xsl:value-of select="substring-after(@marker, ':')"/></xsl:variable> <span class="bold_code"><a href="javascript:erlhref('{$topdocdir}/../','{$modulepart}','{$filepart1}.html');"><xsl:apply-templates/></a></span> </xsl:when> <xsl:otherwise> - <span class="bold_code"><a href="{@marker}.html"><xsl:apply-templates/></a></span> + <span class="bold_code"><a href="{@marker}.html"><xsl:apply-templates/></a></span> </xsl:otherwise> - </xsl:choose> + </xsl:choose> </xsl:otherwise> </xsl:choose> </xsl:otherwise> @@ -1342,16 +1700,16 @@ <xsl:choose> <xsl:when test="ancestor::parts"> <a href="users_guide_glossary.html#{@id}"><xsl:value-of select="@id"/></a> - </xsl:when> - <xsl:when test="ancestor::applications"> + </xsl:when> + <xsl:when test="ancestor::applications"> <a href="ref_man_glossary.html#{@id}"><xsl:value-of select="@id"/></a> - </xsl:when> + </xsl:when> </xsl:choose> </xsl:when> <xsl:otherwise> <a href="{$topdocdir}/glossary.html#{@id}"><xsl:value-of select="@id"/></a> </xsl:otherwise> - </xsl:choose --> + </xsl:choose --> </xsl:template> <xsl:template match="cite"> @@ -1375,9 +1733,9 @@ <center><h4>Version <xsl:value-of select="$appver"/></h4></center> <center><h4><xsl:value-of select="$gendate"/></h4></center> - + <xsl:apply-templates select="chapter"/> - + </xsl:template> <!-- Menu.rn --> @@ -1410,7 +1768,7 @@ </xsl:call-template> </ul> </div> - </div> + </div> </xsl:template> <!-- Glossary --> @@ -1423,14 +1781,14 @@ <title>Erlang Documentation -- <xsl:value-of select="header/title"/></title> </head> <body bgcolor="white" text="#000000" link="#0000ff" vlink="#ff00ff" alink="#ff0000"> - + <div id="container"> <script id="js" type="text/javascript" language="JavaScript" src="{$topdocdir}/js/flipmenu/flipmenu.js"/> <script id="js2" type="text/javascript" src="{$topdocdir}/js/erlresolvelinks.js"></script> <!-- Generate menu --> <xsl:call-template name="menu"/> - + <div id="content"> <div class="innertube"> <h1>Glossary</h1> @@ -1478,14 +1836,14 @@ <title>Erlang Documentation -- <xsl:value-of select="header/title"/></title> </head> <body bgcolor="white" text="#000000" link="#0000ff" vlink="#ff00ff" alink="#ff0000"> - + <div id="container"> <script id="js" type="text/javascript" language="JavaScript" src="{$topdocdir}/js/flipmenu/flipmenu.js"/> <script id="js2" type="text/javascript" src="{$topdocdir}/js/erlresolvelinks.js"></script> <!-- Generate menu --> <xsl:call-template name="menu"/> - + <div id="content"> <div class="innertube"> <h1>Bibliography</h1> @@ -1498,8 +1856,8 @@ <tr> <td><xsl:value-of select="@id"/></td> <td><xsl:value-of select="citedef"/></td> - </tr> - </xsl:if> + </tr> + </xsl:if> </xsl:for-each> </table> @@ -1529,7 +1887,7 @@ <xsl:template name="calc-arity"> <xsl:param name="string"/> <xsl:param name="no-of-pars"/> - + <xsl:variable name="length"> <xsl:value-of select="string-length($string)"/> </xsl:variable> @@ -1538,8 +1896,8 @@ <xsl:when test="$length > 0"> <xsl:call-template name="calc-arity"> <xsl:with-param name="string" select="substring-after($string, ',')"/> - <xsl:with-param name="no-of-pars" select="$no-of-pars+1"/> - </xsl:call-template> + <xsl:with-param name="no-of-pars" select="$no-of-pars+1"/> + </xsl:call-template> </xsl:when> <xsl:otherwise> <xsl:value-of select="$no-of-pars"/> @@ -1554,9 +1912,9 @@ <xsl:variable name="str1"> <xsl:call-template name="remove-paren-1"> <xsl:with-param name="string" select="$string"/> - <xsl:with-param name="start">(</xsl:with-param> - <xsl:with-param name="end">)</xsl:with-param> - </xsl:call-template> + <xsl:with-param name="start">(</xsl:with-param> + <xsl:with-param name="end">)</xsl:with-param> + </xsl:call-template> </xsl:variable> <xsl:variable name="str2"> @@ -1564,7 +1922,7 @@ <xsl:with-param name="string" select="$str1"/> <xsl:with-param name="start">{</xsl:with-param> <xsl:with-param name="end">}</xsl:with-param> - </xsl:call-template> + </xsl:call-template> </xsl:variable> <xsl:variable name="str3"> @@ -1572,7 +1930,7 @@ <xsl:with-param name="string" select="$str2"/> <xsl:with-param name="start">[</xsl:with-param> <xsl:with-param name="end">]</xsl:with-param> - </xsl:call-template> + </xsl:call-template> </xsl:variable> <xsl:value-of select="$str3"/> @@ -1584,7 +1942,7 @@ <xsl:param name="string"/> <xsl:param name="start"/> <xsl:param name="end"/> - + <xsl:variable name="tmp1"> <xsl:value-of select="substring-before($string, $start)"/> </xsl:variable> @@ -1597,7 +1955,7 @@ <xsl:variable name="retstring"> <xsl:call-template name="remove-paren"> <xsl:with-param name="string" select="$tmp2"/> - </xsl:call-template> + </xsl:call-template> </xsl:variable> <xsl:value-of select="concat(concat($tmp1, 'x'), $retstring)"/> </xsl:when> diff --git a/lib/erl_docgen/priv/xsl/db_man.xsl b/lib/erl_docgen/priv/xsl/db_man.xsl index 71c4a66707..2a8fb9fe3e 100644 --- a/lib/erl_docgen/priv/xsl/db_man.xsl +++ b/lib/erl_docgen/priv/xsl/db_man.xsl @@ -1,5 +1,5 @@ <?xml version="1.0" encoding="utf-8"?> -<!-- +<!-- # # %CopyrightBegin% # @@ -17,24 +17,294 @@ # under the License. # # %CopyrightEnd% - + --> <xsl:stylesheet version="1.0" - xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> + xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:preserve-space elements="code pre"/> <xsl:strip-space elements="*"/> <xsl:output method="text" encoding="UTF-8" indent="no"/> + <!-- Start of Dialyzer type/spec tags. See also the template matching "name" + --> + + <!-- Note: specs data for *one* module (as opposed to html and pdf) --> + <xsl:param name="specs_file" select="''"/> + <xsl:variable name="i" select="document($specs_file)"></xsl:variable> + + <xsl:template name="err"> + <xsl:param name="m"/> + <xsl:param name="n"/> + <xsl:param name="a"/> + <xsl:param name="s"/> + <xsl:message terminate="yes"> + Error <xsl:if test="$m != ''"><xsl:value-of select ="$m"/>:</xsl:if> + <xsl:value-of + select="$n"/>/<xsl:value-of + select="$a"/>: <xsl:value-of select="$s"/> + </xsl:message> + </xsl:template> + + <xsl:template name="spec_name"> + <xsl:variable name="curModule" select="ancestor::erlref/module"/> + <xsl:variable name="mod" select="@mod"/> + <xsl:variable name="name" select="@name"/> + <xsl:variable name="arity" select="@arity"/> + <xsl:variable name="clause" select="@clause"/> + <xsl:variable name="spec0" select= + "$i/module[@name=$curModule]/spec + [name=$name and arity=$arity + and (string-length($mod) = 0 or module = $mod)]"/> + <xsl:variable name="spec" select="$spec0[string-length($clause) = 0 + or position() = $clause]"/> + <xsl:if test="count($spec) = 0"> + <xsl:call-template name="err"> + <xsl:with-param name="m" select="$mod"/> + <xsl:with-param name="n" select="$name"/> + <xsl:with-param name="a" select="$arity"/> + <xsl:with-param name="s">unknown spec</xsl:with-param> + </xsl:call-template> + </xsl:if> + + <xsl:choose> + <xsl:when test="ancestor::cref"> + <xsl:message terminate="yes"> + Error: did not expect a 'name' tag with name/arity attributes here! + </xsl:message> + </xsl:when> + <xsl:when test="ancestor::erlref"> + <xsl:choose> + <xsl:when test="string(@with_guards) = 'no'"> + <xsl:apply-templates select="$spec/contract/clause/head"/> + </xsl:when> + <xsl:otherwise> + <xsl:call-template name="contract"> + <xsl:with-param name="contract" select="$spec/contract"/> + </xsl:call-template> + </xsl:otherwise> + </xsl:choose> + <xsl:text> .br</xsl:text> + </xsl:when> + </xsl:choose> + </xsl:template> + + <xsl:template name="contract"> + <xsl:param name="contract"/> + <xsl:call-template name="clause"> + <xsl:with-param name="clause" select="$contract/clause"/> + </xsl:call-template> + </xsl:template> + + <xsl:template name="clause"> + <xsl:param name="clause"/> + <xsl:variable name="type_desc" select="../type_desc"/> + <xsl:for-each select="$clause"> + <xsl:apply-templates select="head"/> + <xsl:if test="count(guard) > 0"> + <xsl:call-template name="guard"> + <xsl:with-param name="guard" select="guard"/> + <xsl:with-param name="type_desc" select="$type_desc"/> + </xsl:call-template> + </xsl:if> + </xsl:for-each> + </xsl:template> + + <xsl:template match="head"> + <xsl:text> .nf </xsl:text> + <xsl:text> .B </xsl:text> + <xsl:apply-templates/> + <xsl:text> .br</xsl:text> + <xsl:text> .fi</xsl:text> + </xsl:template> + + <xsl:template name="guard"> + <xsl:param name="guard"/> + <xsl:param name="type_desc"/> + <xsl:text> .RS</xsl:text> + <xsl:text> .TP</xsl:text> + <xsl:text> Types</xsl:text> + <xsl:call-template name="subtype"> + <xsl:with-param name="subtype" select="$guard/subtype"/> + <xsl:with-param name="type_desc" select="$type_desc"/> + </xsl:call-template> + <xsl:text> .RE</xsl:text> + </xsl:template> + + <xsl:template name="subtype"> + <xsl:param name="subtype"/> + <xsl:param name="type_desc"/> + <xsl:for-each select="$subtype"> + <xsl:variable name="tname" select="typename"/> + <xsl:variable name="tdesc" select="$type_desc[@name = $tname]"/> + <xsl:text> </xsl:text> + <xsl:apply-templates select="string"/> + <xsl:text> .br</xsl:text> + <xsl:apply-templates select="$type_desc[@name = $tname]"/> + </xsl:for-each> + </xsl:template> + + <!-- Note: <type_desc> has not been implemented for data types. --> + + <!-- Similar to <d> --> + <xsl:template match="type_desc"> + <xsl:text> </xsl:text><xsl:apply-templates/> + <xsl:text> .br</xsl:text> + </xsl:template> + + <!-- Datatypes --> + <xsl:template match="datatypes"> + <xsl:text> .SH DATA TYPES</xsl:text> + <xsl:apply-templates/> + </xsl:template> + + <!-- Datatype --> + <xsl:template match="datatype"> + <xsl:apply-templates/> + </xsl:template> + + <xsl:template match="typehead"> + <xsl:text> .nf </xsl:text> + <xsl:text> .B </xsl:text> + <xsl:apply-templates/> + <xsl:text> .br</xsl:text> + <xsl:text> .fi</xsl:text> + </xsl:template> + + <xsl:template match="local_defs"> + <xsl:text> .RS</xsl:text> + <xsl:apply-templates/> + <xsl:text> .RE</xsl:text> + </xsl:template> + + <xsl:template match="local_def"> + <xsl:text> </xsl:text> + <xsl:apply-templates/> + <xsl:text> .br</xsl:text> + </xsl:template> + + <xsl:template name="type_name"> + <xsl:variable name="curModule" select="ancestor::erlref/module"/> + <xsl:variable name="mod" select="@mod"/> + <xsl:variable name="name" select="@name"/> + <xsl:variable name="n_vars"> + <xsl:choose> + <xsl:when test="string-length(@n_vars) > 0"> + <xsl:value-of select="@n_vars"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="0"/> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + + <xsl:choose> + <xsl:when test="string-length($name) > 0"> + <xsl:variable name="type" select= + "$i/module[@name=$curModule]/type + [name=$name and n_vars=$n_vars + and (string-length($mod) = 0 or module = $mod)]"/> + + <xsl:if test="count($type) != 1"> + <xsl:call-template name="err"> + <xsl:with-param name="m" select="$mod"/> + <xsl:with-param name="n" select="$name"/> + <xsl:with-param name="a" select="$n_vars"/> + <xsl:with-param name="s">unknown type</xsl:with-param> + </xsl:call-template> + </xsl:if> + <xsl:apply-templates select="$type/typedecl"/> + </xsl:when> + <xsl:otherwise> + <xsl:text> .nf </xsl:text> + <xsl:text> .B </xsl:text> + <xsl:apply-templates/> + <xsl:text> .br</xsl:text> + <xsl:text> .fi</xsl:text> + </xsl:otherwise> + </xsl:choose> + </xsl:template> + + <!-- Used both in <datatype> and in <func>! --> + <xsl:template match="anno"> + <xsl:variable name="curModule" select="ancestor::erlref/module"/> + <xsl:variable name="anno" select="normalize-space(text())"/> + <xsl:variable name="namespec" + select="ancestor::desc/preceding-sibling::name"/> + <xsl:if test="count($namespec) = 0 and string-length($specs_file) > 0"> + <xsl:call-template name="err"> + <xsl:with-param name="s">cannot find 'name' (<xsl:value-of select="$anno"/>) + </xsl:with-param> + </xsl:call-template> + </xsl:if> + + <xsl:variable name="mod" select="$namespec/@mod"/> + <xsl:variable name="name" select="$namespec/@name"/> + <xsl:variable name="arity" select="$namespec/@arity"/> + <xsl:variable name="clause" select="$namespec/@clause"/> + <xsl:variable name="tmp_n_vars" select="$namespec/@n_vars"/> + <xsl:variable name="n_vars"> + <xsl:choose> + <xsl:when test="string-length($tmp_n_vars) > 0"> + <xsl:value-of select="$tmp_n_vars"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="0"/> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + <xsl:variable name="spec0" select= + "$i/module[@name=$curModule]/spec + [name=$name and arity=$arity + and (string-length($mod) = 0 or module = $mod)]"/> + <xsl:variable name="spec_annos" select= + "$spec0[string-length($clause) = 0 + or position() = $clause]/anno[.=$anno]"/> + <xsl:variable name="type_annos" select= + "$i/module[@name=$curModule]/type + [name=$name and n_vars=$n_vars + and (string-length($mod) = 0 or module = $mod)]/anno[.=$anno]"/> + + <xsl:if test="count($spec_annos) = 0 + and count($type_annos) = 0 + and string-length($specs_file) > 0"> + <xsl:variable name="n"> + <xsl:choose> + <xsl:when test="string-length($arity) = 0"> + <xsl:value-of select="$n_vars"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="$arity"/> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + <xsl:call-template name="err"> + <xsl:with-param name="m" select="$mod"/> + <xsl:with-param name="n" select="$name"/> + <xsl:with-param name="a" select="$n"/> + <xsl:with-param name="s">unknown annotation <xsl:value-of select="$anno"/> + </xsl:with-param> + </xsl:call-template> + </xsl:if> + <xsl:value-of select="$anno"/> + </xsl:template> + + <!-- Used for indentation of formatted types and specs --> + <xsl:template match="nbsp"> + <xsl:text> </xsl:text> + </xsl:template> + + <!-- End of Dialyzer type/spec tags --> + <!-- Header --> <xsl:template match="header"> </xsl:template> - + <!-- Section/Title --> <xsl:template match="section/title"> </xsl:template> - + <!-- *ref/Section --> <xsl:template match="erlref/section|comref/section|cref/section|fileref/section|appref/section"> <xsl:text> .SH "</xsl:text><xsl:value-of select="translate(title, 'abcdefghijklmnopqrstuvwxyz','ABCDEFGHIJKLMNOPQRSTUVWXYZ')"/><xsl:text>" </xsl:text> @@ -49,11 +319,11 @@ <!-- Lists --> - + <xsl:template match="list"> <xsl:text> .RS 2</xsl:text> <xsl:apply-templates/> - <xsl:text> .RE</xsl:text> + <xsl:text> .RE </xsl:text> </xsl:template> <xsl:template match="list/item"> @@ -66,9 +336,9 @@ <xsl:template match="taglist"> <xsl:text> .RS 2</xsl:text> <xsl:apply-templates select="tag|item"/> - <xsl:text> .RE</xsl:text> + <xsl:text> .RE </xsl:text> </xsl:template> - + <xsl:template match="taglist/tag"> <xsl:text> .TP 2 </xsl:text> <xsl:text>.B </xsl:text> @@ -76,7 +346,7 @@ </xsl:template> <xsl:template match="taglist/item"> - <xsl:apply-templates/> + <xsl:apply-templates/> </xsl:template> <xsl:template match="item/p"> @@ -88,10 +358,10 @@ <xsl:value-of select="$content"/> </xsl:when> <xsl:otherwise> - <xsl:text> .RS 2</xsl:text> - <xsl:text> .LP .LP </xsl:text> + <xsl:text> .RS 2</xsl:text> + <xsl:text> .LP .LP </xsl:text> <xsl:value-of select="$content"/> - <xsl:text> .RE</xsl:text> + <xsl:text> .RE</xsl:text> </xsl:otherwise> </xsl:choose> </xsl:template> @@ -171,7 +441,7 @@ <xsl:template match="application"> <xsl:apply-templates/> </xsl:template> - + <!-- Erlref --> <xsl:template match="/erlref"> <xsl:variable name="companyname"> @@ -184,7 +454,7 @@ <xsl:text>.TH </xsl:text><xsl:value-of select="module"/><xsl:text> 3 "</xsl:text><xsl:value-of select="$appname"/><xsl:text> </xsl:text><xsl:value-of select="$appver"/><xsl:text>" "</xsl:text><xsl:value-of select="$companyname"/><xsl:text>" "Erlang Module Definition" </xsl:text> <xsl:text>.SH NAME </xsl:text> - <xsl:value-of select="module"/><xsl:text> \- </xsl:text><xsl:value-of select="modulesummary"/><xsl:text> </xsl:text> + <xsl:value-of select="module"/><xsl:text> \- </xsl:text><xsl:value-of select="modulesummary"/><xsl:text> </xsl:text> <xsl:apply-templates/> </xsl:template> @@ -199,7 +469,7 @@ </xsl:variable> <xsl:text>.TH </xsl:text><xsl:value-of select="com"/><xsl:text> 1 "</xsl:text><xsl:value-of select="$appname"/><xsl:text> </xsl:text><xsl:value-of select="$appver"/><xsl:text>" "</xsl:text><xsl:value-of select="$companyname"/><xsl:text>" "User Commands" </xsl:text> <xsl:text>.SH NAME </xsl:text> - <xsl:value-of select="com"/><xsl:text> \- </xsl:text><xsl:value-of select="comsummary"/><xsl:text> </xsl:text> + <xsl:value-of select="com"/><xsl:text> \- </xsl:text><xsl:value-of select="comsummary"/><xsl:text> </xsl:text> <xsl:apply-templates/> </xsl:template> @@ -214,7 +484,7 @@ </xsl:variable> <xsl:text>.TH </xsl:text><xsl:value-of select="lib"/><xsl:text> 3 "</xsl:text><xsl:value-of select="$appname"/><xsl:text> </xsl:text><xsl:value-of select="$appver"/><xsl:text>" "</xsl:text><xsl:value-of select="$companyname"/><xsl:text>" "C Library Functions" </xsl:text> <xsl:text>.SH NAME </xsl:text> - <xsl:value-of select="lib"/><xsl:text> \- </xsl:text><xsl:value-of select="libsummary"/><xsl:text> </xsl:text> + <xsl:value-of select="lib"/><xsl:text> \- </xsl:text><xsl:value-of select="libsummary"/><xsl:text> </xsl:text> <xsl:apply-templates/> </xsl:template> @@ -229,7 +499,7 @@ </xsl:variable> <xsl:text>.TH </xsl:text><xsl:value-of select="file"/><xsl:text> 5 "</xsl:text><xsl:value-of select="$appname"/><xsl:text> </xsl:text><xsl:value-of select="$appver"/><xsl:text>" "</xsl:text><xsl:value-of select="$companyname"/><xsl:text>" "Files" </xsl:text> <xsl:text>.SH NAME </xsl:text> - <xsl:value-of select="file"/><xsl:text> \- </xsl:text><xsl:value-of select="filesummary"/><xsl:text> </xsl:text> + <xsl:value-of select="file"/><xsl:text> \- </xsl:text><xsl:value-of select="filesummary"/><xsl:text> </xsl:text> <xsl:apply-templates/> </xsl:template> @@ -244,7 +514,7 @@ </xsl:variable> <xsl:text>.TH </xsl:text><xsl:value-of select="app"/><xsl:text> 7 "</xsl:text><xsl:value-of select="$appname"/><xsl:text> </xsl:text><xsl:value-of select="$appver"/><xsl:text>" "</xsl:text><xsl:value-of select="$companyname"/><xsl:text>" "Erlang Application Definition" </xsl:text> <xsl:text>.SH NAME </xsl:text> - <xsl:value-of select="app"/><xsl:text> \- </xsl:text><xsl:value-of select="appsummary"/><xsl:text> </xsl:text> + <xsl:value-of select="app"/><xsl:text> \- </xsl:text><xsl:value-of select="appsummary"/><xsl:text> </xsl:text> <xsl:apply-templates/> </xsl:template> @@ -271,10 +541,26 @@ <!-- Func --> <xsl:template match="func"> <xsl:text> .LP</xsl:text> - <xsl:apply-templates/> + <xsl:apply-templates select="name"/> + <xsl:apply-templates select="fsummary|type|desc"/> </xsl:template> <xsl:template match="name"> + <xsl:choose> + <!-- @arity is mandatory when referring to a specification --> + <xsl:when test="string-length(@arity) > 0"> + <xsl:call-template name="spec_name"/> + </xsl:when> + <xsl:when test="ancestor::datatype"> + <xsl:call-template name="type_name"/> + </xsl:when> + <xsl:otherwise> + <xsl:call-template name="name"/> + </xsl:otherwise> + </xsl:choose> + </xsl:template> + + <xsl:template name="name"> <xsl:text> .B </xsl:text> <xsl:apply-templates/> <xsl:text> .br</xsl:text> @@ -296,7 +582,7 @@ <xsl:text> </xsl:text><xsl:value-of select="normalize-space(text())"/> <xsl:text> .br</xsl:text> </xsl:template> - + <!-- D --> <xsl:template match="d"> <xsl:text> </xsl:text><xsl:apply-templates/> @@ -316,7 +602,7 @@ <!-- This tag is skipped for now. --> </xsl:template> - + <!-- Authors --> <xsl:template match="authors"> <xsl:text> .SH AUTHORS</xsl:text> @@ -338,19 +624,26 @@ <!-- Do not noramlize any text within pre and code tags. --> <xsl:template match="pre/text()"> - <xsl:value-of select="."/> + <xsl:call-template name="replace-string"> + <xsl:with-param name="text" select="." /> + <xsl:with-param name="replace" select=""\"" /> + <xsl:with-param name="with" select=""\\"" /> + </xsl:call-template> </xsl:template> <xsl:template match="code/text()"> - <xsl:value-of select="."/> + <xsl:call-template name="replace-string"> + <xsl:with-param name="text" select="." /> + <xsl:with-param name="replace" select=""\"" /> + <xsl:with-param name="with" select=""\\"" /> + </xsl:call-template> </xsl:template> - <!-- Replace ' by \&' ans . by \&. --> <xsl:template match="text()"> <xsl:variable name="startstring"> <xsl:value-of select="normalize-space()"/><xsl:text> </xsl:text> - </xsl:variable> + </xsl:variable> <xsl:variable name="rep1"> <xsl:call-template name="replace-string"> <xsl:with-param name="text" select="$startstring" /> diff --git a/lib/erl_docgen/priv/xsl/db_pdf.xsl b/lib/erl_docgen/priv/xsl/db_pdf.xsl index e12b4d219a..1e80c360b8 100644 --- a/lib/erl_docgen/priv/xsl/db_pdf.xsl +++ b/lib/erl_docgen/priv/xsl/db_pdf.xsl @@ -1,5 +1,5 @@ <?xml version="1.0" encoding="utf-8"?> -<!-- +<!-- # # %CopyrightBegin% # @@ -17,7 +17,7 @@ # under the License. # # %CopyrightEnd% - + --> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" @@ -27,16 +27,310 @@ <xsl:include href="db_pdf_params.xsl"/> + <!-- Start of Dialyzer type/spec tags. + See also the template matching "name" and the template "bookmarks6" + --> + + <xsl:param name="specs_file" select="''"/> + <xsl:variable name="i" select="document($specs_file)"></xsl:variable> + + <xsl:template name="err"> + <xsl:param name="m"/> + <xsl:param name="n"/> + <xsl:param name="a"/> + <xsl:param name="s"/> + <xsl:message terminate="yes"> + Error <xsl:if test="$m != ''"><xsl:value-of select ="$m"/>:</xsl:if> + <xsl:value-of + select="$n"/>/<xsl:value-of + select="$a"/>: <xsl:value-of select="$s"/> + </xsl:message> + </xsl:template> + + <xsl:template name="spec_name"> + <xsl:variable name="curModule" select="ancestor::erlref/module"/> + <xsl:variable name="mod" select="@mod"/> + <xsl:variable name="name" select="@name"/> + <xsl:variable name="arity" select="@arity"/> + <xsl:variable name="clause" select="@clause"/> + <xsl:variable name="spec0" select= + "$i/specs/module[@name=$curModule]/spec + [name=$name and arity=$arity + and (string-length($mod) = 0 or module = $mod)]"/> + <xsl:variable name="spec" select="$spec0[string-length($clause) = 0 + or position() = $clause]"/> + <xsl:if test="count($spec) = 0"> + <xsl:call-template name="err"> + <xsl:with-param name="m" select="$mod"/> + <xsl:with-param name="n" select="$name"/> + <xsl:with-param name="a" select="$arity"/> + <xsl:with-param name="s">unknown spec</xsl:with-param> + </xsl:call-template> + </xsl:if> + + <xsl:choose> + <xsl:when test="ancestor::cref"> + <xsl:message terminate="yes"> + Error: did not expect a 'name' tag with name/arity attributes here! + </xsl:message> + </xsl:when> + <xsl:when test="ancestor::erlref"> + <fo:block id="{generate-id()}"> + <xsl:choose> + <xsl:when test="string(@with_guards) = 'no'"> + <xsl:apply-templates select="$spec/contract/clause/head"/> + </xsl:when> + <xsl:otherwise> + <xsl:call-template name="contract"> + <xsl:with-param name="contract" select="$spec/contract"/> + </xsl:call-template> + </xsl:otherwise> + </xsl:choose> + </fo:block> + </xsl:when> + </xsl:choose> + </xsl:template> + + <xsl:template name="contract"> + <xsl:param name="contract"/> + <xsl:call-template name="clause"> + <xsl:with-param name="clause" select="$contract/clause"/> + </xsl:call-template> + </xsl:template> + + <xsl:template name="clause"> + <xsl:param name="clause"/> + <xsl:variable name="type_desc" select="../type_desc"/> + <xsl:for-each select="$clause"> + <xsl:apply-templates select="head"/> + <xsl:if test="count(guard) > 0"> + <xsl:call-template name="guard"> + <xsl:with-param name="guard" select="guard"/> + <xsl:with-param name="type_desc" select="$type_desc"/> + </xsl:call-template> + </xsl:if> + </xsl:for-each> + </xsl:template> + + <xsl:template match="head"> + <fo:block xsl:use-attribute-sets="function-name"> + <xsl:apply-templates/> + </fo:block> + </xsl:template> + + <xsl:template name="guard"> + <fo:block> + <xsl:text>Types:</xsl:text> + </fo:block> + <fo:list-block xsl:use-attribute-sets="type-listblock"> + <xsl:call-template name="subtype"> + <xsl:with-param name="subtype" select="$guard/subtype"/> + <xsl:with-param name="type_desc" select="$type_desc"/> + </xsl:call-template> + </fo:list-block> + </xsl:template> + + <xsl:template name="subtype"> + <xsl:param name="subtype"/> + <xsl:param name="type_desc"/> + <xsl:for-each select="$subtype"> + <xsl:variable name="tname" select="typename"/> + <xsl:variable name="tdesc" select="$type_desc[@name = $tname]"/> + <fo:list-item xsl:use-attribute-sets="type-listitem"> + <fo:list-item-label end-indent="label-end()"> + <fo:block> + </fo:block> + </fo:list-item-label> + <fo:list-item-body start-indent="body-start()" format="justify"> + <fo:block font-weight="bold"> + <xsl:apply-templates select="string"/> + </fo:block> + </fo:list-item-body> + </fo:list-item> + <xsl:apply-templates select="$type_desc[@name = $tname]"/> + </xsl:for-each> + </xsl:template> + + <!-- Note: <type_desc> has not been implemented for data types. --> + + <!-- Similar to <d> --> + <xsl:template match="type_desc"> + <fo:list-item xsl:use-attribute-sets="type-listitem"> + <fo:list-item-label end-indent="label-end()"><fo:block></fo:block> + </fo:list-item-label> + <fo:list-item-body start-indent="body-start()" format="justify"> + <fo:block> + <xsl:apply-templates/> + </fo:block> + </fo:list-item-body> + </fo:list-item> + </xsl:template> + + <!-- Datatypes --> + <xsl:template match="datatypes"> + <fo:block xsl:use-attribute-sets="h3"> + <xsl:text>Data Types</xsl:text> + </fo:block> + <xsl:apply-templates/> + </xsl:template> + + <!-- Datatype --> + <xsl:template match="datatype"> + <fo:block xsl:use-attribute-sets="function-name"> + <xsl:apply-templates select="name"/> + </fo:block> + <xsl:apply-templates select="desc"/> + </xsl:template> + + <!-- Like <head>... --> + <xsl:template match="typehead"> + <fo:block xsl:use-attribute-sets="function-name"> + <xsl:apply-templates/> + </fo:block> + </xsl:template> + + <!-- Like <guard>, except "Types:"... --> + <xsl:template match="local_defs"> + <fo:list-block xsl:use-attribute-sets="type-listblock"> + <xsl:apply-templates/> + </fo:list-block> + </xsl:template> + + <!-- Like <subtype>... --> + <xsl:template match="local_def"> + <fo:list-item xsl:use-attribute-sets="type-listitem"> + <fo:list-item-label end-indent="label-end()"> + <fo:block> + </fo:block> + </fo:list-item-label> + <fo:list-item-body start-indent="body-start()" format="justify"> + <fo:block font-weight="bold"> + <xsl:apply-templates/> + </fo:block> + </fo:list-item-body> + </fo:list-item> + </xsl:template> + + <xsl:template name="type_name"> + <xsl:variable name="curModule" select="ancestor::erlref/module"/> + <xsl:variable name="mod" select="@mod"/> + <xsl:variable name="name" select="@name"/> + <xsl:variable name="n_vars"> + <xsl:choose> + <xsl:when test="string-length(@n_vars) > 0"> + <xsl:value-of select="@n_vars"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="0"/> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + + <xsl:choose> + <xsl:when test="string-length($name) > 0"> + <xsl:variable name="type" select= + "$i/specs/module[@name=$curModule]/type + [name=$name and n_vars=$n_vars + and (string-length($mod) = 0 or module = $mod)]"/> + + <xsl:if test="count($type) != 1"> + <xsl:call-template name="err"> + <xsl:with-param name="m" select="$mod"/> + <xsl:with-param name="n" select="$name"/> + <xsl:with-param name="a" select="$n_vars"/> + <xsl:with-param name="s">unknown type</xsl:with-param> + </xsl:call-template> + </xsl:if> + <xsl:apply-templates select="$type/typedecl"/> + </xsl:when> + <xsl:otherwise> + <fo:inline font-weight="bold" xsl:use-attribute-sets="type-listitem"> + <xsl:value-of select="."/> + </fo:inline> + </xsl:otherwise> + </xsl:choose> + </xsl:template> + + <!-- Used both in <datatype> and in <func>! --> + <xsl:template match="anno"> + <xsl:variable name="curModule" select="ancestor::erlref/module"/> + <xsl:variable name="anno" select="normalize-space(text())"/> + <xsl:variable name="namespec" + select="ancestor::desc/preceding-sibling::name"/> + <xsl:if test="count($namespec) = 0 and string-length($specs_file) > 0"> + <xsl:call-template name="err"> + <xsl:with-param name="s">cannot find 'name' (<xsl:value-of select="$anno"/>) + </xsl:with-param> + </xsl:call-template> + </xsl:if> + + <xsl:variable name="mod" select="$namespec/@mod"/> + <xsl:variable name="name" select="$namespec/@name"/> + <xsl:variable name="arity" select="$namespec/@arity"/> + <xsl:variable name="clause" select="$namespec/@clause"/> + <xsl:variable name="tmp_n_vars" select="$namespec/@n_vars"/> + <xsl:variable name="n_vars"> + <xsl:choose> + <xsl:when test="string-length($tmp_n_vars) > 0"> + <xsl:value-of select="$tmp_n_vars"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="0"/> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + <xsl:variable name="spec0" select= + "$i/specs/module[@name=$curModule]/spec + [name=$name and arity=$arity + and (string-length($mod) = 0 or module = $mod)]"/> + <xsl:variable name="spec_annos" select= + "$spec0[string-length($clause) = 0 + or position() = $clause]/anno[.=$anno]"/> + <xsl:variable name="type_annos" select= + "$i/specs/module[@name=$curModule]/type + [name=$name and n_vars=$n_vars + and (string-length($mod) = 0 or module = $mod)]/anno[.=$anno]"/> + + <xsl:if test="count($spec_annos) = 0 + and count($type_annos) = 0 + and string-length($specs_file) > 0"> + <xsl:variable name="n"> + <xsl:choose> + <xsl:when test="string-length($arity) = 0"> + <xsl:value-of select="$n_vars"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="$arity"/> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + <xsl:call-template name="err"> + <xsl:with-param name="m" select="$mod"/> + <xsl:with-param name="n" select="$name"/> + <xsl:with-param name="a" select="$n"/> + <xsl:with-param name="s">unknown annotation <xsl:value-of select="$anno"/> + </xsl:with-param> + </xsl:call-template> + </xsl:if> + <xsl:value-of select="$anno"/> + </xsl:template> + + <!-- Used for indentation of formatted types and specs --> + <xsl:template match="nbsp"> + <xsl:text> </xsl:text> + </xsl:template> + + <!-- End of Dialyzer type/spec tags --> <xsl:template match="/"> <xsl:apply-templates select="book"/> </xsl:template> - + <xsl:template match="book"> <fo:root xmlns:fo="http://www.w3.org/1999/XSL/Format"> - <!-- Master pages --> + <!-- Master pages --> <fo:layout-master-set> <fo:simple-page-master master-name="cover" @@ -47,7 +341,7 @@ <xsl:attribute name="page-width"> <xsl:value-of select="$page-width"/> </xsl:attribute> - <fo:region-body + <fo:region-body margin="0mm"/> </fo:simple-page-master> @@ -63,7 +357,7 @@ <xsl:attribute name="page-width"> <xsl:value-of select="$page-width"/> </xsl:attribute> - <fo:region-body + <fo:region-body margin-top="15mm" margin-bottom="20mm"/> <fo:region-before @@ -100,10 +394,10 @@ <fo:page-sequence-master master-name="document"> <fo:repeatable-page-master-alternatives> - <fo:conditional-page-master-reference + <fo:conditional-page-master-reference master-reference="left-page" odd-or-even="even"/> - <fo:conditional-page-master-reference + <fo:conditional-page-master-reference master-reference="right-page" odd-or-even="odd"/> </fo:repeatable-page-master-alternatives> @@ -166,7 +460,7 @@ <fo:flow flow-name="xsl-region-body"> <fo:block> - + </fo:block> <xsl:apply-templates select="parts"/> @@ -189,7 +483,7 @@ <!-- Cover page --> <xsl:template match="header/title"> - <fo:page-sequence + <fo:page-sequence font-family="sans-serif" force-page-count="even" master-reference="cover"> @@ -242,7 +536,7 @@ the License for the specific language governing rights and limitations under the License. - The Initial Developer of the Original Code is + The Initial Developer of the Original Code is --> <xsl:value-of select="$companyname"/>. </fo:block> @@ -281,22 +575,22 @@ <xsl:template name="bookmarks1"> <xsl:param name="entries"/> <xsl:if test="$entries != ''"> - + <fo:bookmark internal-destination="{generate-id(/book/parts/part)}" starting-state="hide"> <fo:bookmark-title>User's Guide</fo:bookmark-title> - + <xsl:for-each select="$entries"> <xsl:call-template name="bookmarks2"> <xsl:with-param name="entries" select="chapter[header/title]"/> </xsl:call-template> </xsl:for-each> - + </fo:bookmark> </xsl:if> </xsl:template> - + <xsl:template name="bookmarks2"> <xsl:param name="entries"/> <xsl:for-each select="$entries"> @@ -341,7 +635,7 @@ starting-state="hide"> <fo:bookmark-title>Reference Manual</fo:bookmark-title> <xsl:for-each select="$entries"> - + <xsl:call-template name="bookmarks5"> <xsl:with-param name="entries" select="erlref[module]|comref[com]|cref[lib]|fileref[file]|appref[app]"/> @@ -387,7 +681,7 @@ <fo:bookmark internal-destination="{generate-id(nametext)}" starting-state="hide"> <xsl:variable name="fname"> <xsl:value-of select="substring-before(nametext, '(')"/> - </xsl:variable> + </xsl:variable> <fo:bookmark-title> <xsl:choose> <xsl:when test="string-length($fname) > 0"> @@ -396,7 +690,7 @@ <xsl:otherwise> <xsl:value-of select="nametext"/>() </xsl:otherwise> - </xsl:choose> + </xsl:choose> </fo:bookmark-title> </fo:bookmark> </xsl:when> @@ -404,60 +698,76 @@ <fo:bookmark internal-destination="{generate-id(.)}" starting-state="hide"> <xsl:variable name="tmpstring"> <xsl:value-of select="substring-before(substring-after(., '('), '->')"/> - </xsl:variable> + </xsl:variable> <xsl:variable name="ustring"> <xsl:choose> <xsl:when test="string-length($tmpstring) > 0"> <xsl:call-template name="remove-paren"> <xsl:with-param name="string" select="$tmpstring"/> - </xsl:call-template> + </xsl:call-template> </xsl:when> <xsl:otherwise> <xsl:call-template name="remove-paren"> <xsl:with-param name="string" select="substring-after(., '(')"/> - </xsl:call-template> + </xsl:call-template> </xsl:otherwise> </xsl:choose> - </xsl:variable> + </xsl:variable> <xsl:variable name="arity"> - <xsl:call-template name="calc-arity"> - <xsl:with-param name="string" select="substring-before($ustring, ')')"/> - <xsl:with-param name="no-of-pars" select="0"/> - </xsl:call-template> - </xsl:variable> - - <xsl:variable name="fname"> - <xsl:variable name="fname1"> - <xsl:value-of select="substring-before(., '(')"/> - </xsl:variable> - <xsl:variable name="fname2"> - <xsl:value-of select="substring-after($fname1, 'erlang:')"/> - </xsl:variable> - <xsl:choose> - <xsl:when test="string-length($fname2) > 0"> - <xsl:value-of select="$fname2"/> - </xsl:when> + <xsl:choose> + <xsl:when test="string-length(@arity) > 0"> + <!-- Dialyzer spec --> + <xsl:value-of select="@arity"/> + </xsl:when> <xsl:otherwise> - <xsl:value-of select="$fname1"/> + <xsl:call-template name="calc-arity"> + <xsl:with-param name="string" select="substring-before($ustring, ')')"/> + <xsl:with-param name="no-of-pars" select="0"/> + </xsl:call-template> </xsl:otherwise> </xsl:choose> </xsl:variable> + <xsl:variable name="fname"> + <xsl:choose> + <xsl:when test="string-length(@name) > 0"> + <!-- Dialyzer spec --> + <xsl:value-of select="@name"/> + </xsl:when> + <xsl:otherwise> + <xsl:variable name="fname1"> + <xsl:value-of select="substring-before(., '(')"/> + </xsl:variable> + <xsl:variable name="fname2"> + <xsl:value-of select="substring-after($fname1, 'erlang:')"/> + </xsl:variable> + <xsl:choose> + <xsl:when test="string-length($fname2) > 0"> + <xsl:value-of select="$fname2"/> + </xsl:when> + <xsl:otherwise> + <xsl:value-of select="$fname1"/> + </xsl:otherwise> + </xsl:choose> + </xsl:otherwise> + </xsl:choose> + </xsl:variable> + <fo:bookmark-title> <xsl:value-of select="$fname"/>/<xsl:value-of select="$arity"/> </fo:bookmark-title> </fo:bookmark> </xsl:when> </xsl:choose> - + </xsl:for-each> </xsl:template> <!-- UG part --> - + <!-- Parts --> <xsl:template match="parts"> <xsl:apply-templates select="part"/> @@ -491,7 +801,7 @@ <xsl:value-of select="$partnum"/>.<xsl:number/>  <xsl:value-of select="header/title"/> </fo:marker> <xsl:value-of select="$partnum"/>.<xsl:number/>  <xsl:value-of select="header/title"/> - + </fo:block> <xsl:apply-templates select="section|quote|warning|note|br|image|marker|table|p|pre|code|list|taglist|codeinclude|erleval"> @@ -567,7 +877,7 @@ </xsl:template> <!-- Lists --> - + <xsl:template match="list"> <xsl:param name="partnum"/> <fo:list-block xsl:use-attribute-sets="listblock"> @@ -692,7 +1002,7 @@ </xsl:variable> <fo:block xsl:use-attribute-sets="code"> - <xsl:apply-templates select="text()"/> + <xsl:apply-templates select="text()"/> </fo:block> <xsl:if test="@caption"> @@ -711,7 +1021,7 @@ </xsl:variable> <fo:block xsl:use-attribute-sets="code"> - <xsl:apply-templates/> + <xsl:apply-templates/> </fo:block> <xsl:if test="@caption"> @@ -734,23 +1044,23 @@ <xsl:variable name="partnum"> <xsl:number level="any" from="book" count="part|application"/> </xsl:variable> - - <fo:block xsl:use-attribute-sets="h1" id="{generate-id()}"> + + <fo:block xsl:use-attribute-sets="h1" id="{generate-id()}"> <xsl:if test="/book/header/title"> <xsl:value-of select="$partnum"/>    <xsl:text>Reference Manual</xsl:text> - </xsl:if> + </xsl:if> </fo:block> - - + + <xsl:apply-templates select="description"> <xsl:with-param name="partnum" select="$partnum"/> </xsl:apply-templates> - + <xsl:apply-templates select="erlref|comref|cref|fileref|appref"> <xsl:with-param name="partnum" select="$partnum"/> </xsl:apply-templates> - + </xsl:template> <!-- Erlref --> @@ -763,7 +1073,7 @@ <fo:marker marker-class-name="chapter-title"> <xsl:value-of select="module"/> </fo:marker> - <xsl:value-of select="module"/> + <xsl:value-of select="module"/> </fo:block> <xsl:text>Erlang module</xsl:text> </fo:block> @@ -784,7 +1094,7 @@ <fo:marker marker-class-name="chapter-title"> <xsl:value-of select="com"/> </fo:marker> - <xsl:value-of select="com"/> + <xsl:value-of select="com"/> </fo:block> <xsl:text>Command</xsl:text> </fo:block> @@ -805,7 +1115,7 @@ <fo:marker marker-class-name="chapter-title"> <xsl:value-of select="lib"/> </fo:marker> - <xsl:value-of select="lib"/> + <xsl:value-of select="lib"/> </fo:block> <xsl:text>C Library</xsl:text> </fo:block> @@ -826,7 +1136,7 @@ <fo:marker marker-class-name="chapter-title"> <xsl:value-of select="file"/> </fo:marker> - <xsl:value-of select="file"/> + <xsl:value-of select="file"/> </fo:block> <xsl:text>Name</xsl:text> </fo:block> @@ -847,7 +1157,7 @@ <fo:marker marker-class-name="chapter-title"> <xsl:value-of select="app"/> </fo:marker> - <xsl:value-of select="app"/> + <xsl:value-of select="app"/> </fo:block> <xsl:text>Application</xsl:text> </fo:block> @@ -900,9 +1210,7 @@ <xsl:template match="func"> <xsl:param name="partnum"/> - <fo:block xsl:use-attribute-sets="function-name"> - <xsl:apply-templates select="name"/> - </fo:block> + <xsl:apply-templates select="name"/> <xsl:apply-templates select="fsummary|type|desc"> <xsl:with-param name="partnum" select="$partnum"/> @@ -914,15 +1222,35 @@ <xsl:template match="name"> <xsl:param name="partnum"/> <xsl:choose> + <!-- @arity is mandatory when referring to a specification --> + <xsl:when test="string-length(@arity) > 0"> + <xsl:call-template name="spec_name"/> + </xsl:when> + <xsl:when test="ancestor::datatype"> + <xsl:call-template name="type_name"/> + </xsl:when> + <xsl:otherwise> + <fo:block xsl:use-attribute-sets="function-name"> + <xsl:call-template name="name"> + <xsl:with-param name="partnum" select="$partnum"/> + </xsl:call-template> + </fo:block> + </xsl:otherwise> + </xsl:choose> + </xsl:template> + + <xsl:template name="name"> + <xsl:param name="partnum"/> + <xsl:choose> <xsl:when test="ancestor::cref"> <fo:block id="{generate-id(nametext)}"> - <xsl:value-of select="ret"/><xsl:text> </xsl:text><xsl:value-of select="nametext"/> - </fo:block> + <xsl:value-of select="ret"/><xsl:text> </xsl:text><xsl:value-of select="nametext"/> + </fo:block> </xsl:when> <xsl:otherwise> <fo:block id="{generate-id(.)}"> - <xsl:value-of select="."/> - </fo:block> + <xsl:value-of select="."/> + </fo:block> </xsl:otherwise> </xsl:choose> </xsl:template> @@ -931,9 +1259,9 @@ <!-- Type --> <xsl:template match="type"> <xsl:param name="partnum"/> - + <fo:block> - <xsl:text>Types:</xsl:text> + <xsl:text>Types:</xsl:text> </fo:block> <fo:list-block xsl:use-attribute-sets="type-listblock"> @@ -1001,9 +1329,9 @@ <xsl:param name="chapnum"/> <xsl:variable name="tabnum"> <xsl:number level="any" from="chapter" count="table"/> - </xsl:variable> + </xsl:variable> <fo:table xsl:use-attribute-sets="table"> - <fo:table-body> + <fo:table-body> <xsl:apply-templates select="row"> <xsl:with-param name="chapnum" select="$chapnum"/> <xsl:with-param name="tabnum" select="$tabnum"/> @@ -1107,7 +1435,7 @@ <xsl:template name="calc-arity"> <xsl:param name="string"/> <xsl:param name="no-of-pars"/> - + <xsl:variable name="length"> <xsl:value-of select="string-length($string)"/> </xsl:variable> @@ -1116,8 +1444,8 @@ <xsl:when test="$length > 0"> <xsl:call-template name="calc-arity"> <xsl:with-param name="string" select="substring-after($string, ',')"/> - <xsl:with-param name="no-of-pars" select="$no-of-pars+1"/> - </xsl:call-template> + <xsl:with-param name="no-of-pars" select="$no-of-pars+1"/> + </xsl:call-template> </xsl:when> <xsl:otherwise> <xsl:value-of select="$no-of-pars"/> @@ -1131,9 +1459,9 @@ <xsl:variable name="str1"> <xsl:call-template name="remove-paren-1"> <xsl:with-param name="string" select="$string"/> - <xsl:with-param name="start">(</xsl:with-param> - <xsl:with-param name="end">)</xsl:with-param> - </xsl:call-template> + <xsl:with-param name="start">(</xsl:with-param> + <xsl:with-param name="end">)</xsl:with-param> + </xsl:call-template> </xsl:variable> <xsl:variable name="str2"> @@ -1141,7 +1469,7 @@ <xsl:with-param name="string" select="$str1"/> <xsl:with-param name="start">{</xsl:with-param> <xsl:with-param name="end">}</xsl:with-param> - </xsl:call-template> + </xsl:call-template> </xsl:variable> <xsl:variable name="str3"> @@ -1149,7 +1477,7 @@ <xsl:with-param name="string" select="$str2"/> <xsl:with-param name="start">[</xsl:with-param> <xsl:with-param name="end">]</xsl:with-param> - </xsl:call-template> + </xsl:call-template> </xsl:variable> <xsl:value-of select="$str3"/> @@ -1161,7 +1489,7 @@ <xsl:param name="string"/> <xsl:param name="start"/> <xsl:param name="end"/> - + <xsl:variable name="tmp1"> <xsl:value-of select="substring-before($string, $start)"/> </xsl:variable> @@ -1174,7 +1502,7 @@ <xsl:variable name="retstring"> <xsl:call-template name="remove-paren"> <xsl:with-param name="string" select="$tmp2"/> - </xsl:call-template> + </xsl:call-template> </xsl:variable> <xsl:value-of select="concat(concat($tmp1, 'x'), $retstring)"/> </xsl:when> diff --git a/lib/erl_docgen/src/Makefile b/lib/erl_docgen/src/Makefile new file mode 100644 index 0000000000..8e81bccd59 --- /dev/null +++ b/lib/erl_docgen/src/Makefile @@ -0,0 +1,96 @@ +# +# %CopyrightBegin% +# +# Copyright Ericsson AB 1996-2010. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% +# + +include $(ERL_TOP)/make/target.mk +include $(ERL_TOP)/make/$(TARGET)/otp.mk + +# ---------------------------------------------------- +# Application version +# ---------------------------------------------------- +include ../vsn.mk +VSN=$(ERL_DOCGEN_VSN) + +# ---------------------------------------------------- +# Release directory specification +# ---------------------------------------------------- +RELSYSDIR = $(RELEASE_PATH)/lib/erl_docgen-$(VSN) + +# ---------------------------------------------------- +# Target Specs +# ---------------------------------------------------- +MODULES = \ + otp_specs + +HRL_FILES = + +ERL_FILES = $(MODULES:%=%.erl) + +TARGET_FILES = $(MODULES:%=$(EBIN)/%.$(EMULATOR)) $(APP_TARGET) $(APPUP_TARGET) + +APP_FILE = erl_docgen.app + +APP_SRC = $(APP_FILE).src +APP_TARGET = $(EBIN)/$(APP_FILE) + +APPUP_FILE = erl_docgen.appup + +APPUP_SRC= $(APPUP_FILE).src +APPUP_TARGET= $(EBIN)/$(APPUP_FILE) + +# ---------------------------------------------------- +# FLAGS +# ---------------------------------------------------- +ERL_COMPILE_FLAGS += -I../../xmerl/include + +# ---------------------------------------------------- +# Targets +# ---------------------------------------------------- + +debug opt: $(TARGET_FILES) + +clean: + rm -f $(TARGET_FILES) + rm -f core + +docs: + + +# ---------------------------------------------------- +# Special Build Targets +# ---------------------------------------------------- + +$(APP_TARGET): $(APP_SRC) ../vsn.mk + sed -e 's;%VSN%;$(VSN);' $< > $@ + +$(APPUP_TARGET): $(APPUP_SRC) ../vsn.mk + sed -e 's;%VSN%;$(VSN);' $< > $@ + +# ---------------------------------------------------- +# Release Target +# ---------------------------------------------------- +include $(ERL_TOP)/make/otp_release_targets.mk + +release_spec: opt + $(INSTALL_DIR) $(RELSYSDIR)/src + $(INSTALL_DATA) $(ERL_FILES) $(RELSYSDIR)/src + $(INSTALL_DIR) $(RELSYSDIR)/ebin + $(INSTALL_DATA) $(TARGET_FILES) $(RELSYSDIR)/ebin + +release_docs_spec: + diff --git a/lib/erl_docgen/src/erl_docgen.app.src b/lib/erl_docgen/src/erl_docgen.app.src new file mode 100644 index 0000000000..1720464b6d --- /dev/null +++ b/lib/erl_docgen/src/erl_docgen.app.src @@ -0,0 +1,12 @@ +{application, erl_docgen, + [{description, "Misc tools for building documentation"}, + {vsn, "%VSN%"}, + {modules, [otp_specs + ] + }, + {registered,[]}, + {applications, [kernel,stdlib]}, + {env, [] + } + ] +}. diff --git a/lib/erl_docgen/src/erl_docgen.appup.src b/lib/erl_docgen/src/erl_docgen.appup.src new file mode 100644 index 0000000000..54a63833e6 --- /dev/null +++ b/lib/erl_docgen/src/erl_docgen.appup.src @@ -0,0 +1 @@ +{"%VSN%",[],[]}. diff --git a/lib/erl_docgen/src/otp_specs.erl b/lib/erl_docgen/src/otp_specs.erl new file mode 100644 index 0000000000..728ddb2e6e --- /dev/null +++ b/lib/erl_docgen/src/otp_specs.erl @@ -0,0 +1,701 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1996-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-module(otp_specs). + +-export([module/2, package/2, overview/2, type/1]). + +-include("xmerl.hrl"). + +-define(XML_EXPORT, xmerl_xml). +-define(DEFAULT_XML_EXPORT, ?XML_EXPORT). +-define(DEFAULT_PP, erl_pp). +-define(IND(N), #xmlText{value="\n" ++ lists:duplicate(N, $\s)}). +-define(NL, "\n"). + +module(Element, Options) -> + XML = layout_module(Element, init_opts(Options)), + Export = proplists:get_value(xml_export, Options, + ?DEFAULT_XML_EXPORT), + xmerl:export_simple(XML, Export, [#xmlAttribute{name=prolog, + value=""}]). + +-record(opts, {pretty_print, file_suffix}). + +init_opts(Options) -> + #opts{pretty_print = proplists:get_value(pretty_print, + Options, ?DEFAULT_PP), + %% It *is* depending on edoc.hrl! + file_suffix = proplists:get_value(file_suffix, Options, ".html")}. + +layout_module(#xmlElement{name = module, content = Es}=E, Opts) -> + Name = get_attrval(name, E), + Functions = [{function_name(Elem), Elem} || + Elem <- get_content(functions, Es)], + Types = [{type_name(Elem), Elem} || Elem <- get_content(typedecls, Es)], + Body = [{module, + [{name,[Name]}], + ([?NL] ++ types(lists:sort(Types), Opts) + ++ functions(lists:sort(Functions), Opts) + ++ timestamp())}], + Body. + +timestamp() -> + [{timestamp, [io_lib:fwrite("Generated by EDoc, ~s, ~s.", + [edoc_lib:datestr(date()), + edoc_lib:timestr(time())])]},?NL]. + +functions(Fs, Opts) -> + lists:flatmap(fun ({Name, E}) -> function(Name, E, Opts) end, Fs). + +function(Name, #xmlElement{content = Es}, Opts) -> + TS = get_content(typespec, Es), + Spec = typespec(TS, Opts), + [{spec,(Name + ++ [?IND(2),{contract,Spec}] + ++ typespec_annos(TS))}, + ?NL]. + +function_name(E) -> + [] = get_attrval(module, E), + [?IND(2),{name,[atom(get_attrval(name, E))]}, + ?IND(2),{arity,[get_attrval(arity, E)]}]. + +label_anchor(Content, E) -> + case get_attrval(label, E) of + "" -> Content; + Ref -> [{marker, [{id, Ref}], Content}] + end. + +typespec([], _Opts) -> []; +typespec(Es, Opts) -> + {Head, LDefs} = collect_clause(Es, Opts), + clause(Head, LDefs) ++ [?IND(2)]. + +collect_clause(Es, Opts) -> + Name = t_name(get_elem(erlangName, Es)), + Defs = get_elem(localdef, Es), + [Type] = get_elem(type, Es), + {format_spec(Name, Type, Opts), collect_local_defs(Defs, Opts)}. + +clause(Head, LDefs) -> + FC = [?IND(6),{head,Head}] ++ local_clause_defs(LDefs), + [?IND(4),{clause,FC}]. + +local_clause_defs([]) -> []; +local_clause_defs(LDefs) -> + LocalDefs = [{subtype,T} || T <- coalesce_local_defs(LDefs, [])], + [?IND(6),{guard,margin(8, LocalDefs)}]. + +types(Ts, Opts) -> + lists:flatmap(fun ({Name, E}) -> typedecl(Name, E, Opts) end, Ts). + +typedecl(Name, E=#xmlElement{content = Es}, Opts) -> + TD = get_content(typedef, Es), + TypeDef = typedef(E, TD, Opts), + [{type,(Name + ++ [?IND(2),{typedecl, TypeDef}] + ++ typedef_annos(TD))}, + ?NL]. + +type_name(#xmlElement{content = Es}) -> + Typedef = get_content(typedef, Es), + [E] = get_elem(erlangName, Typedef), + Args = get_content(argtypes, Typedef), + [] = get_attrval(module, E), + [?IND(2),{name,[atom(get_attrval(name, E))]}, + ?IND(2),{n_vars,[integer_to_list(length(Args))]}]. + +typedef(E, Es, Opts) -> + Ns = get_elem(erlangName, Es), + Name = + ([t_name(Ns), "("] + ++ seq(fun t_utype_elem/1, get_content(argtypes, Es), [")"])), + LDefs = collect_local_defs(get_elem(localdef, Es), Opts), + TypeHead = case get_elem(type, Es) of + [] -> label_anchor(Name, E); + Type -> (label_anchor(Name, E) + ++ format_type(Name, Type, Opts)) + end, + ([?IND(6),{typehead,TypeHead}] + ++ local_type_defs(LDefs, [])). + +local_type_defs([], _) -> []; +local_type_defs(LDefs, Last) -> + LocalDefs = [{local_def,T} || T <- coalesce_local_defs(LDefs, Last)], + [?IND(6),{local_defs,margin(8, LocalDefs)}]. + +collect_local_defs(Es, Opts) -> + [collect_localdef(E, Opts) || E <- Es]. + +collect_localdef(E = #xmlElement{content = Es}, Opts) -> + Name = case get_elem(typevar, Es) of + [] -> + label_anchor(N0 = t_abstype(get_content(abstype, Es)), E); + [V] -> + N0 = t_var(V) + end, + {Name,N0,format_type(N0, get_elem(type, Es), Opts)}. + +%% "A = t(), B = t()" is coalesced into "A = B = t()". +%% Names as B above are kept, but the formated string is empty. +coalesce_local_defs([], _Last) -> + []; +coalesce_local_defs([{Name,N0,TypeS} | L], Last) when Name =:= N0 -> + cld(L, [{Name,N0}], TypeS, Last); +coalesce_local_defs([{Name,N0,TypeS} | L], Last) -> + [local_def(N0, Name, TypeS, Last, L) | coalesce_local_defs(L, Last)]. + +cld([{Name,N0,TypeS} | L], Names, TypeS, Last) when Name =:= N0 -> + cld(L, [{Name,N0} | Names], TypeS, Last); +cld(L, Names0, TypeS, Last) -> + Names = [{_,Name0} | Names1] = lists:reverse(Names0), + NS = join([N || {N,_} <- Names], [" = "]), + ([local_def(Name0, NS, TypeS, Last, L) | + [local_def(N0, "", "", [], L) || {_,N0} <- Names1]] + ++ coalesce_local_defs(L, Last)). + +local_def(Name, NS, TypeS, Last, L) -> + [{typename,Name},{string,NS ++ TypeS ++ [Last || L =:= []]}]. + +%% join([], Sep) when is_list(Sep) -> +%% []; +join([H|T], Sep) -> + H ++ lists:append([Sep ++ X || X <- T]). + +%% Use the default formatting of EDoc, which creates references, and +%% then insert newlines and indentation according to erl_pp (the +%% (fast) Erlang pretty printer). +format_spec(Name, Type, #opts{pretty_print = erl_pp}=Opts) -> + try + L = t_clause(Name, Type), + O = pp_clause(Name, Type), + {R, ".\n"} = diaf(L, O, Opts), + R + catch _:_ -> + %% Example: "@spec ... -> record(a)" + format_spec(Name, Type, Opts#opts{pretty_print=default}) + end; +format_spec(Sep, Type, _Opts) -> + t_clause(Sep, Type). + +t_clause(Name, Type) -> + #xmlElement{content = [#xmlElement{name = 'fun', content = C}]} = Type, + [Name] ++ t_fun(C). + +pp_clause(Pre, Type) -> + Types = ot_utype([Type]), + Atom = lists:duplicate(iolist_size(Pre), $a), + L1 = erl_pp:attribute({attribute,0,spec,{{list_to_atom(Atom),0},[Types]}}), + "-spec " ++ L2 = lists:flatten(L1), + L3 = Pre ++ lists:nthtail(length(Atom), L2), + re:replace(L3, "\n ", "\n", [{return,list},global]). + +format_type(Name, Type, #opts{pretty_print = erl_pp}=Opts) -> + try + L = t_utype(Type), + O = pp_type(Name, Type), + {R, ".\n"} = diaf(L, O, Opts), + [" = "] ++ R + catch _:_ -> + %% Example: "t() = record(a)." + format_type(Name, Type, Opts#opts{pretty_print=default}) + end; +format_type(_Name, Type, _Opts) -> + [" = "] ++ t_utype(Type). + +pp_type(Prefix, Type) -> + Atom = list_to_atom(lists:duplicate(iolist_size(Prefix), $a)), + L1 = erl_pp:attribute({attribute,0,type,{Atom,ot_utype(Type),[]}}), + {L2,N} = case lists:dropwhile(fun(C) -> C =/= $: end, lists:flatten(L1)) of + ":: " ++ L3 -> {L3,9}; % compensation for extra "()" and ":" + "::\n" ++ L3 -> {"\n"++L3,6} + end, + Ss = lists:duplicate(N, $\s), + re:replace(L2, "\n"++Ss, "\n", [{return,list},global]). + +diaf(L, O0, Opts) -> + {R0, O} = diaf(L, [], O0, [], Opts), + R1 = rewrite_some_predefs(lists:reverse(R0)), + R = indentation(lists:flatten(R1)), + {R, O}. + +diaf([C | L], St, [C | O], R, Opts) -> + diaf(L, St, O, [[C] | R], Opts); +diaf(" "++L, St, O, R, Opts) -> + diaf(L, St, O, R, Opts); +diaf("", [Cs | St], O, R, Opts) -> + diaf(Cs, St, O, R, Opts); +diaf("", [], O, R, _Opts) -> + {R, O}; +diaf(L, St, " "++O, R, Opts) -> + diaf(L, St, O, [" " | R], Opts); +diaf(L, St, "\n"++O, R, Opts) -> + Ss = lists:takewhile(fun(C) -> C =:= $\s end, O), + diaf(L, St, lists:nthtail(length(Ss), O), ["\n"++Ss | R], Opts); +diaf([{seealso, HRef0, S0} | L], St, O0, R, Opts) -> + {S, O} = diaf(S0, app_fix(O0), Opts), + HRef = fix_mod_ref(HRef0, Opts), + diaf(L, St, O, [{seealso, HRef, S} | R], Opts); +diaf("="++L, St, "::"++O, R, Opts) -> + %% EDoc uses "=" for record field types; Dialyzer uses "::". Maybe + %% there should be an option for this, possibly affecting other + %% similar discrepancies. + diaf(L, St, O, ["=" | R], Opts); +diaf([Cs | L], St, O, R, Opts) -> + diaf(Cs, [L | St], O, R, Opts). + +rewrite_some_predefs(S) -> + xpredef(lists:flatten(S)). + +xpredef([]) -> + []; +xpredef("neg_integer()"++L) -> + ["integer() =< -1"] ++ xpredef(L); +xpredef("non_neg_integer()"++L) -> + ["integer() >= 0"] ++ xpredef(L); +xpredef("pos_integer()"++L) -> + ["integer() >= 1"] ++ xpredef(L); +xpredef([T | Es]) when is_tuple(T) -> + [T | xpredef(Es)]; +xpredef([E | Es]) -> + [[E] | xpredef(Es)]. + +indentation([]) -> + []; +indentation([$\n|L]) -> + [{br,[]}|indent(L)]; +indentation([T | Es]) when is_tuple(T) -> + [T | indentation(Es)]; +indentation([E|L]) -> + [[E]|indentation(L)]. + +indent([$\s|L]) -> + [{nbsp,[]}|indent(L)]; +indent(L) -> + indentation(L). + +app_fix(L) -> + try + {"//" ++ R1,L2} = app_fix(L, 1), + [App, Mod] = string:tokens(R1, "/"), + "//" ++ atom(App) ++ "/" ++ atom(Mod) ++ L2 + catch _:_ -> L + end. + +app_fix(L, I) -> % a bit slow + {L1, L2} = lists:split(I, L), + case erl_scan:tokens([], L1 ++ ". ", 1) of + {done, {ok,[{atom,_,Atom}|_],_}, _} -> {atom_to_list(Atom), L2}; + _ -> app_fix(L, I+1) + end. + +%% Remove the file suffix from module references. +fix_mod_ref(HRef, #opts{file_suffix = ""}) -> + HRef; +fix_mod_ref([{marker, S}]=HRef0, #opts{file_suffix = FS}) -> + {A, B} = lists:splitwith(fun(C) -> C =/= $# end, S), + case lists:member($:, A) of + true -> + HRef0; % should "save" most application references "http:" + false -> + case {lists:suffix(FS, A), B} of + {true, "#"++_} -> + [{marker, lists:sublist(A, length(A)-length(FS)) ++ B}]; + _ -> + HRef0 + end + end. + +see(E, Es) -> + case href(E) of + [] -> Es; + Ref -> + [{seealso, Ref, Es}] + end. + +href(E) -> + case get_attrval(href, E) of + "" -> []; + URI -> + [{marker, URI}] + end. + +atom(String) -> + io_lib:write_atom(list_to_atom(String)). + +t_name([E]) -> + N = get_attrval(name, E), + case get_attrval(module, E) of + "" -> atom(N); + M -> + S = atom(M) ++ ":" ++ atom(N), + case get_attrval(app, E) of + "" -> S; + A -> "//" ++ atom(A) ++ "/" ++ S + end + end. + +t_utype([E]) -> + t_utype_elem(E). + +t_utype_elem(E=#xmlElement{content = Es}) -> + case get_attrval(name, E) of + "" -> t_type(Es); + Name -> + T = t_type(Es), + case T of + [Name] -> T; % avoid generating "Foo::Foo" + T -> [Name] ++ ["::"] ++ T + end + end. + +t_type([E=#xmlElement{name = typevar}]) -> + t_var(E); +t_type([E=#xmlElement{name = atom}]) -> + t_atom(E); +t_type([E=#xmlElement{name = integer}]) -> + t_integer(E); +t_type([E=#xmlElement{name = range}]) -> + t_range(E); +t_type([E=#xmlElement{name = binary}]) -> + t_binary(E); +t_type([E=#xmlElement{name = float}]) -> + t_float(E); +t_type([#xmlElement{name = nil}]) -> + t_nil(); +t_type([#xmlElement{name = list, content = Es}]) -> + t_list(Es); +t_type([#xmlElement{name = nonempty_list, content = Es}]) -> + t_nonempty_list(Es); +t_type([#xmlElement{name = tuple, content = Es}]) -> + t_tuple(Es); +t_type([#xmlElement{name = 'fun', content = Es}]) -> + ["fun("] ++ t_fun(Es) ++ [")"]; +t_type([E = #xmlElement{name = record, content = Es}]) -> + t_record(E, Es); +t_type([E = #xmlElement{name = abstype, content = Es}]) -> + t_abstype(E, Es); +t_type([#xmlElement{name = union, content = Es}]) -> + t_union(Es). + +t_var(E) -> + [get_attrval(name, E)]. + +t_atom(E) -> + [get_attrval(value, E)]. + +t_integer(E) -> + [get_attrval(value, E)]. + +t_range(E) -> + [get_attrval(value, E)]. + +t_binary(E) -> + [get_attrval(value, E)]. + +t_float(E) -> + [get_attrval(value, E)]. + +t_nil() -> + ["[]"]. + +t_list(Es) -> + ["["] ++ t_utype(get_elem(type, Es)) ++ ["]"]. + +t_nonempty_list(Es) -> + ["["] ++ t_utype(get_elem(type, Es)) ++ [", ...]"]. + +t_tuple(Es) -> + ["{"] ++ seq(fun t_utype_elem/1, Es, ["}"]). + +t_fun(Es) -> + ["("] ++ seq(fun t_utype_elem/1, get_content(argtypes, Es), + [") -> "] ++ t_utype(get_elem(type, Es))). + +t_record(E, Es) -> + Name = ["#"] ++ t_type(get_elem(atom, Es)), + case get_elem(field, Es) of + [] -> + see(E, [Name, "{}"]); + Fs -> + see(E, Name) ++ ["{"] ++ seq(fun t_field/1, Fs, ["}"]) + end. + +t_field(#xmlElement{content = Es}) -> + t_type(get_elem(atom, Es)) ++ [" = "] ++ t_utype(get_elem(type, Es)). + +t_abstype(E, Es) -> + Name = t_name(get_elem(erlangName, Es)), + case get_elem(type, Es) of + [] -> + see(E, [Name, "()"]); + Ts -> + see(E, [Name]) ++ ["("] ++ seq(fun t_utype_elem/1, Ts, [")"]) + end. + +t_abstype(Es) -> + ([t_name(get_elem(erlangName, Es)), "("] + ++ seq(fun t_utype_elem/1, get_elem(type, Es), [")"])). + +t_union(Es) -> + seq(fun t_utype_elem/1, Es, " | ", []). + +seq(F, Es, Tail) -> + seq(F, Es, ", ", Tail). + +seq(F, [E], _Sep, Tail) -> + F(E) ++ Tail; +seq(F, [E | Es], Sep, Tail) -> + F(E) ++ [Sep] ++ seq(F, Es, Sep, Tail); +seq(_F, [], _Sep, Tail) -> + Tail. + +get_elem(Name, [#xmlElement{name = Name} = E | Es]) -> + [E | get_elem(Name, Es)]; +get_elem(Name, [_ | Es]) -> + get_elem(Name, Es); +get_elem(_, []) -> + []. + +get_attr(Name, [#xmlAttribute{name = Name} = A | As]) -> + [A | get_attr(Name, As)]; +get_attr(Name, [_ | As]) -> + get_attr(Name, As); +get_attr(_, []) -> + []. + +get_attrval(Name, #xmlElement{attributes = As}) -> + case get_attr(Name, As) of + [#xmlAttribute{value = V}] -> + V; + [] -> "" + end. + +get_content(Name, Es) -> + case get_elem(Name, Es) of + [#xmlElement{content = Es1}] -> + Es1; + [] -> [] + end. + +overview(_, _Options) -> []. + +package(_, _Options) -> []. + +type(_) -> []. + +%% --------------------------------------------------------------------- + +ot_utype([E]) -> + ot_utype_elem(E). + +ot_utype_elem(E=#xmlElement{content = Es}) -> + case get_attrval(name, E) of + "" -> ot_type(Es); + N -> + Name = {var,0,list_to_atom(N)}, + T = ot_type(Es), + case T of + Name -> T; + T -> {ann_type,0,[Name, T]} + end + end. + +ot_type([E=#xmlElement{name = typevar}]) -> + ot_var(E); +ot_type([E=#xmlElement{name = atom}]) -> + ot_atom(E); +ot_type([E=#xmlElement{name = integer}]) -> + ot_integer(E); +ot_type([E=#xmlElement{name = range}]) -> + ot_range(E); +ot_type([E=#xmlElement{name = binary}]) -> + ot_binary(E); +ot_type([E=#xmlElement{name = float}]) -> + ot_float(E); +ot_type([#xmlElement{name = nil}]) -> + ot_nil(); +ot_type([#xmlElement{name = list, content = Es}]) -> + ot_list(Es); +ot_type([#xmlElement{name = nonempty_list, content = Es}]) -> + ot_nonempty_list(Es); +ot_type([#xmlElement{name = tuple, content = Es}]) -> + ot_tuple(Es); +ot_type([#xmlElement{name = 'fun', content = Es}]) -> + ot_fun(Es); +ot_type([#xmlElement{name = record, content = Es}]) -> + ot_record(Es); +ot_type([#xmlElement{name = abstype, content = Es}]) -> + ot_abstype(Es); +ot_type([#xmlElement{name = union, content = Es}]) -> + ot_union(Es). + +ot_var(E) -> + {var,0,list_to_atom(get_attrval(name, E))}. + +ot_atom(E) -> + {ok, [Atom], _} = erl_scan:string(get_attrval(value, E), 0), + Atom. + +ot_integer(E) -> + {integer,0,list_to_integer(get_attrval(value, E))}. + +ot_range(E) -> + [I1, I2] = string:tokens(get_attrval(value, E), "."), + {type,0,range,[{integer,0,list_to_integer(I1)}, + {integer,0,list_to_integer(I2)}]}. + +ot_binary(E) -> + {Base, Unit} = + case string:tokens(get_attrval(value, E), ",:*><") of + [] -> + {0, 0}; + ["_",B] -> + {list_to_integer(B), 0}; + ["_","_",U] -> + {0, list_to_integer(U)}; + ["_",B,_,"_",U] -> + {list_to_integer(B), list_to_integer(U)} + end, + {type,0,binary,[{integer,0,Base},{integer,0,Unit}]}. + +ot_float(E) -> + {float,0,list_to_float(get_attrval(value, E))}. + +ot_nil() -> + {nil,0}. + +ot_list(Es) -> + {type,0,list,[ot_utype(get_elem(type, Es))]}. + +ot_nonempty_list(Es) -> + {type,0,nonempty_list,[ot_utype(get_elem(type, Es))]}. + +ot_tuple(Es) -> + {type,0,tuple,[ot_utype_elem(E) || E <- Es]}. + +ot_fun(Es) -> + Range = ot_utype(get_elem(type, Es)), + Args = [ot_utype_elem(A) || A <- get_content(argtypes, Es)], + {type,0,'fun',[{type,0,product,Args},Range]}. + +ot_record(Es) -> + {type,0,record,[ot_type(get_elem(atom, Es)) | + [ot_field(F) || F <- get_elem(field, Es)]]}. + +ot_field(#xmlElement{content = Es}) -> + {type,0,field_type, + [ot_type(get_elem(atom, Es)), ot_utype(get_elem(type, Es))]}. + +ot_abstype(Es) -> + ot_name(get_elem(erlangName, Es), + [ot_utype_elem(Elem) || Elem <- get_elem(type, Es)]). + +ot_union(Es) -> + {type,0,union,[ot_utype_elem(E) || E <- Es]}. + +ot_name(Es, T) -> + case ot_name(Es) of + [Mod, ":", Atom] -> + {remote_type,0,[{atom,0,list_to_atom(Mod)}, + {atom,0,list_to_atom(Atom)},T]}; + "tuple" when T =:= [] -> + {type,0,tuple,any}; + Atom -> + {type,0,list_to_atom(Atom),T} + end. + +ot_name([E]) -> + Atom = get_attrval(name, E), + case get_attrval(module, E) of + "" -> Atom; + M -> + case get_attrval(app, E) of + "" -> + [M, ":", Atom]; + A -> + ["//"++A++"/" ++ M, ":", Atom] % EDoc only! + end + end. + +%% Returns exactly those annotations that can be referred to. Note +%% that a Dialyzer type/spec (currently) can have more annotations +%% than can be represented by EDoc types. Note also that edoc_dia +%% has annotated all type variables with themselves. +typespec_annos([]) -> [?NL]; +typespec_annos([_|Es]) -> + annotations(clause_annos(Es)). + +clause_annos(Es) -> + [annos(get_elem(type, Es)), local_defs_annos(get_elem(localdef, Es))]. + +typedef_annos(Es) -> + annotations([(case get_elem(type, Es) of + [] -> []; + T -> annos(T) + end + ++ lists:flatmap(fun annos_elem/1, + get_content(argtypes, Es))), + local_defs_annos(get_elem(localdef, Es))]). + +local_defs_annos(Es) -> + lists:flatmap(fun localdef_annos/1, Es). + +localdef_annos(#xmlElement{content = Es}) -> + annos(get_elem(type, Es)). + +annotations(AnnoL) -> + Annos = lists:usort(lists:flatten(AnnoL)), + margin(2, Annos). + +margin(N, L) -> + lists:append([[?IND(N),E] || E <- L]) ++ [?IND(N-2)]. + +annos([E]) -> + annos_elem(E). + +annos_elem(E=#xmlElement{content = Es}) -> + case get_attrval(name, E) of + "" -> annos_type(Es); + "..." -> annos_type(Es); % compensate for a kludge in edoc_dia.erl + N -> + [{anno,[N]} | annos_type(Es)] + end. + +annos_type([#xmlElement{name = list, content = Es}]) -> + annos(get_elem(type, Es)); +annos_type([#xmlElement{name = nonempty_list, content = Es}]) -> + annos(get_elem(type, Es)); +annos_type([#xmlElement{name = tuple, content = Es}]) -> + lists:flatmap(fun annos_elem/1, Es); +annos_type([#xmlElement{name = 'fun', content = Es}]) -> + (annos(get_elem(type, Es)) + ++ lists:flatmap(fun annos_elem/1, get_content(argtypes, Es))); +annos_type([#xmlElement{name = record, content = Es}]) -> + lists:append([annos(get_elem(type, Es1)) || + #xmlElement{content = Es1} <- get_elem(field, Es)]); +annos_type([#xmlElement{name = abstype, content = Es}]) -> + lists:flatmap(fun annos_elem/1, get_elem(type, Es)); +annos_type([#xmlElement{name = union, content = Es}]) -> + lists:flatmap(fun annos_elem/1, Es); +annos_type([E=#xmlElement{name = typevar}]) -> + annos_elem(E); +annos_type(_) -> + []. diff --git a/lib/erl_docgen/vsn.mk b/lib/erl_docgen/vsn.mk index 0bc01f7d49..fb0f5ca0cd 100644 --- a/lib/erl_docgen/vsn.mk +++ b/lib/erl_docgen/vsn.mk @@ -1 +1 @@ -ERL_DOCGEN_VSN = 0.2.2 +ERL_DOCGEN_VSN = 0.2.3 diff --git a/lib/erl_interface/doc/src/notes.xml b/lib/erl_interface/doc/src/notes.xml index ff89802599..de5ba61938 100644 --- a/lib/erl_interface/doc/src/notes.xml +++ b/lib/erl_interface/doc/src/notes.xml @@ -30,6 +30,56 @@ </header> <p>This document describes the changes made to the Erl_interface application.</p> +<section><title>Erl_Interface 3.7.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + erl_call: remove get_hostent</p> + <p> + get_hostent does not properly handle IPv4 addresses on + little endian platforms and fails with hostnames + beginning with a number. Remove get_hostent and use + ei_gethostbyname directly since gethostbyname supports + IPv4 addresses.</p> + <p> + (Thanks to Michael Santos)</p> + <p> + Own Id: OTP-8890</p> + </item> + <item> + <p> teach ei_x_format to handle unary - and + (Thanks to + Steve Vinoski)</p> + <p> + Own Id: OTP-8891</p> + </item> + <item> + <p>Fix zero byte allocation in registry. (Thanks to + Michael Santos)</p> + <p> + Own Id: OTP-8893</p> + </item> + <item> + <p> Check the length of the node name to prevent an + overflow. Memory error control of ei_alloc_big. (Thanks + to Michael Santos) </p> + <p> + Own Id: OTP-8943</p> + </item> + <item> + <p> + erl_term_len() in erl_interface could returned too large + values for integers (since R14B) and too small values for + refs (since R9B).</p> + <p> + Own Id: OTP-8945</p> + </item> + </list> + </section> + +</section> + <section><title>Erl_Interface 3.7.1.1</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/erl_interface/src/legacy/erl_marshal.c b/lib/erl_interface/src/legacy/erl_marshal.c index 5084c65230..a6c2f64dd0 100644 --- a/lib/erl_interface/src/legacy/erl_marshal.c +++ b/lib/erl_interface/src/legacy/erl_marshal.c @@ -511,29 +511,28 @@ static int erl_term_len_helper(ETERM *ep, int dist) case ERL_INTEGER: i = ep->uval.ival.i; - if ((i > ERL_MAX) || (i < ERL_MIN)) len = 7; - else if ((i < 256) && (i >= 0)) len = 2; + if ((i < 256) && (i >= 0)) len = 2; else len = 5; break; case ERL_U_INTEGER: u = ep->uval.uival.u; - if (u > ERL_MAX) len = 7; + if ((int)u < 0) len = 7; else if (u < 256) len = 2; else len = 5; break; case ERL_LONGLONG: l = ep->uval.llval.i; - if ((l > ((long long) ERL_MAX)) || - (l < ((long long) ERL_MIN))) len = 11; + if ((l > ((long long) INT_MAX)) || + (l < ((long long) INT_MIN))) len = 11; else if ((l < 256) && (l >= 0)) len = 2; else len = 5; break; case ERL_U_LONGLONG: ul = ep->uval.ullval.u; - if (ul > ((unsigned long long) ERL_MAX)) len = 11; + if (ul > ((unsigned long long) INT_MAX)) len = 11; else if (ul < 256) len = 2; else len = 5; break; @@ -546,12 +545,7 @@ static int erl_term_len_helper(ETERM *ep, int dist) case ERL_REF: i = strlen((char *)ERL_REF_NODE(ep)); - if (dist >= 4 && ERL_REF_LEN(ep) > 1) { - len = 1 + 2 + (i+3) + 1 + ERL_REF_LEN(ep) * 4; - } else { - /* 1 + N + 4 + 1 where N = 3 + strlen */ - len = 9 + i; - } + len = 1 + 2 + (i+3) + 1 + ERL_REF_LEN(ep) * 4; break; case ERL_PORT: diff --git a/lib/erl_interface/test/ei_decode_SUITE.erl b/lib/erl_interface/test/ei_decode_SUITE.erl index 09a37409f2..524a04a3b4 100644 --- a/lib/erl_interface/test/ei_decode_SUITE.erl +++ b/lib/erl_interface/test/ei_decode_SUITE.erl @@ -232,7 +232,7 @@ send_integers(P) -> ?line send_term_as_binary(P, 16#80000000), % SMALL_BIG_EXT new smallest pos(*) ?line send_term_as_binary(P,-16#80000001), % SMALL_BIG_EXT new largest neg (*) - case erlang:system_info(wordsize) of + case erlang:system_info({wordsize,external}) of 4 -> ?line send_term_as_binary(P, 16#80000000),% SMALL_BIG_EXT u32 ?line send_term_as_binary(P, 16#ffffffff),% SMALL_BIG_EXT largest u32 diff --git a/lib/erl_interface/test/erl_eterm_SUITE_data/eterm_test.c b/lib/erl_interface/test/erl_eterm_SUITE_data/eterm_test.c index f273efd532..80d7f69520 100644 --- a/lib/erl_interface/test/erl_eterm_SUITE_data/eterm_test.c +++ b/lib/erl_interface/test/erl_eterm_SUITE_data/eterm_test.c @@ -1,7 +1,7 @@ /* * %CopyrightBegin% * - * Copyright Ericsson AB 1997-2009. All Rights Reserved. + * Copyright Ericsson AB 1997-2010. All Rights Reserved. * * The contents of this file are subject to the Erlang Public License, * Version 1.1, (the "License"); you may not use this file except in @@ -98,14 +98,30 @@ static void encode_decode(ETERM* original, const char* text) { static unsigned char encoded[16*1024]; ETERM* new_terms; - int bytes = erl_encode(original, encoded); + ETERM* head; + int bytes; + int len; + + /* If a list, check the elements one by one first */ + head = erl_hd(original); + if (head != NULL) { + encode_decode(head, "CAR"); + encode_decode(erl_tl(original), "CDR"); + } + bytes = erl_encode(original, encoded); if (bytes == 0) { fail("failed to encode terms"); } else if (bytes > sizeof(encoded)) { fail("encoded terms buffer overflow"); } + else if (bytes != (len=erl_term_len(original))) { + fprintf(stderr, "bytes(%d) != len(%d) for term ", bytes, len); + erl_print_term(stderr, original); + fprintf(stderr, " [%s]\r\n", text); + fail("erl_encode and erl_term_len do not agree"); + } else if ((new_terms = erl_decode(encoded)) == NULL) { fail("failed to decode terms"); } diff --git a/lib/erl_interface/vsn.mk b/lib/erl_interface/vsn.mk index 6c664959a3..ffda886553 100644 --- a/lib/erl_interface/vsn.mk +++ b/lib/erl_interface/vsn.mk @@ -1 +1 @@ -EI_VSN = 3.7.1.1 +EI_VSN = 3.7.2 diff --git a/lib/et/doc/src/notes.xml b/lib/et/doc/src/notes.xml index 4ce7548414..cd4787c5e7 100644 --- a/lib/et/doc/src/notes.xml +++ b/lib/et/doc/src/notes.xml @@ -36,6 +36,21 @@ one section in this document. The title of each section is the version number of <c>Event Tracer (ET)</c>.</p> +<section><title>ET 1.4.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix error when module et was used in et_selector + trace patterns. </p> + <p> + Own Id: OTP-8904</p> + </item> + </list> + </section> + +</section> + <section><title>ET 1.4.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/et/src/et_selector.erl b/lib/et/src/et_selector.erl index f39f21aa70..c8e9c907b2 100644 --- a/lib/et/src/et_selector.erl +++ b/lib/et/src/et_selector.erl @@ -115,13 +115,13 @@ change_pattern({Mod, Pattern}) when is_atom(Mod) -> old_ctp({Mod, _Fun, Args}) -> case Mod of - et -> ignore; + et -> {ok, ignore}; _ -> dbg:ctp({Mod, report_event, Args}) end. old_tp({Mod, _Fun, Args}, Pattern) -> case Mod of - et -> ignore; + et -> {ok, ignore}; _ -> dbg:tp({Mod, report_event, Args}, Pattern) end. diff --git a/lib/et/vsn.mk b/lib/et/vsn.mk index b5b7fa52f4..d7cfd7bc84 100644 --- a/lib/et/vsn.mk +++ b/lib/et/vsn.mk @@ -1 +1 @@ -ET_VSN = 1.4.1 +ET_VSN = 1.4.2 diff --git a/lib/eunit/doc/src/Makefile b/lib/eunit/doc/src/Makefile index 19be96d763..2cdc579275 100644 --- a/lib/eunit/doc/src/Makefile +++ b/lib/eunit/doc/src/Makefile @@ -146,7 +146,7 @@ debug opt: clean clean_docs: rm -rf $(HTMLDIR)/* rm -f $(MAN3DIR)/* - rm -f $(XML_CHAPTER_FILES) *.html + rm -f $(XML_REF3_FILES) $(XML_CHAPTER_FILES) *.html rm -f $(TOP_PDF_FILE) $(TOP_PDF_FILE:%.pdf=%.fo) rm -f errs core *~ diff --git a/lib/eunit/doc/src/notes.xml b/lib/eunit/doc/src/notes.xml index 974ba1db4e..1717dd7988 100644 --- a/lib/eunit/doc/src/notes.xml +++ b/lib/eunit/doc/src/notes.xml @@ -32,6 +32,21 @@ </header> <p>This document describes the changes made to the EUnit application.</p> +<section><title>Eunit 2.1.6</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix format_man_pages so it handles all man sections + and remove warnings/errors in various man pages. </p> + <p> + Own Id: OTP-8600</p> + </item> + </list> + </section> + +</section> + <section><title>Eunit 2.1.5</title> <section><title>Improvements and New Features</title> diff --git a/lib/eunit/src/eunit.erl b/lib/eunit/src/eunit.erl index 59084a52fb..4a86a108cf 100644 --- a/lib/eunit/src/eunit.erl +++ b/lib/eunit/src/eunit.erl @@ -16,7 +16,7 @@ %% $Id: eunit.erl 339 2009-04-05 14:10:47Z rcarlsson $ %% %% @copyright 2004-2009 Micka�l R�mond, Richard Carlsson -%% @author Micka�l R�mond <[email protected]> +%% @author Mickaël Rémond <[email protected]> %% [http://www.process-one.net/] %% @author Richard Carlsson <[email protected]> %% [http://user.it.uu.se/~richardc/] diff --git a/lib/eunit/src/eunit_surefire.erl b/lib/eunit/src/eunit_surefire.erl index aeda31d251..eb994a990a 100644 --- a/lib/eunit/src/eunit_surefire.erl +++ b/lib/eunit/src/eunit_surefire.erl @@ -15,7 +15,7 @@ %% %% $Id: $ %% -%% @author Micka�l R�mond <[email protected]> +%% @author Mickaël Rémond <[email protected]> %% @copyright 2009 Micka�l R�mond, Paul Guyot %% @see eunit %% @doc Surefire reports for EUnit (Format used by Maven and Atlassian diff --git a/lib/eunit/vsn.mk b/lib/eunit/vsn.mk index 3bfa9c8000..e1965630e3 100644 --- a/lib/eunit/vsn.mk +++ b/lib/eunit/vsn.mk @@ -1 +1 @@ -EUNIT_VSN = 2.1.5 +EUNIT_VSN = 2.1.6 diff --git a/lib/hipe/cerl/erl_bif_types.erl b/lib/hipe/cerl/erl_bif_types.erl index 696414eb7a..ed5bf03804 100644 --- a/lib/hipe/cerl/erl_bif_types.erl +++ b/lib/hipe/cerl/erl_bif_types.erl @@ -192,7 +192,7 @@ type(binary, referenced_byte_size, 1, Xs) -> type(code, add_path, 1, Xs) -> strict(arg_types(code, add_path, 1), Xs, fun (_) -> - t_sup(t_boolean(), + t_sup(t_atom('true'), t_tuple([t_atom('error'), t_atom('bad_directory')])) end); type(code, add_patha, 1, Xs) -> @@ -1992,28 +1992,18 @@ type(ets, update_counter, 3, Xs) -> type(ets, update_element, 3, Xs) -> strict(arg_types(ets, update_element, 3), Xs, fun (_) -> t_boolean() end); %%-- file --------------------------------------------------------------------- -type(file, close, 1, Xs) -> - strict(arg_types(file, close, 1), Xs, fun (_) -> t_file_return() end); -type(file, delete, 1, Xs) -> - strict(arg_types(file, delete, 1), Xs, fun (_) -> t_file_return() end); -type(file, get_cwd, 0, _) -> - t_sup(t_tuple([t_atom('ok'), t_string()]), - t_tuple([t_atom('error'), t_file_posix_error()])); -type(file, make_dir, 1, Xs) -> - strict(arg_types(file, make_dir, 1), Xs, fun (_) -> t_file_return() end); -type(file, read_file, 1, Xs) -> - strict(arg_types(file, read_file, 1), Xs, - fun (_) -> - t_sup([t_tuple([t_atom('ok'), t_binary()]), - t_tuple([t_atom('error'), t_file_posix_error()])]) - end); -type(file, set_cwd, 1, Xs) -> - strict(arg_types(file, set_cwd, 1), Xs, - fun (_) -> t_sup(t_atom('ok'), - t_tuple([t_atom('error'), t_file_posix_error()])) - end); -type(file, write_file, 2, Xs) -> - strict(arg_types(file, write_file, 2), Xs, fun (_) -> t_file_return() end); +type(file, native_name_encoding, 0, _) -> + t_file_encoding(); +%%-- prim_file ---------------------------------------------------------------- +type(prim_file, internal_name2native, 1, Xs) -> + strict(arg_types(prim_file, internal_name2native, 1), Xs, + fun (_) -> t_binary() end); +type(prim_file, internal_native2name, 1, Xs) -> + strict(arg_types(prim_file, internal_native2name, 1), Xs, + fun (_) -> t_prim_file_name() end); +type(prim_file, internal_normalize_utf8, 1, Xs) -> + strict(arg_types(prim_file, internal_normalize_utf8, 1), Xs, + fun (_) -> t_binary() end); %%-- gen_tcp ------------------------------------------------------------------ %% NOTE: All type information for this module added to avoid loss of precision type(gen_tcp, accept, 1, Xs) -> @@ -3357,7 +3347,7 @@ arg_types(code, all_loaded, 0) -> arg_types(code, compiler_dir, 0) -> []; arg_types(code, del_path, 1) -> - [t_sup(t_string(), t_atom())]; % OBS: doc differs from add_path/1 - why? + [t_sup(t_string(), t_atom())]; % OBS: differs from add_path/1 arg_types(code, delete, 1) -> [t_atom()]; arg_types(code, ensure_loaded, 1) -> @@ -3405,7 +3395,7 @@ arg_types(code, replace_path, 2) -> arg_types(code, root_dir, 0) -> []; arg_types(code, set_path, 1) -> - [t_string()]; + [t_list(t_string())]; arg_types(code, soft_purge, 1) -> arg_types(code, delete, 1); arg_types(code, stick_mod, 1) -> @@ -4200,22 +4190,15 @@ arg_types(ets, update_element, 3) -> PosValue = t_tuple([t_integer(), t_any()]), [t_tab(), t_any(), t_sup(PosValue, t_list(PosValue))]; %%------- file ---------------------------------------------------------------- -arg_types(file, close, 1) -> - [t_file_io_device()]; -arg_types(file, delete, 1) -> - [t_file_name()]; -arg_types(file, get_cwd, 0) -> +arg_types(file, native_name_encoding, 0) -> []; -arg_types(file, make_dir, 1) -> - [t_file_name()]; -arg_types(file, read_file, 1) -> - [t_file_name()]; -arg_types(file, set_cwd, 1) -> - [t_file_name()]; -arg_types(file, write, 2) -> - [t_file_io_device(), t_iodata()]; -arg_types(file, write_file, 2) -> - [t_file_name(), t_sup(t_binary(), t_list())]; +%%-- prim_file ---------------------------------------------------------------- +arg_types(prim_file, internal_name2native, 1) -> + [t_prim_file_name()]; +arg_types(prim_file, internal_native2name, 1) -> + [t_binary()]; +arg_types(prim_file, internal_normalize_utf8, 1) -> + [t_binary()]; %%------- gen_tcp ------------------------------------------------------------- arg_types(gen_tcp, accept, 1) -> [t_socket()]; @@ -4647,7 +4630,7 @@ t_httppacket() -> t_HttpHeader(), t_atom('http_eoh'), t_HttpError()]). t_endian() -> - t_sup([t_atom('big'), t_atom('little')]). + t_sup(t_atom('big'), t_atom('little')). %% ===================================================================== %% Types for the binary module @@ -4678,16 +4661,16 @@ t_HttpRequest() -> t_tuple([t_atom('http_request'), t_HttpMethod(), t_HttpUri(), t_HttpVersion()]). t_HttpResponse() -> - t_tuple([t_atom('http_response'), t_HttpVersion(), t_integer(), t_string()]). + t_tuple([t_atom('http_response'), t_HttpVersion(), t_integer(), t_HttpString()]). t_HttpHeader() -> - t_tuple([t_atom('http_header'), t_integer(), t_HttpField(), t_any(), t_string()]). + t_tuple([t_atom('http_header'), t_integer(), t_HttpField(), t_any(), t_HttpString()]). t_HttpError() -> - t_tuple([t_atom('http_error'), t_string()]). + t_tuple([t_atom('http_error'), t_HttpString()]). t_HttpMethod() -> - t_sup(t_HttpMethodAtom(), t_string()). + t_sup(t_HttpMethodAtom(), t_HttpString()). t_HttpMethodAtom() -> t_atoms(['OPTIONS', 'GET', 'HEAD', 'POST', 'PUT', 'DELETE', 'TRACE']). @@ -4696,18 +4679,18 @@ t_HttpUri() -> t_sup([t_atom('*'), t_tuple([t_atom('absoluteURI'), t_sup(t_atom('http'), t_atom('https')), - t_string(), + t_HttpString(), t_sup(t_non_neg_integer(), t_atom('undefined')), - t_string()]), - t_tuple([t_atom('scheme'), t_string(), t_string()]), - t_tuple([t_atom('abs_path'), t_string()]), - t_string()]). + t_HttpString()]), + t_tuple([t_atom('scheme'), t_HttpString(), t_HttpString()]), + t_tuple([t_atom('abs_path'), t_HttpString()]), + t_HttpString()]). t_HttpVersion() -> t_tuple([t_non_neg_integer(), t_non_neg_integer()]). t_HttpField() -> - t_sup(t_HttpFieldAtom(), t_string()). + t_sup(t_HttpFieldAtom(), t_HttpString()). t_HttpFieldAtom() -> t_atoms(['Cache-Control', 'Connection', 'Date', 'Pragma', 'Transfer-Encoding', @@ -4724,6 +4707,9 @@ t_HttpFieldAtom() -> 'Set-Cookie', 'Set-Cookie2', 'X-Forwarded-For', 'Cookie', 'Keep-Alive', 'Proxy-Connection']). +t_HttpString() -> + t_sup(t_string(),t_binary()). + %% ===================================================================== %% These are used for the built-in functions of 'code' %% ===================================================================== @@ -4988,68 +4974,12 @@ t_ets_info_items() -> t_atom('type')]). %% ===================================================================== -%% These are used for the built-in functions of 'file' +%% These are used for the built-in functions of 'prim_file' %% ===================================================================== -t_file_io_device() -> - t_sup(t_pid(), t_tuple([t_atom('file_descriptor'), t_atom(), t_any()])). - -t_file_name() -> - t_sup([t_atom(), - t_string(), - %% DeepList = [char() | atom() | DeepList] -- approximation below - t_list(t_sup([t_atom(), t_string(), t_list()]))]). - -t_file_open_option() -> - t_sup([t_atom('read'), - t_atom('write'), - t_atom('append'), - t_atom('raw'), - t_atom('binary'), - t_atom('delayed_write'), - t_atom('read_ahead'), - t_atom('compressed'), - t_tuple([t_atom('delayed_write'), - t_pos_integer(), t_non_neg_integer()]), - t_tuple([t_atom('read_ahead'), t_pos_integer()])]). - -%% This lists all Posix errors that can occur in file:*/* functions -t_file_posix_error() -> - t_sup([t_atom('eacces'), - t_atom('eagain'), - t_atom('ebadf'), - t_atom('ebusy'), - t_atom('edquot'), - t_atom('eexist'), - t_atom('efault'), - t_atom('efbig'), - t_atom('eintr'), - t_atom('einval'), - t_atom('eio'), - t_atom('eisdir'), - t_atom('eloop'), - t_atom('emfile'), - t_atom('emlink'), - t_atom('enametoolong'), - t_atom('enfile'), - t_atom('enodev'), - t_atom('enoent'), - t_atom('enomem'), - t_atom('enospc'), - t_atom('enotblk'), - t_atom('enotdir'), - t_atom('enotsup'), - t_atom('enxio'), - t_atom('eperm'), - t_atom('epipe'), - t_atom('erofs'), - t_atom('espipe'), - t_atom('esrch'), - t_atom('estale'), - t_atom('exdev')]). - -t_file_return() -> - t_sup(t_atom('ok'), t_tuple([t_atom('error'), t_file_posix_error()])). +t_prim_file_name() -> + t_sup([t_string(), + t_binary()]). %% ===================================================================== %% These are used for the built-in functions of 'gen_tcp' @@ -5251,7 +5181,12 @@ t_ML() -> % a binary or a possibly deep list of integers or binaries t_sup(t_list(t_sup([t_integer(), t_binary(), t_list()])), t_binary()). t_encoding() -> - t_atoms(['latin1', 'unicode', 'utf8', 'utf16', 'utf32']). + t_sup([t_atoms(['latin1', 'unicode', 'utf8', 'utf16', 'utf32']), + t_tuple([t_atom('utf16'), t_endian()]), + t_tuple([t_atom('utf32'), t_endian()])]). + +t_file_encoding() -> + t_atoms(['latin1', 'utf8']). t_encoding_a2b() -> % for the 2nd arg of atom_to_binary/2 and binary_to_atom/2 t_atoms(['latin1', 'unicode', 'utf8']). diff --git a/lib/hipe/doc/src/notes.xml b/lib/hipe/doc/src/notes.xml index cf30db0482..c188db483d 100644 --- a/lib/hipe/doc/src/notes.xml +++ b/lib/hipe/doc/src/notes.xml @@ -30,6 +30,21 @@ </header> <p>This document describes the changes made to HiPE.</p> +<section><title>Hipe 3.7.8</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Compiler warnings were eliminated.</p> + <p> + Own Id: OTP-8855</p> + </item> + </list> + </section> + +</section> + <section><title>Hipe 3.7.7</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/hipe/vsn.mk b/lib/hipe/vsn.mk index 8e421ce9b2..fa9dc91ff0 100644 --- a/lib/hipe/vsn.mk +++ b/lib/hipe/vsn.mk @@ -1 +1 @@ -HIPE_VSN = 3.7.7 +HIPE_VSN = 3.7.8 diff --git a/lib/inets/doc/src/ftp.xml b/lib/inets/doc/src/ftp.xml index 25dfe716fc..ca902d8d9d 100644 --- a/lib/inets/doc/src/ftp.xml +++ b/lib/inets/doc/src/ftp.xml @@ -107,7 +107,7 @@ <tag>{mode, Mode}</tag> <item> <marker id="mode"></marker> - <p>Mode = <c>active | passive</c> </p>> + <p>Mode = <c>active | passive</c> </p> <p>Default is <c>passive</c>. </p> </item> diff --git a/lib/inets/doc/src/http_client.xml b/lib/inets/doc/src/http_client.xml index ea8053cafa..672ea3fa98 100644 --- a/lib/inets/doc/src/http_client.xml +++ b/lib/inets/doc/src/http_client.xml @@ -1,4 +1,4 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE chapter SYSTEM "chapter.dtd"> <chapter> @@ -57,7 +57,7 @@ [{inets, [{services, [{httpc, PropertyList}]}]}] </pre> <p>For valid properties see - <seealso marker="http">httpc(3)</seealso>. </p> + <seealso marker="httpc">httpc(3)</seealso>. </p> </section> <section> diff --git a/lib/inets/doc/src/httpc.xml b/lib/inets/doc/src/httpc.xml index 9c8df28fec..c20358178b 100644 --- a/lib/inets/doc/src/httpc.xml +++ b/lib/inets/doc/src/httpc.xml @@ -1,4 +1,4 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd"> <erlref> @@ -172,7 +172,8 @@ ssl_options() = {verify, code()} | {autoredirect, boolean()} | {proxy_auth, {userstring(), passwordstring()}} | {version, http_version()} | - {relaxed, boolean()}</v> + {relaxed, boolean()} | + {url_encode, boolean()}</v> <v>timeout() = integer() >= 0 | infinity</v> <v>Options = options()</v> <v>options() = [option()]</v> @@ -276,6 +277,11 @@ ssl_options() = {verify, code()} | <p>Defaults to <c>false</c>. </p> </item> + <tag><c><![CDATA[url_encode]]></c></tag> + <item> + <p>Will apply Percent-encoding, also known as URL encoding on the URL.</p> + <p>Defaults to <c>false</c>. </p> + </item> </taglist> <p>Option (<c>option()</c>) details: </p> @@ -342,7 +348,7 @@ ssl_options() = {verify, code()} | <p>Socket options to be used for this and subsequent request(s). </p> <p>Overrides any value set by the - <seealso marker="set_options">set_options</seealso> + <seealso marker="#set_options">set_options</seealso> function. </p> <p>Note that the validity of the options are <em>not</em> checked in any way. </p> @@ -632,4 +638,3 @@ apply(Module, Function, [ReplyInfo | Args]) </section> </erlref> - diff --git a/lib/inets/doc/src/httpd.xml b/lib/inets/doc/src/httpd.xml index 847605fe93..62f4e18f82 100644 --- a/lib/inets/doc/src/httpd.xml +++ b/lib/inets/doc/src/httpd.xml @@ -1,4 +1,4 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE erlref SYSTEM "erlref.dtd"> <erlref> @@ -421,7 +421,7 @@ bytes Beware of trailing space in Replacement that will be used. If you must have a space in Re use e.g the character encoding - <code>\040</code> see <seealso marker="re">re(3)</seealso>. + <code>\040</code> see <seealso marker="stdlib:re">re(3)</seealso>. </item> <tag>{directory_index, [string()]}</tag> @@ -931,6 +931,10 @@ bytes connection }). </code> + + <p>To acess the record in your callback-module use </p> + <code> -include_lib("inets/include/httpd.hrl"). </code> + <p>The fields of the <c>mod</c> record has the following meaning: </p> <taglist> @@ -978,10 +982,10 @@ bytes <c>parsed_header</c> contains all HTTP header fields from the HTTP-request stored in a list as key-value tuples. See RFC 2616 for a listing of all header fields. For example the date field - would be stored as: <c>{"date","Wed, 15 Oct 1997 14:35:17 GMT"}. + would be stored as: <c>{"date","Wed, 15 Oct 1997 14:35:17 GMT"} </c>. RFC 2616 defines that HTTP is a case insensitive protocol and the header fields may be in lower case or upper case. Httpd will - ensure that all header field names are in lower case. </c>. + ensure that all header field names are in lower case. </item> <tag><c>entity_body</c></tag> <item>The <c>Entity-Body</c> as defined diff --git a/lib/inets/doc/src/notes.xml b/lib/inets/doc/src/notes.xml index 7b16189860..11b0af4310 100644 --- a/lib/inets/doc/src/notes.xml +++ b/lib/inets/doc/src/notes.xml @@ -32,7 +32,50 @@ <file>notes.xml</file> </header> - <section><title>Inets 5.5</title> + <section><title>Inets 5.5.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> Fix format_man_pages so it handles all man sections + and remove warnings/errors in various man pages. </p> + <p> + Own Id: OTP-8600</p> + </item> + <item> + <p> + [httpc] Pipelined and queued requests not processed when + connection closed remotelly.</p> + <p> + Own Id: OTP-8906</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Miscellaneous inet6 related problems.</p> + <p> + Own Id: OTP-8927</p> + </item> + <item> + <p> + Updated http-server to make sure URLs in error-messages + are URL-encoded. Added support in http-client to use + URL-encoding. Also added the missing include directory + for the inets application.</p> + <p> + Own Id: OTP-8940 Aux Id: seq11735 </p> + </item> + </list> + </section> + +</section> + +<section><title>Inets 5.5</title> <section><title>Fixed Bugs and Malfunctions</title> <list> diff --git a/lib/inets/include/httpd.hrl b/lib/inets/include/httpd.hrl new file mode 100644 index 0000000000..a7e63ca670 --- /dev/null +++ b/lib/inets/include/httpd.hrl @@ -0,0 +1,41 @@ +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% +%% + +-ifndef(httpd_hrl). +-define(httpd_hrl, true). + +-include_lib("kernel/include/file.hrl"). + +-record(init_data,{peername,resolve}). + +-record(mod,{init_data, + data=[], + socket_type=ip_comm, + socket, + config_db, + method, + absolute_uri=[], + request_uri, + http_version, + request_line, + parsed_header=[], + entity_body, + connection}). +-endif. % -ifdef(httpd_hrl). diff --git a/lib/inets/src/http_client/Makefile b/lib/inets/src/http_client/Makefile index 575c6efaec..0397b48ab2 100644 --- a/lib/inets/src/http_client/Makefile +++ b/lib/inets/src/http_client/Makefile @@ -51,7 +51,6 @@ MODULES = \ httpc_profile_sup \ httpc_response \ httpc_request \ - http_uri \ HRL_FILES = httpc_internal.hrl diff --git a/lib/inets/src/http_client/httpc.erl b/lib/inets/src/http_client/httpc.erl index 851364001c..ae754fab94 100644 --- a/lib/inets/src/http_client/httpc.erl +++ b/lib/inets/src/http_client/httpc.erl @@ -442,18 +442,23 @@ handle_request(Method, Url, HeadersRecord = header_record(NewHeaders, Host2, HTTPOptions), Receiver = proplists:get_value(receiver, Options), SocketOpts = proplists:get_value(socket_opts, Options), + UrlEncodeBool = HTTPOptions#http_options.url_encode, + MaybeEscPath = url_encode(Path, UrlEncodeBool), + MaybeEscQuery = url_encode(Query, UrlEncodeBool), + AbsUri = url_encode(Url, UrlEncodeBool), + Request = #request{from = Receiver, scheme = Scheme, address = {Host, Port}, - path = Path, - pquery = Query, + path = MaybeEscPath, + pquery = MaybeEscQuery, method = Method, headers = HeadersRecord, content = {ContentType, Body}, settings = HTTPOptions, - abs_uri = Url, + abs_uri = AbsUri, userinfo = UserInfo, - stream = Stream, + stream = Stream, headers_as_is = headers_as_is(Headers, Options), socket_opts = SocketOpts, started = Started}, @@ -471,6 +476,10 @@ handle_request(Method, Url, Error end. +url_encode(URI, true) -> + http_uri:encode(URI); +url_encode(URI, false) -> + URI. handle_answer(RequestId, false, _) -> {ok, RequestId}; @@ -578,12 +587,8 @@ http_options_default() -> (_) -> error end, - AutoRedirectPost = fun(Value) when (Value =:= true) orelse - (Value =:= false) -> - {ok, Value}; - (_) -> - error - end, + AutoRedirectPost = boolfun(), + SslPost = fun(Value) when is_list(Value) -> {ok, {?HTTP_DEFAULT_SSL_KIND, Value}}; ({ssl, SslOptions}) when is_list(SslOptions) -> @@ -601,12 +606,8 @@ http_options_default() -> (_) -> error end, - RelaxedPost = fun(Value) when (Value =:= true) orelse - (Value =:= false) -> - {ok, Value}; - (_) -> - error - end, + RelaxedPost = boolfun(), + ConnTimeoutPost = fun(Value) when is_integer(Value) andalso (Value >= 0) -> {ok, Value}; @@ -615,6 +616,8 @@ http_options_default() -> (_) -> error end, + + UrlDecodePost = boolfun(), [ {version, {value, "HTTP/1.1"}, #http_options.version, VersionPost}, {timeout, {value, ?HTTP_REQUEST_TIMEOUT}, #http_options.timeout, TimeoutPost}, @@ -622,18 +625,21 @@ http_options_default() -> {ssl, {value, {?HTTP_DEFAULT_SSL_KIND, []}}, #http_options.ssl, SslPost}, {proxy_auth, {value, undefined}, #http_options.proxy_auth, ProxyAuthPost}, {relaxed, {value, false}, #http_options.relaxed, RelaxedPost}, + {url_encode, {value, false}, #http_options.url_encode, UrlDecodePost}, %% this field has to be *after* the timeout option (as that field is used for the default value) {connect_timeout, {field, #http_options.timeout}, #http_options.connect_timeout, ConnTimeoutPost} ]. +boolfun() -> + fun(Value) when (Value =:= true) orelse + (Value =:= false) -> + {ok, Value}; + (_) -> + error + end. request_options_defaults() -> - VerifyBoolean = - fun(Value) when ((Value =:= true) orelse (Value =:= false)) -> - ok; - (_) -> - error - end, + VerifyBoolean = boolfun(), VerifySync = VerifyBoolean, diff --git a/lib/inets/src/http_client/httpc_handler.erl b/lib/inets/src/http_client/httpc_handler.erl index 8af6613fa2..cb6f3e2841 100644 --- a/lib/inets/src/http_client/httpc_handler.erl +++ b/lib/inets/src/http_client/httpc_handler.erl @@ -713,33 +713,38 @@ terminate(normal, profile_name = ProfileName, request = Request, timers = Timers, - pipeline = Pipeline}) -> + pipeline = Pipeline, + keep_alive = KeepAlive} = State) -> ?hcrt("terminate(normal) - remote close", [{id, Id}, {profile, ProfileName}]), %% Clobber session (catch httpc_manager:delete_session(Id, ProfileName)), + maybe_retry_queue(Pipeline, State), + maybe_retry_queue(KeepAlive, State), + %% Cancel timers - #timers{request_timers = ReqTmrs, queue_timer = QTmr} = Timers, - cancel_timer(QTmr, timeout_queue), - lists:foreach(fun({_, Timer}) -> cancel_timer(Timer, timeout) end, - ReqTmrs), + cancel_timers(Timers), %% Maybe deliver answers to requests - deliver_answers([Request | queue:to_list(Pipeline)]), + deliver_answer(Request), %% And, just in case, close our side (**really** overkill) http_transport:close(SocketType, Socket); -terminate(_, #state{session = #session{id = Id, - socket = Socket, - socket_type = SocketType}, +terminate(Reason, #state{session = #session{id = Id, + socket = Socket, + socket_type = SocketType}, request = undefined, profile_name = ProfileName, timers = Timers, pipeline = Pipeline, keep_alive = KeepAlive} = State) -> + ?hcrt("terminate", + [{id, Id}, {profile, ProfileName}, {reason, Reason}]), + + %% Clobber session (catch httpc_manager:delete_session(Id, ProfileName)), maybe_retry_queue(Pipeline, State), @@ -772,59 +777,55 @@ maybe_send_answer(#request{from = answer_sent}, _Reason, State) -> maybe_send_answer(Request, Answer, State) -> answer_request(Request, Answer, State). -deliver_answers([]) -> - ?hcrd("deliver answer done", []), - ok; -deliver_answers([#request{id = Id, from = From} = Request | Requests]) +deliver_answer(#request{id = Id, from = From} = Request) when is_pid(From) -> Response = httpc_response:error(Request, socket_closed_remotely), ?hcrd("deliver answer", [{id, Id}, {from, From}, {response, Response}]), - httpc_response:send(From, Response), - deliver_answers(Requests); -deliver_answers([Request|Requests]) -> + httpc_response:send(From, Response); +deliver_answer(Request) -> ?hcrd("skip deliver answer", [{request, Request}]), - deliver_answers(Requests). + ok. %%-------------------------------------------------------------------- %% Func: code_change(_OldVsn, State, Extra) -> {ok, NewState} %% Purpose: Convert process state when code is changed %%-------------------------------------------------------------------- -code_change(_, #state{request = Request, pipeline = Queue} = State, - [{from, '5.0.1'}, {to, '5.0.2'}]) -> - Settings = new_http_options(Request#request.settings), - NewRequest = Request#request{settings = Settings}, - NewQueue = new_queue(Queue, fun new_http_options/1), - {ok, State#state{request = NewRequest, pipeline = NewQueue}}; - -code_change(_, #state{request = Request, pipeline = Queue} = State, - [{from, '5.0.2'}, {to, '5.0.1'}]) -> - Settings = old_http_options(Request#request.settings), - NewRequest = Request#request{settings = Settings}, - NewQueue = new_queue(Queue, fun old_http_options/1), - {ok, State#state{request = NewRequest, pipeline = NewQueue}}; +%% code_change(_, #state{request = Request, pipeline = Queue} = State, +%% [{from, '5.0.1'}, {to, '5.0.2'}]) -> +%% Settings = new_http_options(Request#request.settings), +%% NewRequest = Request#request{settings = Settings}, +%% NewQueue = new_queue(Queue, fun new_http_options/1), +%% {ok, State#state{request = NewRequest, pipeline = NewQueue}}; + +%% code_change(_, #state{request = Request, pipeline = Queue} = State, +%% [{from, '5.0.2'}, {to, '5.0.1'}]) -> +%% Settings = old_http_options(Request#request.settings), +%% NewRequest = Request#request{settings = Settings}, +%% NewQueue = new_queue(Queue, fun old_http_options/1), +%% {ok, State#state{request = NewRequest, pipeline = NewQueue}}; code_change(_, State, _) -> {ok, State}. -new_http_options({http_options, TimeOut, AutoRedirect, SslOpts, - Auth, Relaxed}) -> - {http_options, "HTTP/1.1", TimeOut, AutoRedirect, SslOpts, - Auth, Relaxed}. - -old_http_options({http_options, _, TimeOut, AutoRedirect, - SslOpts, Auth, Relaxed}) -> - {http_options, TimeOut, AutoRedirect, SslOpts, Auth, Relaxed}. - -new_queue(Queue, Fun) -> - List = queue:to_list(Queue), - NewList = - lists:map(fun(Request) -> - Settings = - Fun(Request#request.settings), - Request#request{settings = Settings} - end, List), - queue:from_list(NewList). +%% new_http_options({http_options, TimeOut, AutoRedirect, SslOpts, +%% Auth, Relaxed}) -> +%% {http_options, "HTTP/1.1", TimeOut, AutoRedirect, SslOpts, +%% Auth, Relaxed}. + +%% old_http_options({http_options, _, TimeOut, AutoRedirect, +%% SslOpts, Auth, Relaxed}) -> +%% {http_options, TimeOut, AutoRedirect, SslOpts, Auth, Relaxed}. + +%% new_queue(Queue, Fun) -> +%% List = queue:to_list(Queue), +%% NewList = +%% lists:map(fun(Request) -> +%% Settings = +%% Fun(Request#request.settings), +%% Request#request{settings = Settings} +%% end, List), +%% queue:from_list(NewList). %%%-------------------------------------------------------------------- @@ -854,12 +855,18 @@ connect(SocketType, ToAddress, inet6fb4 -> Opts3 = [inet6 | Opts2], case http_transport:connect(SocketType, ToAddress, Opts3, Timeout) of - {error, Reason} when ((Reason =:= nxdomain) orelse - (Reason =:= eafnosupport)) -> + {error, _Reason} = Error -> Opts4 = [inet | Opts2], - http_transport:connect(SocketType, ToAddress, Opts4, Timeout); - Other -> - Other + case http_transport:connect(SocketType, + ToAddress, Opts4, Timeout) of + {error, _} -> + %% Reply with the "original" error + Error; + OK -> + OK + end; + OK -> + OK end; _ -> Opts3 = [IpFamily | Opts2], @@ -1440,6 +1447,12 @@ answer_request(#request{id = RequestId, from = From} = Request, Msg, timers = Timers#timers{request_timers = lists:delete(Timer, RequestTimers)}}. + +cancel_timers(#timers{request_timers = ReqTmrs, queue_timer = QTmr}) -> + cancel_timer(QTmr, timeout_queue), + CancelTimer = fun({_, Timer}) -> cancel_timer(Timer, timeout) end, + lists:foreach(CancelTimer, ReqTmrs). + cancel_timer(undefined, _) -> ok; cancel_timer(Timer, TimeoutMsg) -> diff --git a/lib/inets/src/http_client/httpc_internal.hrl b/lib/inets/src/http_client/httpc_internal.hrl index 3cdd95a02b..1d8a5b6a92 100644 --- a/lib/inets/src/http_client/httpc_internal.hrl +++ b/lib/inets/src/http_client/httpc_internal.hrl @@ -60,7 +60,11 @@ relaxed = false, %% integer() - ms before a connect times out - connect_timeout = ?HTTP_REQUEST_CTIMEOUT + connect_timeout = ?HTTP_REQUEST_CTIMEOUT, + + %% bool() - Use %-encoding rfc 2396 + url_encode + } ). diff --git a/lib/inets/src/http_client/httpc_manager.erl b/lib/inets/src/http_client/httpc_manager.erl index 1e1bde220b..591cb78c29 100644 --- a/lib/inets/src/http_client/httpc_manager.erl +++ b/lib/inets/src/http_client/httpc_manager.erl @@ -734,10 +734,11 @@ handle_connect_and_send(_StarterPid, ReqId, HandlerPid, Result, ok; [] -> - error_report(Profile, - "handler (~p) successfully started " - "for unknown request ~p => canceling", - [HandlerPid, ReqId]), + ?hcri("handler successfully started " + "for unknown request => canceling", + [{profile, Profile}, + {handler, HandlerPid}, + {request, ReqId}]), httpc_handler:cancel(ReqId, HandlerPid) end. diff --git a/lib/inets/src/http_lib/Makefile b/lib/inets/src/http_lib/Makefile index 5dac3b0c00..aaf3cfb995 100644 --- a/lib/inets/src/http_lib/Makefile +++ b/lib/inets/src/http_lib/Makefile @@ -45,7 +45,8 @@ MODULES = \ http_transport\ http_util \ http_request \ - http_response + http_response \ + http_uri HRL_FILES = http_internal.hrl diff --git a/lib/inets/src/http_lib/http_transport.erl b/lib/inets/src/http_lib/http_transport.erl index b8121852b8..0024d19fc1 100644 --- a/lib/inets/src/http_lib/http_transport.erl +++ b/lib/inets/src/http_lib/http_transport.erl @@ -192,24 +192,31 @@ listen_ip_comm(Addr, Port) -> case IpFamily of inet6fb4 -> Opts2 = [inet6 | Opts], + ?hlrt("try ipv6 listen", [{port, NewPort}, {opts, Opts2}]), case (catch gen_tcp:listen(NewPort, Opts2)) of {error, Reason} when ((Reason =:= nxdomain) orelse (Reason =:= eafnosupport)) -> Opts3 = [inet | Opts], + ?hlrt("ipv6 listen failed - try ipv4 instead", + [{reason, Reason}, {port, NewPort}, {opts, Opts3}]), gen_tcp:listen(NewPort, Opts3); %% This is when a given hostname has resolved to a %% IPv4-address. The inet6-option together with a %% {ip, IPv4} option results in badarg - {'EXIT', _} -> + {'EXIT', Reason} -> Opts3 = [inet | Opts], + ?hlrt("ipv6 listen exit - try ipv4 instead", + [{reason, Reason}, {port, NewPort}, {opts, Opts3}]), gen_tcp:listen(NewPort, Opts3); Other -> + ?hlrt("ipv6 listen done", [{other, Other}]), Other end; _ -> Opts2 = [IpFamily | Opts], + ?hlrt("listen", [{port, NewPort}, {opts, Opts2}]), gen_tcp:listen(NewPort, Opts2) end. diff --git a/lib/inets/src/http_client/http_uri.erl b/lib/inets/src/http_lib/http_uri.erl index 615a0d8ec4..44b9face0b 100644 --- a/lib/inets/src/http_client/http_uri.erl +++ b/lib/inets/src/http_lib/http_uri.erl @@ -1,26 +1,26 @@ %% %% %CopyrightBegin% -%% -%% Copyright Ericsson AB 2006-2009. All Rights Reserved. -%% +%% +%% Copyright Ericsson AB 2006-2010. All Rights Reserved. +%% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in %% compliance with the License. You should have received a copy of the %% Erlang Public License along with this software. If not, it can be %% retrieved online at http://www.erlang.org/. -%% +%% %% Software distributed under the License is distributed on an "AS IS" %% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See %% the License for the specific language governing rights and limitations %% under the License. -%% +%% %% %CopyrightEnd% %% %% -module(http_uri). --export([parse/1]). +-export([parse/1, encode/1, decode/1]). %%%========================================================================= %%% API @@ -34,10 +34,25 @@ parse(AbsURI) -> {UserInfo, Host, Port, Path, Query} -> {Scheme, UserInfo, Host, Port, Path, Query}; _ -> - {error, {malformed_url, AbsURI}} + {error, {malformed_url, AbsURI}} end end. +encode(URI) -> + Reserved = sets:from_list([$;, $:, $@, $&, $=, $+, $,, $/, $?, + $#, $[, $], $<, $>, $\", ${, $}, $|, + $\\, $', $^, $%, $ ]), + lists:append(lists:map(fun(Char) -> + uri_encode(Char, Reserved) + end, URI)). + +decode([$%,Hex1,Hex2|Rest]) -> + [hex2dec(Hex1)*16+hex2dec(Hex2)|decode(Rest)]; +decode([First|Rest]) -> + [First|decode(Rest)]; +decode([]) -> + []. + %%%======================================================================== %%% Internal functions %%%======================================================================== @@ -56,7 +71,7 @@ parse_scheme(AbsURI) -> parse_uri_rest(Scheme, "//" ++ URIPart) -> - {Authority, PathQuery} = + {Authority, PathQuery} = case split_uri(URIPart, "/", URIPart, 1, 0) of Split = {_, _} -> Split; @@ -68,7 +83,7 @@ parse_uri_rest(Scheme, "//" ++ URIPart) -> {URIPart,""} end end, - + {UserInfo, HostPort} = split_uri(Authority, "@", {"", Authority}, 1, 1), {Host, Port} = parse_host_port(Scheme, HostPort), {Path, Query} = parse_path_query(PathQuery), @@ -78,7 +93,6 @@ parse_uri_rest(Scheme, "//" ++ URIPart) -> parse_path_query(PathQuery) -> {Path, Query} = split_uri(PathQuery, "\\?", {PathQuery, ""}, 1, 0), {path(Path), Query}. - parse_host_port(Scheme,"[" ++ HostPort) -> %ipv6 DefaultPort = default_port(Scheme), @@ -90,12 +104,12 @@ parse_host_port(Scheme, HostPort) -> DefaultPort = default_port(Scheme), {Host, Port} = split_uri(HostPort, ":", {HostPort, DefaultPort}, 1, 1), {Host, int_port(Port)}. - + split_uri(UriPart, SplitChar, NoMatchResult, SkipLeft, SkipRight) -> case inets_regexp:first_match(UriPart, SplitChar) of {match, Match, _} -> {string:substr(UriPart, 1, Match - SkipLeft), - string:substr(UriPart, Match + SkipRight, length(UriPart))}; + string:substr(UriPart, Match + SkipRight, length(UriPart))}; nomatch -> NoMatchResult end. @@ -114,3 +128,15 @@ path("") -> "/"; path(Path) -> Path. + +uri_encode(Char, Reserved) -> + case sets:is_element(Char, Reserved) of + true -> + [ $% | http_util:integer_to_hexlist(Char)]; + false -> + [Char] + end. + +hex2dec(X) when (X>=$0) andalso (X=<$9) -> X-$0; +hex2dec(X) when (X>=$A) andalso (X=<$F) -> X-$A+10; +hex2dec(X) when (X>=$a) andalso (X=<$f) -> X-$a+10. diff --git a/lib/inets/src/http_server/Makefile b/lib/inets/src/http_server/Makefile index 879e605217..bdd8c5ee3c 100644 --- a/lib/inets/src/http_server/Makefile +++ b/lib/inets/src/http_server/Makefile @@ -82,7 +82,9 @@ MODULES = \ mod_security \ mod_security_server -HRL_FILES = httpd.hrl httpd_internal.hrl mod_auth.hrl +INCLUDE = ../../include + +HRL_FILES = $(INCLUDE)/httpd.hrl httpd_internal.hrl mod_auth.hrl ERL_FILES = $(MODULES:%=%.erl) @@ -98,9 +100,9 @@ include ../inets_app/inets.mk ERL_COMPILE_FLAGS += \ $(INETS_FLAGS) \ $(INETS_ERL_COMPILE_FLAGS) \ - -I../../include \ + -I$(INCLUDE) \ -I../inets_app \ - -I../http_lib + -I../http_lib \ # ---------------------------------------------------- diff --git a/lib/inets/src/http_server/httpd.erl b/lib/inets/src/http_server/httpd.erl index fb5fa1c758..93608dbf96 100644 --- a/lib/inets/src/http_server/httpd.erl +++ b/lib/inets/src/http_server/httpd.erl @@ -24,7 +24,6 @@ -include("httpd.hrl"). - %% Behavior callbacks -export([ start_standalone/1, @@ -271,8 +270,8 @@ foreach([KeyValue|Rest]) -> {ok, Plus2Space, _} = inets_regexp:gsub(KeyValue,"[\+]"," "), case inets_regexp:split(Plus2Space,"=") of {ok,[Key|Value]} -> - [{httpd_util:decode_hex(Key), - httpd_util:decode_hex(lists:flatten(Value))}|foreach(Rest)]; + [{http_uri:decode(Key), + http_uri:decode(lists:flatten(Value))}|foreach(Rest)]; {ok,_} -> foreach(Rest) end. diff --git a/lib/inets/src/http_server/httpd.hrl b/lib/inets/src/http_server/httpd.hrl deleted file mode 100644 index 0db8a029bb..0000000000 --- a/lib/inets/src/http_server/httpd.hrl +++ /dev/null @@ -1,82 +0,0 @@ -%% -%% %CopyrightBegin% -%% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. -%% -%% The contents of this file are subject to the Erlang Public License, -%% Version 1.1, (the "License"); you may not use this file except in -%% compliance with the License. You should have received a copy of the -%% Erlang Public License along with this software. If not, it can be -%% retrieved online at http://www.erlang.org/. -%% -%% Software distributed under the License is distributed on an "AS IS" -%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See -%% the License for the specific language governing rights and limitations -%% under the License. -%% -%% %CopyrightEnd% -%% -%% - --include_lib("kernel/include/file.hrl"). - --ifndef(SERVER_SOFTWARE). --define(SERVER_SOFTWARE,"inets/develop"). % Define in Makefile! --endif. --define(SERVER_PROTOCOL,"HTTP/1.1"). --define(DEFAULT_MODS, [mod_alias, mod_auth, mod_esi, mod_actions, mod_cgi, - mod_dir, mod_get, mod_head, mod_log, mod_disk_log]). --define(SOCKET_CHUNK_SIZE,8192). --define(SOCKET_MAX_POLL,25). --define(FILE_CHUNK_SIZE,64*1024). --define(GATEWAY_INTERFACE,"CGI/1.1"). --define(NICE(Reason),lists:flatten(atom_to_list(?MODULE)++": "++Reason)). --define(DEFAULT_CONTEXT, - [{errmsg,"[an error occurred while processing this directive]"}, - {timefmt,"%A, %d-%b-%y %T %Z"}, - {sizefmt,"abbrev"}]). - - --ifdef(inets_error). --define(ERROR(Format, Args), io:format("E(~p:~p:~p) : "++Format++"~n", - [self(),?MODULE,?LINE]++Args)). --else. --define(ERROR(F,A),[]). --endif. - --ifdef(inets_log). --define(LOG(Format, Args), io:format("L(~p:~p:~p) : "++Format++"~n", - [self(),?MODULE,?LINE]++Args)). --else. --define(LOG(F,A),[]). --endif. - --ifdef(inets_debug). --define(DEBUG(Format, Args), io:format("D(~p:~p:~p) : "++Format++"~n", - [self(),?MODULE,?LINE]++Args)). --else. --define(DEBUG(F,A),[]). --endif. - --ifdef(inets_cdebug). --define(CDEBUG(Format, Args), io:format("C(~p:~p:~p) : "++Format++"~n", - [self(),?MODULE,?LINE]++Args)). --else. --define(CDEBUG(F,A),[]). --endif. - - --record(init_data,{peername,resolve}). --record(mod,{init_data, - data=[], - socket_type=ip_comm, - socket, - config_db, - method, - absolute_uri=[], - request_uri, - http_version, - request_line, - parsed_header=[], - entity_body, - connection}). diff --git a/lib/inets/src/http_server/httpd_acceptor.erl b/lib/inets/src/http_server/httpd_acceptor.erl index c261eff6b2..bcebb6a9e3 100644 --- a/lib/inets/src/http_server/httpd_acceptor.erl +++ b/lib/inets/src/http_server/httpd_acceptor.erl @@ -21,6 +21,7 @@ -include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). %% Internal application API -export([start_link/5, start_link/6]). diff --git a/lib/inets/src/http_server/httpd_conf.erl b/lib/inets/src/http_server/httpd_conf.erl index 8438c4037e..f4d8a6c09f 100644 --- a/lib/inets/src/http_server/httpd_conf.erl +++ b/lib/inets/src/http_server/httpd_conf.erl @@ -31,8 +31,8 @@ validate_properties/1]). -define(VMODULE,"CONF"). --include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("httpd.hrl"). -include_lib("inets/src/http_lib/http_internal.hrl"). diff --git a/lib/inets/src/http_server/httpd_file.erl b/lib/inets/src/http_server/httpd_file.erl index 5fd529100e..7e21d9e158 100644 --- a/lib/inets/src/http_server/httpd_file.erl +++ b/lib/inets/src/http_server/httpd_file.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2006-2009. All Rights Reserved. +%% Copyright Ericsson AB 2006-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,11 +22,13 @@ -export([handle_error/4]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). + handle_error(eacces, Op, ModData, Path) -> - handle_error(403, Op, ModData, Path,""); + handle_error(403, Op, ModData, Path,"Forbidden"); handle_error(enoent, Op, ModData, Path) -> - handle_error(404, Op, ModData, Path,""); + handle_error(404, Op, ModData, Path,"File not found"); handle_error(enotdir, Op, ModData, Path) -> handle_error(404, Op, ModData, Path, ": A component of the file name is not a directory"); @@ -34,8 +36,8 @@ handle_error(emfile, Op, _ModData, Path) -> handle_error(500, Op, none, Path, ": To many open files"); handle_error({enfile,_}, Op, _ModData, Path) -> handle_error(500, Op, none, Path, ": File table overflow"); -handle_error(_Reason, Op, _ModData, Path) -> - handle_error(500, Op, none, Path, ""). +handle_error(_Reason, Op, ModData, Path) -> + handle_error(404, Op, ModData, Path, "File not found"). handle_error(StatusCode, Op, none, Path, Reason) -> {StatusCode, none, ?NICE("Can't " ++ Op ++ Path ++ Reason)}; diff --git a/lib/inets/src/http_server/httpd_internal.hrl b/lib/inets/src/http_server/httpd_internal.hrl index 38b0ddefd3..108469ea0a 100644 --- a/lib/inets/src/http_server/httpd_internal.hrl +++ b/lib/inets/src/http_server/httpd_internal.hrl @@ -21,7 +21,50 @@ -ifndef(httpd_internal_hrl). -define(httpd_internal_hrl, true). --include_lib("inets/src/inets_app/inets_internal.hrl"). +-ifndef(SERVER_SOFTWARE). +-define(SERVER_SOFTWARE,"inets/develop"). % Define in Makefile! +-endif. +-define(SERVER_PROTOCOL,"HTTP/1.1"). +-define(DEFAULT_MODS, [mod_alias, mod_auth, mod_esi, mod_actions, mod_cgi, + mod_dir, mod_get, mod_head, mod_log, mod_disk_log]). +-define(SOCKET_CHUNK_SIZE,8192). +-define(SOCKET_MAX_POLL,25). +-define(FILE_CHUNK_SIZE,64*1024). +-define(GATEWAY_INTERFACE,"CGI/1.1"). +-define(NICE(Reason),lists:flatten(atom_to_list(?MODULE)++": "++Reason)). +-define(DEFAULT_CONTEXT, + [{errmsg,"[an error occurred while processing this directive]"}, + {timefmt,"%A, %d-%b-%y %T %Z"}, + {sizefmt,"abbrev"}]). + + +-ifdef(inets_error). +-define(ERROR(Format, Args), io:format("E(~p:~p:~p) : "++Format++"~n", + [self(),?MODULE,?LINE]++Args)). +-else. +-define(ERROR(F,A),[]). +-endif. + +-ifdef(inets_log). +-define(LOG(Format, Args), io:format("L(~p:~p:~p) : "++Format++"~n", + [self(),?MODULE,?LINE]++Args)). +-else. +-define(LOG(F,A),[]). +-endif. + +-ifdef(inets_debug). +-define(DEBUG(Format, Args), io:format("D(~p:~p:~p) : "++Format++"~n", + [self(),?MODULE,?LINE]++Args)). +-else. +-define(DEBUG(F,A),[]). +-endif. + +-ifdef(inets_cdebug). +-define(CDEBUG(Format, Args), io:format("C(~p:~p:~p) : "++Format++"~n", + [self(),?MODULE,?LINE]++Args)). +-else. +-define(CDEBUG(F,A),[]). +-endif. -define(SERVICE, httpd). -define(hdri(Label, Content), ?report_important(Label, ?SERVICE, Content)). diff --git a/lib/inets/src/http_server/httpd_request.erl b/lib/inets/src/http_server/httpd_request.erl index 883acbf585..7084d9824a 100644 --- a/lib/inets/src/http_server/httpd_request.erl +++ b/lib/inets/src/http_server/httpd_request.erl @@ -304,9 +304,9 @@ validate_uri(RequestURI) -> UriNoQueryNoHex = case string:str(RequestURI, "?") of 0 -> - (catch httpd_util:decode_hex(RequestURI)); + (catch http_uri:decode(RequestURI)); Ndx -> - (catch httpd_util:decode_hex(string:left(RequestURI, Ndx))) + (catch http_uri:decode(string:left(RequestURI, Ndx))) end, case UriNoQueryNoHex of {'EXIT',_Reason} -> diff --git a/lib/inets/src/http_server/httpd_script_env.erl b/lib/inets/src/http_server/httpd_script_env.erl index a742cbef76..d3115150b0 100644 --- a/lib/inets/src/http_server/httpd_script_env.erl +++ b/lib/inets/src/http_server/httpd_script_env.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2005-2009. All Rights Reserved. +%% Copyright Ericsson AB 2005-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -23,6 +23,7 @@ -export([create_env/3]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). %%%========================================================================= %%% Internal application API diff --git a/lib/inets/src/http_server/httpd_sup.erl b/lib/inets/src/http_server/httpd_sup.erl index 1507c6852a..f94e5459c1 100644 --- a/lib/inets/src/http_server/httpd_sup.erl +++ b/lib/inets/src/http_server/httpd_sup.erl @@ -37,7 +37,7 @@ -define(TIMEOUT, 15000). -include("httpd_internal.hrl"). - +-include("inets_internal.hrl"). %%%========================================================================= %%% API diff --git a/lib/inets/src/http_server/httpd_util.erl b/lib/inets/src/http_server/httpd_util.erl index cfad79638f..789f12652b 100644 --- a/lib/inets/src/http_server/httpd_util.erl +++ b/lib/inets/src/http_server/httpd_util.erl @@ -21,7 +21,7 @@ -export([ip_address/2, lookup/2, lookup/3, multi_lookup/2, lookup_mime/2, lookup_mime/3, lookup_mime_default/2, lookup_mime_default/3, reason_phrase/1, message/3, rfc1123_date/0, - rfc1123_date/1, day/1, month/1, decode_hex/1, + rfc1123_date/1, day/1, month/1, flatlength/1, split_path/1, split_script_path/1, suffix/1, split/3, uniq/1, make_name/2,make_name/3,make_name/4,strip/1, @@ -32,7 +32,7 @@ dir_validate/2, file_validate/2, mime_type_validate/1, mime_types_validate/1, custom_date/0]). --export([encode_hex/1]). +-export([encode_hex/1, decode_hex/1]). -include_lib("kernel/include/file.hrl"). ip_address({_,_,_,_} = Address, _IpFamily) -> @@ -175,13 +175,13 @@ reason_phrase(_) -> "Internal Server Error". %% message message(301,URL,_) -> - "The document has moved <A HREF=\""++URL++"\">here</A>."; + "The document has moved <A HREF=\""++ maybe_encode(URL) ++"\">here</A>."; message(304, _URL,_) -> "The document has not been changed."; message(400,none,_) -> "Your browser sent a query that this server could not understand."; message(400,Msg,_) -> - "Your browser sent a query that this server could not understand. "++Msg; + "Your browser sent a query that this server could not understand. "++ maybe_encode(Msg); message(401,none,_) -> "This server could not verify that you are authorized to access the document you @@ -190,9 +190,9 @@ credentials (e.g., bad password), or your browser doesn't understand how to supply the credentials required."; message(403,RequestURI,_) -> - "You don't have permission to access "++RequestURI++" on this server."; + "You don't have permission to access "++ maybe_encode(RequestURI) ++" on this server."; message(404,RequestURI,_) -> - "The requested URL "++RequestURI++" was not found on this server."; + "The requested URL " ++ maybe_encode(RequestURI) ++ " was not found on this server."; message(408, Timeout, _) -> Timeout; message(412,none,_) -> @@ -200,7 +200,7 @@ message(412,none,_) -> message(413, Reason,_) -> "Entity: " ++ Reason; message(414,ReasonPhrase,_) -> - "Message "++ReasonPhrase++"."; + "Message "++ ReasonPhrase ++"."; message(416,ReasonPhrase,_) -> ReasonPhrase; @@ -216,15 +216,23 @@ message(501,{Method, RequestURI, HTTPVersion}, _ConfigDB) -> if is_atom(Method) -> atom_to_list(Method)++ - " to "++RequestURI++" ("++HTTPVersion++") not supported."; + " to "++ maybe_encode(RequestURI)++" ("++HTTPVersion++") not supported."; is_list(Method) -> Method++ - " to "++RequestURI++" ("++HTTPVersion++") not supported." + " to "++ maybe_encode(RequestURI)++" ("++HTTPVersion++") not supported." end; message(503, String, _ConfigDB) -> "This service in unavailable due to: "++String. +maybe_encode(URI) -> + case lists:member($%, URI) of + true -> + URI; + false -> + http_uri:encode(URI) + end. + %%convert_rfc_date(Date)->{{YYYY,MM,DD},{HH,MIN,SEC}} convert_request_date([D,A,Y,DateType| Rest])-> @@ -381,16 +389,11 @@ month(12) -> "Dec". %% decode_hex -decode_hex([$%,Hex1,Hex2|Rest]) -> - [hex2dec(Hex1)*16+hex2dec(Hex2)|decode_hex(Rest)]; -decode_hex([First|Rest]) -> - [First|decode_hex(Rest)]; -decode_hex([]) -> - []. +decode_hex(URI) -> + http_uri:decode(URI). -hex2dec(X) when (X>=$0) andalso (X=<$9) -> X-$0; -hex2dec(X) when (X>=$A) andalso (X=<$F) -> X-$A+10; -hex2dec(X) when (X>=$a) andalso (X=<$f) -> X-$a+10. +encode_hex(URI) -> + http_uri:encode(URI). %% flatlength flatlength(List) -> @@ -411,7 +414,7 @@ split_path(Path) -> case inets_regexp:match(Path,"[\?].*\$") of %% A QUERY_STRING exists! {match,Start,Length} -> - {httpd_util:decode_hex(string:substr(Path,1,Start-1)), + {http_uri:decode(string:substr(Path,1,Start-1)), string:substr(Path,Start,Length)}; %% A possible PATH_INFO exists! nomatch -> @@ -419,9 +422,9 @@ split_path(Path) -> end. split_path([],SoFar) -> - {httpd_util:decode_hex(lists:reverse(SoFar)),[]}; + {http_uri:decode(lists:reverse(SoFar)),[]}; split_path([$/|Rest],SoFar) -> - Path=httpd_util:decode_hex(lists:reverse(SoFar)), + Path=http_uri:decode(lists:reverse(SoFar)), case file:read_file_info(Path) of {ok,FileInfo} when FileInfo#file_info.type =:= regular -> {Path,[$/|Rest]}; @@ -454,7 +457,7 @@ pathinfo_querystring([C|Rest], SoFar) -> pathinfo_querystring(Rest, [C|SoFar]). split_script_path([$?|QueryString], SoFar) -> - Path = httpd_util:decode_hex(lists:reverse(SoFar)), + Path = http_uri:decode(lists:reverse(SoFar)), case file:read_file_info(Path) of {ok,FileInfo} when FileInfo#file_info.type =:= regular -> {Path, [$?|QueryString]}; @@ -464,7 +467,7 @@ split_script_path([$?|QueryString], SoFar) -> not_a_script end; split_script_path([], SoFar) -> - Path = httpd_util:decode_hex(lists:reverse(SoFar)), + Path = http_uri:decode(lists:reverse(SoFar)), case file:read_file_info(Path) of {ok,FileInfo} when FileInfo#file_info.type =:= regular -> {Path, []}; @@ -474,7 +477,7 @@ split_script_path([], SoFar) -> not_a_script end; split_script_path([$/|Rest], SoFar) -> - Path = httpd_util:decode_hex(lists:reverse(SoFar)), + Path = http_uri:decode(lists:reverse(SoFar)), case file:read_file_info(Path) of {ok, FileInfo} when FileInfo#file_info.type =:= regular -> {Path, [$/|Rest]}; @@ -608,9 +611,6 @@ hexlist_to_integer(List)-> %%---------------------------------------------------------------------- %%Converts an integer to an hexlist %%---------------------------------------------------------------------- -encode_hex(Num)-> - integer_to_hexlist(Num). - integer_to_hexlist(Num) when is_integer(Num) -> http_util:integer_to_hexlist(Num). @@ -735,7 +735,6 @@ valid_accept_timeout(A) -> valid_config(_) -> ok. - %%---------------------------------------------------------------------- %% Enable debugging, %%---------------------------------------------------------------------- diff --git a/lib/inets/src/http_server/mod_actions.erl b/lib/inets/src/http_server/mod_actions.erl index d50ed4b16c..c3946ff9b4 100644 --- a/lib/inets/src/http_server/mod_actions.erl +++ b/lib/inets/src/http_server/mod_actions.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -21,6 +21,7 @@ -export([do/1,load/2, store/2]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). %% do diff --git a/lib/inets/src/http_server/mod_alias.erl b/lib/inets/src/http_server/mod_alias.erl index 9c5a8cc1c6..0b9fe4cfe0 100644 --- a/lib/inets/src/http_server/mod_alias.erl +++ b/lib/inets/src/http_server/mod_alias.erl @@ -29,6 +29,7 @@ -include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). -define(VMODULE,"ALIAS"). diff --git a/lib/inets/src/http_server/mod_auth.erl b/lib/inets/src/http_server/mod_auth.erl index 07cafb4726..85a87ab884 100644 --- a/lib/inets/src/http_server/mod_auth.erl +++ b/lib/inets/src/http_server/mod_auth.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -38,6 +38,7 @@ -include("httpd.hrl"). -include("mod_auth.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). -define(VMODULE,"AUTH"). diff --git a/lib/inets/src/http_server/mod_auth_dets.erl b/lib/inets/src/http_server/mod_auth_dets.erl index bc6c2b70a0..a48725d5d9 100644 --- a/lib/inets/src/http_server/mod_auth_dets.erl +++ b/lib/inets/src/http_server/mod_auth_dets.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2009. All Rights Reserved. +%% Copyright Ericsson AB 1998-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -35,6 +35,7 @@ -export([store_directory_data/3]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). -include("mod_auth.hrl"). store_directory_data(_Directory, DirData, Server_root) -> diff --git a/lib/inets/src/http_server/mod_auth_plain.erl b/lib/inets/src/http_server/mod_auth_plain.erl index d88859d28a..c0a83711ba 100644 --- a/lib/inets/src/http_server/mod_auth_plain.erl +++ b/lib/inets/src/http_server/mod_auth_plain.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2009. All Rights Reserved. +%% Copyright Ericsson AB 1998-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,6 +22,8 @@ -include("httpd.hrl"). -include("mod_auth.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). + -define(VMODULE,"AUTH_PLAIN"). diff --git a/lib/inets/src/http_server/mod_auth_server.erl b/lib/inets/src/http_server/mod_auth_server.erl index 5f9e59be9d..947273bd9e 100644 --- a/lib/inets/src/http_server/mod_auth_server.erl +++ b/lib/inets/src/http_server/mod_auth_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,6 +22,7 @@ -include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). -behaviour(gen_server). diff --git a/lib/inets/src/http_server/mod_cgi.erl b/lib/inets/src/http_server/mod_cgi.erl index 33605b9698..c854166c29 100644 --- a/lib/inets/src/http_server/mod_cgi.erl +++ b/lib/inets/src/http_server/mod_cgi.erl @@ -27,6 +27,7 @@ -export([do/1, load/2, store/2]). -include("http_internal.hrl"). +-include("httpd_internal.hrl"). -include("httpd.hrl"). -define(VMODULE,"CGI"). diff --git a/lib/inets/src/http_server/mod_dir.erl b/lib/inets/src/http_server/mod_dir.erl index cdc7cc01e4..d791ee28e9 100644 --- a/lib/inets/src/http_server/mod_dir.erl +++ b/lib/inets/src/http_server/mod_dir.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -18,9 +18,11 @@ %% %% -module(mod_dir). --export([do/1]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). + +-export([do/1]). %% do @@ -57,7 +59,7 @@ do_dir(Info) -> case file:read_file_info(DefaultPath) of {ok,FileInfo} when FileInfo#file_info.type == directory -> DecodedRequestURI = - httpd_util:decode_hex(Info#mod.request_uri), + http_uri:decode(Info#mod.request_uri), ?DEBUG("do_dir -> ~n" " Path: ~p~n" " DefaultPath: ~p~n" diff --git a/lib/inets/src/http_server/mod_disk_log.erl b/lib/inets/src/http_server/mod_disk_log.erl index 95e0d00c70..5a3766de66 100644 --- a/lib/inets/src/http_server/mod_disk_log.erl +++ b/lib/inets/src/http_server/mod_disk_log.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -28,7 +28,7 @@ -define(VMODULE,"DISK_LOG"). -include("httpd.hrl"). - +-include("httpd_internal.hrl"). %%%========================================================================= %%% API diff --git a/lib/inets/src/http_server/mod_esi.erl b/lib/inets/src/http_server/mod_esi.erl index f7877aa9e2..929185a67a 100644 --- a/lib/inets/src/http_server/mod_esi.erl +++ b/lib/inets/src/http_server/mod_esi.erl @@ -30,6 +30,7 @@ -include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). -define(VMODULE,"ESI"). -define(DEFAULT_ERL_TIMEOUT,15000). diff --git a/lib/inets/src/http_server/mod_get.erl b/lib/inets/src/http_server/mod_get.erl index 9fd1fcec47..5cb30e3d97 100644 --- a/lib/inets/src/http_server/mod_get.erl +++ b/lib/inets/src/http_server/mod_get.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -20,7 +20,7 @@ -module(mod_get). -export([do/1]). -include("httpd.hrl"). - +-include("httpd_internal.hrl"). %% do do(Info) -> diff --git a/lib/inets/src/http_server/mod_head.erl b/lib/inets/src/http_server/mod_head.erl index 8b08d61651..c346fd4d23 100644 --- a/lib/inets/src/http_server/mod_head.erl +++ b/lib/inets/src/http_server/mod_head.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/inets/src/http_server/mod_htaccess.erl b/lib/inets/src/http_server/mod_htaccess.erl index d8835198f5..e1f66d01c8 100644 --- a/lib/inets/src/http_server/mod_htaccess.erl +++ b/lib/inets/src/http_server/mod_htaccess.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -23,6 +23,7 @@ -export([do/1, load/2, store/2]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %% Public methods that interface the eswapi %% diff --git a/lib/inets/src/http_server/mod_include.erl b/lib/inets/src/http_server/mod_include.erl index 534eba8a36..35f45bdd33 100644 --- a/lib/inets/src/http_server/mod_include.erl +++ b/lib/inets/src/http_server/mod_include.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -21,6 +21,7 @@ -export([do/1,parse/2,config/6,include/6,echo/6,fsize/6,flastmod/6,exec/6]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). -define(VMODULE,"INCLUDE"). @@ -186,9 +187,9 @@ document_uri(ConfigDB, RequestURI) -> FileName = string:substr(Path,Start,Length), case inets_regexp:match(VirtualPath, FileName++"\$") of {match, _, _} -> - httpd_util:decode_hex(VirtualPath)++AfterPath; + http_uri:decode(VirtualPath)++AfterPath; nomatch -> - string:strip(httpd_util:decode_hex(VirtualPath),right,$/)++ + string:strip(http_uri:decode(VirtualPath),right,$/)++ "/"++FileName++AfterPath end. diff --git a/lib/inets/src/http_server/mod_log.erl b/lib/inets/src/http_server/mod_log.erl index de24d5a569..c8a2ec0dc4 100644 --- a/lib/inets/src/http_server/mod_log.erl +++ b/lib/inets/src/http_server/mod_log.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -26,6 +26,7 @@ -export([do/1, load/2, store/2, remove/1]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). -define(VMODULE,"LOG"). %%%========================================================================= diff --git a/lib/inets/src/http_server/mod_range.erl b/lib/inets/src/http_server/mod_range.erl index 0698fb9099..a0408cba79 100644 --- a/lib/inets/src/http_server/mod_range.erl +++ b/lib/inets/src/http_server/mod_range.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -20,7 +20,7 @@ -module(mod_range). -export([do/1]). -include("httpd.hrl"). - +-include("httpd_internal.hrl"). %% do do(Info) -> diff --git a/lib/inets/src/http_server/mod_responsecontrol.erl b/lib/inets/src/http_server/mod_responsecontrol.erl index 79e2e1bdba..5d5b60cdbd 100644 --- a/lib/inets/src/http_server/mod_responsecontrol.erl +++ b/lib/inets/src/http_server/mod_responsecontrol.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,6 +22,7 @@ -export([do/1]). -include("httpd.hrl"). +-include("httpd_internal.hrl"). do(Info) -> ?DEBUG("do -> response_control",[]), diff --git a/lib/inets/src/http_server/mod_security.erl b/lib/inets/src/http_server/mod_security.erl index 95793e1cfb..41988732ad 100644 --- a/lib/inets/src/http_server/mod_security.erl +++ b/lib/inets/src/http_server/mod_security.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1998-2009. All Rights Reserved. +%% Copyright Ericsson AB 1998-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -32,6 +32,7 @@ -include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). -define(VMODULE,"SEC"). diff --git a/lib/inets/src/http_server/mod_security_server.erl b/lib/inets/src/http_server/mod_security_server.erl index 58060686b3..784b3eba70 100644 --- a/lib/inets/src/http_server/mod_security_server.erl +++ b/lib/inets/src/http_server/mod_security_server.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -45,6 +45,7 @@ -include("httpd.hrl"). -include("httpd_internal.hrl"). +-include("inets_internal.hrl"). -behaviour(gen_server). diff --git a/lib/inets/src/http_server/mod_trace.erl b/lib/inets/src/http_server/mod_trace.erl index df482228d8..7233925783 100644 --- a/lib/inets/src/http_server/mod_trace.erl +++ b/lib/inets/src/http_server/mod_trace.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2001-2009. All Rights Reserved. +%% Copyright Ericsson AB 2001-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/inets/src/inets_app/inets.appup.src b/lib/inets/src/inets_app/inets.appup.src index 84d8c9278d..0194c65db9 100644 --- a/lib/inets/src/inets_app/inets.appup.src +++ b/lib/inets/src/inets_app/inets.appup.src @@ -18,6 +18,11 @@ {"%VSN%", [ + {"5.5", + [ + {restart_application, inets} + ] + }, {"5.4", [ {restart_application, inets} @@ -29,6 +34,11 @@ [ {restart_application, inets} ] + }, + {"5.4", + [ + {restart_application, inets} + ] } ] }. diff --git a/lib/inets/test/Makefile b/lib/inets/test/Makefile index bb7f2186af..0492d33eab 100644 --- a/lib/inets/test/Makefile +++ b/lib/inets/test/Makefile @@ -288,16 +288,20 @@ release_spec: opt $(INSTALL_DATA) $(INETS_FILES) $(RELSYSDIR)/test @for d in $(DATADIRS); do \ echo "installing data dir $$d"; \ - echo $$d/TAR.exclude2 > $$d/TAR.exclude2; \ - cat $$d/TAR.exclude >> $$d/TAR.exclude2; \ - find $$d -name '*.contrib*' >> $$d/TAR.exclude2; \ - find $$d -name '*.keep*' >> $$d/TAR.exclude2; \ - find $$d -name '*.mkelem*' >> $$d/TAR.exclude2; \ - find $$d -name '*~' >> $$d/TAR.exclude2; \ - find $$d -name 'erl_crash.dump' >> $$d/TAR.exclude2; \ - find $$d -name 'core' >> $$d/TAR.exclude2; \ - find $$d -name '.cmake.state' >> $$d/TAR.exclude2; \ - tar cfX - $$d/TAR.exclude2 $$d | (cd $(RELSYSDIR)/test; tar xf -); \ + if test -f $$d/TAR.exclude; then \ + echo $$d/TAR.exclude2 > $$d/TAR.exclude2; \ + cat $$d/TAR.exclude >> $$d/TAR.exclude2; \ + find $$d -name '*.contrib*' >> $$d/TAR.exclude2; \ + find $$d -name '*.keep*' >> $$d/TAR.exclude2; \ + find $$d -name '*.mkelem*' >> $$d/TAR.exclude2; \ + find $$d -name '*~' >> $$d/TAR.exclude2; \ + find $$d -name 'erl_crash.dump' >> $$d/TAR.exclude2; \ + find $$d -name 'core' >> $$d/TAR.exclude2; \ + find $$d -name '.cmake.state' >> $$d/TAR.exclude2; \ + tar cfX - $$d/TAR.exclude2 $$d | (cd $(RELSYSDIR)/test; tar xf -); \ + else \ + tar cf - $$d | (cd $(RELSYSDIR)/test; tar xf -); \ + fi; \ done release_tests_spec: opt diff --git a/lib/inets/test/httpc_SUITE.erl b/lib/inets/test/httpc_SUITE.erl index 902e440c80..94d5a48ef6 100644 --- a/lib/inets/test/httpc_SUITE.erl +++ b/lib/inets/test/httpc_SUITE.erl @@ -254,9 +254,14 @@ init_per_testcase(Case, Timeout, Config) -> [{watchdog, Dog}, {local_server, Server} | TmpConfig2] end, + %% httpc:set_options([{proxy, {{?PROXY, ?PROXY_PORT}, + %% ["localhost", ?IPV6_LOCAL_HOST]}}]), + httpc:set_options([{proxy, {{?PROXY, ?PROXY_PORT}, - ["localhost", ?IPV6_LOCAL_HOST]}}]), - %% snmp:set_trace([gen_tcp, inet_tcp, prim_inet]), + ["localhost", ?IPV6_LOCAL_HOST]}}, + {ipfamily, inet6fb4}]), + + %% snmp:set_trace([gen_tcp]), NewConfig. @@ -471,7 +476,7 @@ http_relaxed(Config) when is_list(Config) -> DummyServerPid ! stop, ok = httpc:set_options([{ipv6, enabled}]), - %% ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + %% ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -489,7 +494,7 @@ http_dummy_pipe(Config) when is_list(Config) -> test_pipeline(URL), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. http_inets_pipe(doc) -> @@ -851,7 +856,7 @@ http_headers_dummy(Config) when is_list(Config) -> ], "text/plain", FooBar}, [], []), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -875,7 +880,7 @@ http_bad_response(Config) when is_list(Config) -> test_server:format("Wrong Statusline: ~p~n", [Reason]), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1157,7 +1162,7 @@ http_redirect(Config) when is_list(Config) -> tsp("http_redirect -> stop dummy server"), DummyServerPid ! stop, tsp("http_redirect -> reset ipfamily option (to inet6fb4)"), - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), tsp("http_redirect -> done"), ok; @@ -1181,7 +1186,7 @@ http_redirect_loop(Config) when is_list(Config) -> {ok, {{_,300,_}, [_ | _], _}} = httpc:request(get, {URL, []}, [], []), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. %%------------------------------------------------------------------------- @@ -1215,7 +1220,7 @@ http_internal_server_error(Config) when is_list(Config) -> ets:delete(unavailable), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1242,7 +1247,7 @@ http_userinfo(Config) when is_list(Config) -> httpc:request(get, {URLUnAuth, []}, [], []), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1271,9 +1276,9 @@ http_cookie(Config) when is_list(Config) -> ets:delete(cookie), - ok = httpc:set_options([{cookies, disabled}, {ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{cookies, disabled}]), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6************ + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. %%------------------------------------------------------------------------- @@ -1643,7 +1648,7 @@ http_stream_once(Config) when is_list(Config) -> p("http_stream_once -> stop dummy server", []), DummyServerPid ! stop, p("http_stream_once -> set ipfamily to inet6fb4", []), - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), p("http_stream_once -> done", []), ok. @@ -1847,7 +1852,7 @@ http_invalid_http(Config) when is_list(Config) -> test_server:format("Parse error: ~p ~n", [Reason]), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1901,7 +1906,7 @@ transfer_encoding_otp_6807(Config) when is_list(Config) -> "/capital_transfer_encoding.html", {ok, {{_,200,_}, [_|_], [_ | _]}} = httpc:request(URL), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1933,7 +1938,7 @@ empty_response_header_otp_6830(Config) when is_list(Config) -> URL = ?URL_START ++ integer_to_list(Port) ++ "/no_headers.html", {ok, {{_,200,_}, [], [_ | _]}} = httpc:request(URL), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1950,7 +1955,7 @@ no_content_204_otp_6982(Config) when is_list(Config) -> URL = ?URL_START ++ integer_to_list(Port) ++ "/no_content.html", {ok, {{_,204,_}, [], []}} = httpc:request(URL), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1968,7 +1973,7 @@ missing_CR_otp_7304(Config) when is_list(Config) -> URL = ?URL_START ++ integer_to_list(Port) ++ "/missing_CR.html", {ok, {{_,200,_}, _, [_ | _]}} = httpc:request(URL), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. @@ -1990,7 +1995,7 @@ otp_7883_1(Config) when is_list(Config) -> {error, socket_closed_remotely} = httpc:request(URL), DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. otp_7883_2(doc) -> @@ -2017,7 +2022,7 @@ otp_7883_2(Config) when is_list(Config) -> end, DummyServerPid ! stop, - ok = httpc:set_options([{ipfamily, inet6fb4}]), % ********** ipfamily = inet6 ************* + ok = httpc:set_options([{ipfamily, inet6fb4}]), ok. diff --git a/lib/inets/test/httpd_basic_SUITE.erl b/lib/inets/test/httpd_basic_SUITE.erl index f86c1fcb49..9ba2e73942 100644 --- a/lib/inets/test/httpd_basic_SUITE.erl +++ b/lib/inets/test/httpd_basic_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2007-2009. All Rights Reserved. +%% Copyright Ericsson AB 2007-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -25,13 +25,16 @@ %% Note: This directive should only be used in test suites. -compile(export_all). +-define(URL_START, "http://localhost:"). + all(doc) -> ["Basic test of httpd."]; all(suite) -> [ uri_too_long_414, - header_too_long_413 + header_too_long_413, + escaped_url_in_error_body ]. %%-------------------------------------------------------------------- @@ -131,6 +134,31 @@ header_too_long_413(Config) when is_list(Config) -> {version, "HTTP/1.1"}]), inets:stop(httpd, Pid). +escaped_url_in_error_body(doc) -> + ["Test Url-encoding see OTP-8940"]; +escaped_url_in_error_body(suite) -> + []; +escaped_url_in_error_body(Config) when is_list(Config) -> + HttpdConf = ?config(httpd_conf, Config), + {ok, Pid} = inets:start(httpd, [{port, 0} | HttpdConf]), + Info = httpd:info(Pid), + Port = proplists:get_value(port, Info), + Address = proplists:get_value(bind_address, Info), + Path = "/<b>this_is_bold<b>", + URL = ?URL_START ++ integer_to_list(Port) ++ Path, + EscapedPath = http_uri:encode(Path), + {ok, {404, Body}} = httpc:request(get, {URL, []}, + [{url_encode, true}], + [{version, "HTTP/1.0"}, {full_result, false}]), + EscapedPath = find_URL_path(string:tokens(Body, " ")), + {ok, {404, Body1}} = httpc:request(get, {URL, []}, [], + [{version, "HTTP/1.0"}, {full_result, false}]), + EscapedPath = find_URL_path(string:tokens(Body1, " ")), + inets:stop(httpd, Pid). - - +find_URL_path([]) -> + ""; +find_URL_path(["URL", URL | _]) -> + URL; +find_URL_path([_ | Rest]) -> + find_URL_path(Rest). diff --git a/lib/inets/test/inets_appup_test.erl b/lib/inets/test/inets_appup_test.erl index d580c6c4c5..2c9c687c91 100644 --- a/lib/inets/test/inets_appup_test.erl +++ b/lib/inets/test/inets_appup_test.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2002-2009. All Rights Reserved. +%% Copyright Ericsson AB 2002-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -18,11 +18,12 @@ %% %% %%---------------------------------------------------------------------- -%% Purpose: Verify the application specifics of the Megaco application +%% Purpose: Verify the application specifics of the Inets application %%---------------------------------------------------------------------- -module(inets_appup_test). -compile(export_all). +-compile({no_auto_import,[error/1]}). -include("inets_test_lib.hrl"). diff --git a/lib/inets/test/inets_test_lib.erl b/lib/inets/test/inets_test_lib.erl index 86fc2d1a32..c56a714f5a 100644 --- a/lib/inets/test/inets_test_lib.erl +++ b/lib/inets/test/inets_test_lib.erl @@ -329,6 +329,9 @@ connect(ip_comm, Host, Port, Opts) -> {error, eafnosupport} -> tsp("eafnosupport opts: ~p", [Opts]), connect(ip_comm, Host, Port, lists:delete(inet6, Opts)); + {error, enetunreach} -> + tsp("eafnosupport opts: ~p", [Opts]), + connect(ip_comm, Host, Port, lists:delete(inet6, Opts)); {error, {enfile,_}} -> tsp("Error enfile"), {error, enfile}; diff --git a/lib/inets/vsn.mk b/lib/inets/vsn.mk index 5eff9e4e3f..f462290a99 100644 --- a/lib/inets/vsn.mk +++ b/lib/inets/vsn.mk @@ -1,5 +1,24 @@ +#-*-makefile-*- ; force emacs to enter makefile-mode + +# %CopyrightBegin% +# +# Copyright Ericsson AB 2001-2010. All Rights Reserved. +# +# The contents of this file are subject to the Erlang Public License, +# Version 1.1, (the "License"); you may not use this file except in +# compliance with the License. You should have received a copy of the +# Erlang Public License along with this software. If not, it can be +# retrieved online at http://www.erlang.org/. +# +# Software distributed under the License is distributed on an "AS IS" +# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +# the License for the specific language governing rights and limitations +# under the License. +# +# %CopyrightEnd% + APPLICATION = inets -INETS_VSN = 5.5 +INETS_VSN = 5.5.1 PRE_VSN = APP_VSN = "$(APPLICATION)-$(INETS_VSN)$(PRE_VSN)" diff --git a/lib/jinterface/doc/src/notes.xml b/lib/jinterface/doc/src/notes.xml index a571de6916..879634561b 100644 --- a/lib/jinterface/doc/src/notes.xml +++ b/lib/jinterface/doc/src/notes.xml @@ -30,6 +30,22 @@ </header> <p>This document describes the changes made to the Jinterface application.</p> +<section><title>Jinterface 1.5.3.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The OtpMbox class did not have a hash() method, which it + should have because it overrides equals().</p> + <p> + Own Id: OTP-8854</p> + </item> + </list> + </section> + +</section> + <section><title>Jinterface 1.5.3.1</title> <section><title>Improvements and New Features</title> diff --git a/lib/jinterface/vsn.mk b/lib/jinterface/vsn.mk index ed085b5d4d..24ffe7c5e6 100644 --- a/lib/jinterface/vsn.mk +++ b/lib/jinterface/vsn.mk @@ -1 +1 @@ -JINTERFACE_VSN = 1.5.3.1 +JINTERFACE_VSN = 1.5.3.2 diff --git a/lib/kernel/doc/src/file.xml b/lib/kernel/doc/src/file.xml index 64cdd3a8ea..36fce464c5 100644 --- a/lib/kernel/doc/src/file.xml +++ b/lib/kernel/doc/src/file.xml @@ -36,6 +36,61 @@ other Erlang processes to continue executing in parallel with the file operations. See the command line flag <c>+A</c> in <seealso marker="erts:erl">erl(1)</seealso>.</p> + + <p>The Erlang VM supports file names in Unicode to a limited + extent. Depending on how the VM is started (with the parameter + <c>+fnu</c> or <c>+fnl</c>), file names given can contain + characters > 255 and the VM system will convert file names + back and forth to the native file name encoding.</p> + + <p>The default behavior for Unicode character translation depends + on to what extent the underlying OS/filesystem enforces consistent + naming. On OSes where all file names are ensured to be in one or + another encoding, Unicode is the default (currently this holds for + Windows and MacOSX). On OSes with completely transparent file + naming (i.e. all Unixes except MacOSX), ISO-latin-1 file naming is + the default. The reason for the ISO-latin-1 default is that + file names are not guaranteed to be possible to interpret according to + the Unicode encoding expected (i.e. UTF-8), and file names that + cannot be decoded will only be accessible by using "raw + file names", in other word file names given as binaries.</p> + + <p>As file names are traditionally not binaries in Erlang, + applications that need to handle raw file names need to be + converted, why the Unicode mode for file names is not default on + systems having completely transparent file naming.</p> + + <note>As of R14B01, the most basic file handling modules + (<c>file</c>, <c>prim_file</c>, <c>filelib</c> and + <c>filename</c>) accept raw file names, but the rest of OTP is not + guaranteed to handle them, why Unicode file naming on systems + where it is not default is still considered experimental.</note> + + <p>Raw file names is a new feature in OTP R14B01, which allows the + user to supply completely uninterpreted file names to the + underlying OS/filesystem. They are supplied as binaries, where it + is up to the user to supply a correct encoding for the + environment. The function <c>file:native_name_encoding()</c> can + be used to check what encoding the VM is working in. If the + function returns <c>latin1</c> file names are not in any way + converted to Unicode, if it is <c>utf8</c>, raw file names should + be encoded as UTF-8 if they are to follow the convention of the VM + (and usually the convention of the OS as well). Using raw + file names is useful if you have a filesystem with inconsistent + file naming, where some files are named in UTF-8 encoding while + others are not. A file:list_dir on such mixed file name systems + when the VM is in Unicode file name mode might return file names as + raw binaries as they cannot be interpreted as Unicode + file names. Raw file names can also be used to give UTF-8 encoded + file names even though the VM is not started in Unicode file name + translation mode.</p> + + <p>Note that on Windows, <c>file:native_name_encoding()</c> + returns <c>utf8</c> per default, which is the format for raw + file names even on Windows, although the underlying OS specific + code works in a limited version of little endian UTF16. As far as + the Erlang programmer is concerned, Windows native Unicode format + is UTF-8...</p> </description> <section> @@ -47,8 +102,14 @@ iodata() = iolist() | binary() io_device() as returned by file:open/2, a process handling IO protocols -name() = string() | atom() | DeepList +name() = string() | atom() | DeepList | RawFilename DeepList = [char() | atom() | DeepList] + RawFilename = binary() + If VM is in unicode filename mode, string() and char() are allowed to be > 255. + RawFilename is a filename not subject to Unicode translation, meaning that it + can contain characters not conforming to the Unicode encoding expected from the + filesystem (i.e. non-UTF-8 characters although the VM is started in Unicode + filename mode). posix() an atom which is named from the POSIX error codes used in @@ -598,6 +659,15 @@ f.txt: {person, "kalle", 25}. </desc> </func> <func> + <name>native_name_encoding() -> latin1 | utf8</name> + <fsummary>Return the VM's configured filename encoding.</fsummary> + <desc> + <p>This function returns the configured default file name encoding to use for raw file names. Generally an application supplying file names raw (as binaries), should obey the character encoding returned by this function.</p> + <p>By default, the VM uses ISO-latin-1 file name encoding on filesystems and/or OSes that use completely transparent file naming. This includes all Unix versions except MacOSX, where the vfs layer enforces UTF-8 file naming. By giving the experimental option <c>+fnu</c> when starting Erlang, UTF-8 translation of file names can be turned on even for those systems. If Unicode file name translation is in effect, the system behaves as usual as long as file names conform to the encoding, but will return file names that are not properly encoded in UTF-8 as raw file names (i.e. binaries).</p> + <p>On Windows, this function also returns <c>utf8</c> by default. The OS uses a pure Unicode naming scheme and file names are always possible to interpret as valid Unicode. The fact that the underlying Windows OS actually encodes file names using little endian UTF-16 can be ignored by the Erlang programmer. Windows and MacOSX are the only operating systems where the VM operates in Unicode file name mode by default.</p> + </desc> + </func> + <func> <name>open(Filename, Modes) -> {ok, IoDevice} | {error, Reason}</name> <fsummary>Open a file</fsummary> <type> diff --git a/lib/kernel/doc/src/notes.xml b/lib/kernel/doc/src/notes.xml index edd6ea52b0..29580a4cd1 100644 --- a/lib/kernel/doc/src/notes.xml +++ b/lib/kernel/doc/src/notes.xml @@ -30,6 +30,45 @@ </header> <p>This document describes the changes made to the Kernel application.</p> +<section><title>Kernel 2.14.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The Erlang VM now supports Unicode filenames. The feature + is turned on by default on systems where Unicode + filenames are mandatory (Windows and MacOSX), but can be + enabled on other systems with the '+fnu' emulator option. + Enabling the Unicode filename feature on systems where it + is not default is however considered experimental and not + to be used for production. Together with the Unicode file + name support, the concept of "raw filenames" is + introduced, which means filenames provided without + implicit unicode encoding translation. Raw filenames are + provided as binaries, not lists. For further information, + see stdlib users guide and the chapter about using + Unicode in Erlang. Also see the file module manual page.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8887</p> + </item> + <item> + <p> + There is now a new function inet:getifaddrs/0 modeled + after C library function getifaddrs() on BSD and LInux + that reports existing interfaces and their addresses on + the host. This replaces the undocumented and unsupported + inet:getiflist/0 and inet:ifget/2.</p> + <p> + Own Id: OTP-8926</p> + </item> + </list> + </section> + +</section> + <section><title>Kernel 2.14.1.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/kernel/src/code.erl b/lib/kernel/src/code.erl index ec256d5806..feb5131aad 100644 --- a/lib/kernel/src/code.erl +++ b/lib/kernel/src/code.erl @@ -213,19 +213,20 @@ unstick_mod(Mod) when is_atom(Mod) -> call({unstick_mod,Mod}). -spec is_sticky(Module :: atom()) -> boolean(). is_sticky(Mod) when is_atom(Mod) -> call({is_sticky,Mod}). --spec set_path(Directories :: [file:filename()]) -> 'true' | {'error', term()}. +-spec set_path(Directories :: [file:filename()]) -> + 'true' | {'error', 'bad_directory' | 'bad_path'}. set_path(PathList) when is_list(PathList) -> call({set_path,PathList}). -spec get_path() -> [file:filename()]. get_path() -> call(get_path). --spec add_path(Directory :: file:filename()) -> 'true' | {'error', term()}. +-spec add_path(Directory :: file:filename()) -> 'true' | {'error', 'bad_directory'}. add_path(Dir) when is_list(Dir) -> call({add_path,last,Dir}). --spec add_pathz(Directory :: file:filename()) -> 'true' | {'error', term()}. +-spec add_pathz(Directory :: file:filename()) -> 'true' | {'error', 'bad_directory'}. add_pathz(Dir) when is_list(Dir) -> call({add_path,last,Dir}). --spec add_patha(Directory :: file:filename()) -> 'true' | {'error', term()}. +-spec add_patha(Directory :: file:filename()) -> 'true' | {'error', 'bad_directory'}. add_patha(Dir) when is_list(Dir) -> call({add_path,first,Dir}). -spec add_paths(Directories :: [file:filename()]) -> 'ok'. @@ -237,7 +238,6 @@ add_pathsz(Dirs) when is_list(Dirs) -> call({add_paths,last,Dirs}). -spec add_pathsa(Directories :: [file:filename()]) -> 'ok'. add_pathsa(Dirs) when is_list(Dirs) -> call({add_paths,first,Dirs}). -%% XXX Contract's input argument differs from add_path/1 -- why? -spec del_path(Name :: file:filename() | atom()) -> boolean() | {'error', 'bad_name'}. del_path(Name) when is_list(Name) ; is_atom(Name) -> call({del_path,Name}). @@ -286,6 +286,8 @@ do_start(Flags) -> ets:module_info(module), os:module_info(module), + binary:module_info(module), + unicode:module_info(module), filename:module_info(module), lists:module_info(module), diff --git a/lib/kernel/src/file.erl b/lib/kernel/src/file.erl index 97d914b043..bc95359986 100644 --- a/lib/kernel/src/file.erl +++ b/lib/kernel/src/file.erl @@ -75,7 +75,7 @@ -define(RAM_FILE, ram_file). % Module %% data types --type filename() :: string(). +-type filename() :: string() | binary(). -type file_info() :: #file_info{}. -type fd() :: #file_descriptor{}. -type io_device() :: pid() | fd(). @@ -87,7 +87,7 @@ | 'delayed_write' | {'read_ahead', pos_integer()} | 'read_ahead' | 'compressed' | {'encoding', unicode:encoding()}. --type name() :: string() | atom() | [name()]. +-type name() :: string() | atom() | [name()] | binary(). -type posix() :: 'eacces' | 'eagain' | 'ebadf' | 'ebusy' | 'edquot' | 'eexist' | 'efault' | 'efbig' | 'eintr' | 'einval' | 'eio' | 'eisdir' | 'eloop' | 'emfile' | 'emlink' @@ -183,7 +183,7 @@ make_dir(Name) -> del_dir(Name) -> check_and_call(del_dir, [file_name(Name)]). --spec read_file_info(Name :: name()) -> {'ok', #file_info{}} | {'error', posix()}. +-spec read_file_info(Name :: name()) -> {'ok', file_info()} | {'error', posix()}. read_file_info(Name) -> check_and_call(read_file_info, [file_name(Name)]). @@ -193,7 +193,7 @@ read_file_info(Name) -> altname(Name) -> check_and_call(altname, [file_name(Name)]). --spec read_link_info(Name :: name()) -> {'ok', #file_info{}} | {'error', posix()}. +-spec read_link_info(Name :: name()) -> {'ok', file_info()} | {'error', posix()}. read_link_info(Name) -> check_and_call(read_link_info, [file_name(Name)]). @@ -203,7 +203,7 @@ read_link_info(Name) -> read_link(Name) -> check_and_call(read_link, [file_name(Name)]). --spec write_file_info(Name :: name(), Info :: #file_info{}) -> +-spec write_file_info(Name :: name(), Info :: file_info()) -> 'ok' | {'error', posix()}. write_file_info(Name, Info = #file_info{}) -> @@ -214,7 +214,8 @@ write_file_info(Name, Info = #file_info{}) -> list_dir(Name) -> check_and_call(list_dir, [file_name(Name)]). --spec read_file(Name :: name()) -> {'ok', binary()} | {'error', posix()}. +-spec read_file(Name :: name()) -> + {'ok', binary()} | {'error', posix() | 'terminated' | 'system_limit'}. read_file(Name) -> check_and_call(read_file, [file_name(Name)]). @@ -229,15 +230,15 @@ make_link(Old, New) -> make_symlink(Old, New) -> check_and_call(make_symlink, [file_name(Old), file_name(New)]). --spec write_file(Name :: name(), Bin :: binary()) -> 'ok' | {'error', posix()}. +-spec write_file(Name :: name(), Bin :: iodata()) -> + 'ok' | {'error', posix() | 'terminated' | 'system_limit'}. write_file(Name, Bin) -> check_and_call(write_file, [file_name(Name), make_binary(Bin)]). %% This whole operation should be moved to the file_server and prim_file %% when it is time to change file server protocol again. -%% Meanwhile, it is implemented here, slihtly less efficient. -%% +%% Meanwhile, it is implemented here, slightly less efficient. -spec write_file(Name :: name(), Bin :: binary(), Modes :: [mode()]) -> 'ok' | {'error', posix()}. @@ -348,7 +349,7 @@ open(Item, Mode) -> %%% The File argument must be either a Pid or a handle %%% returned from ?PRIM_FILE:open. --spec close(File :: io_device()) -> 'ok' | {'error', posix()}. +-spec close(File :: io_device()) -> 'ok' | {'error', posix() | 'terminated'}. close(File) when is_pid(File) -> R = file_request(File, close), @@ -367,7 +368,7 @@ close(_) -> {error, badarg}. -spec advise(File :: io_device(), Offset :: integer(), - Length :: integer(), Advise :: posix_file_advise()) -> + Length :: integer(), Advise :: posix_file_advise()) -> 'ok' | {'error', posix()}. advise(File, Offset, Length, Advise) when is_pid(File) -> @@ -449,7 +450,7 @@ pread(_, _, _) -> {error, badarg}. -spec write(File :: io_device() | atom(), Byte :: iodata()) -> - 'ok' | {'error', posix()}. + 'ok' | {'error', posix() | 'terminated'}. write(File, Bytes) when (is_pid(File) orelse is_atom(File)) -> case make_binary(Bytes) of @@ -1033,22 +1034,26 @@ path_open_first([], _Name, _Mode, LastError) -> %% Generates a flat file name from a deep list of atoms and %% characters (integers). +file_name(N) when is_binary(N) -> + N; file_name(N) -> try - file_name_1(N) + file_name_1(N,file:native_name_encoding()) catch Reason -> {error, Reason} end. -file_name_1([C|T]) when is_integer(C), C > 0, C =< 255 -> - [C|file_name_1(T)]; -file_name_1([H|T]) -> - file_name_1(H) ++ file_name_1(T); -file_name_1([]) -> +file_name_1([C|T],latin1) when is_integer(C), C < 256-> + [C|file_name_1(T,latin1)]; +file_name_1([C|T],utf8) when is_integer(C) -> + [C|file_name_1(T,utf8)]; +file_name_1([H|T],E) -> + file_name_1(H,E) ++ file_name_1(T,E); +file_name_1([],_) -> []; -file_name_1(N) when is_atom(N) -> +file_name_1(N,_) when is_atom(N) -> atom_to_list(N); -file_name_1(_) -> +file_name_1(_,_) -> throw(badarg). make_binary(Bin) when is_binary(Bin) -> diff --git a/lib/kernel/src/file_io_server.erl b/lib/kernel/src/file_io_server.erl index 39dc32bb79..14da9c1a55 100644 --- a/lib/kernel/src/file_io_server.erl +++ b/lib/kernel/src/file_io_server.erl @@ -44,11 +44,11 @@ format_error(ErrorId) -> erl_posix_msg:message(ErrorId). start(Owner, FileName, ModeList) - when is_pid(Owner), is_list(FileName), is_list(ModeList) -> + when is_pid(Owner), (is_list(FileName) orelse is_binary(FileName)), is_list(ModeList) -> do_start(spawn, Owner, FileName, ModeList). start_link(Owner, FileName, ModeList) - when is_pid(Owner), is_list(FileName), is_list(ModeList) -> + when is_pid(Owner), (is_list(FileName) orelse is_binary(FileName)), is_list(ModeList) -> do_start(spawn_link, Owner, FileName, ModeList). %%%----------------------------------------------------------------- diff --git a/lib/kernel/test/Makefile b/lib/kernel/test/Makefile index 293c368e2a..f84b343de8 100644 --- a/lib/kernel/test/Makefile +++ b/lib/kernel/test/Makefile @@ -51,6 +51,7 @@ MODULES= \ error_logger_SUITE \ error_logger_warn_SUITE \ file_SUITE \ + file_name_SUITE \ prim_file_SUITE \ ram_file_SUITE \ gen_tcp_api_SUITE \ diff --git a/lib/kernel/test/code_SUITE.erl b/lib/kernel/test/code_SUITE.erl index 6f846ebc56..e52f8a0e37 100644 --- a/lib/kernel/test/code_SUITE.erl +++ b/lib/kernel/test/code_SUITE.erl @@ -645,7 +645,7 @@ analyse([], [This={M,F,A}|Path], Visited, ErrCnt0) -> %% These modules should be loaded by code.erl before %% the code_server is started. OK = [erlang, os, prim_file, erl_prim_loader, init, ets, - code_server, lists, lists_sort, filename, packages, + code_server, lists, lists_sort, unicode, binary, filename, packages, gb_sets, gb_trees, hipe_unified_loader, hipe_bifs, prim_zip, zlib], ErrCnt1 = @@ -674,6 +674,22 @@ analyse2(MFA={_,_,_}, Path, Visited0) -> %%%% We need to check these manually... % fun's are ok as long as they are defined locally. check_funs({'$M_EXPR','$F_EXPR',_}, + [{unicode,characters_to_binary_int,3}, + {unicode,characters_to_binary,3}, + {filename,filename_string_to_binary,1}|_]) -> 0; +check_funs({'$M_EXPR','$F_EXPR',_}, + [{unicode,ml_map,3}, + {unicode,characters_to_binary_int,3}, + {unicode,characters_to_binary,3}, + {filename,filename_string_to_binary,1}|_]) -> 0; +check_funs({'$M_EXPR','$F_EXPR',_}, + [{unicode,do_o_binary2,2}, + {unicode,do_o_binary,2}, + {unicode,o_trans,1}, + {unicode,characters_to_binary_int,3}, + {unicode,characters_to_binary,3}, + {filename,filename_string_to_binary,1}|_]) -> 0; +check_funs({'$M_EXPR','$F_EXPR',_}, [{code_server,load_native_code,4}, {code_server,load_native_code_1,2}, {code_server,load_native_code,2}, @@ -1252,7 +1268,8 @@ on_load_embedded_1(Config) -> ?line LibRoot = code:lib_dir(), ?line LinkName = filename:join(LibRoot, "on_load_app-1.0"), ?line OnLoadApp = filename:join(DataDir, "on_load_app-1.0"), - ?line file:delete(LinkName), + ?line del_link(LinkName), + io:format("LinkName :~p, OnLoadApp: ~p~n",[LinkName,OnLoadApp]), case file:make_symlink(OnLoadApp, LinkName) of {error,enotsup} -> throw({skip,"Support for symlinks required"}); @@ -1281,7 +1298,15 @@ on_load_embedded_1(Config) -> %% Clean up. ?line stop_node(Node), - ?line ok = file:delete(LinkName). + ?line ok = del_link(LinkName). + +del_link(LinkName) -> + case file:delete(LinkName) of + {error,eperm} -> + file:del_dir(LinkName); + Other -> + Other + end. create_boot(Config, Options) -> ?line {ok, OldDir} = file:get_cwd(), diff --git a/lib/kernel/test/code_SUITE_data/on_load_app-1.0/src/on_load_embedded.erl b/lib/kernel/test/code_SUITE_data/on_load_app-1.0/src/on_load_embedded.erl index b7fdd4d9ae..646921026d 100644 --- a/lib/kernel/test/code_SUITE_data/on_load_app-1.0/src/on_load_embedded.erl +++ b/lib/kernel/test/code_SUITE_data/on_load_app-1.0/src/on_load_embedded.erl @@ -8,7 +8,7 @@ run_me() -> LibDir = code:lib_dir(on_load_app), PrivDir = code:priv_dir(on_load_app), LibDir = filename:dirname(PrivDir), - ModPath = code:which(?MODULE), + ModPath = filename:join(filename:split(code:which(?MODULE))), LibDir = filename:dirname(filename:dirname(ModPath)), %% Start a process to remember that the on_load was called. diff --git a/lib/kernel/test/erl_distribution_SUITE.erl b/lib/kernel/test/erl_distribution_SUITE.erl index 21a96f804a..a215ec3608 100644 --- a/lib/kernel/test/erl_distribution_SUITE.erl +++ b/lib/kernel/test/erl_distribution_SUITE.erl @@ -845,13 +845,16 @@ monitor_nodes_otp_6481_test(Config, TestType) when is_list(Config) -> ?line {ok, Node} = start_node(Name, "", this), ?line receive {nodeup, Node} -> ok end, - ?line spawn(Node, + ?line RemotePid = spawn(Node, fun () -> - receive after 1000 -> ok end, - lists:foreach(fun (No) -> - Me ! {NodeMsg, No} - end, - Seq), + receive after 1500 -> ok end, + % infinit loop of msgs + % we want an endless stream of messages and the kill + % the node mercilessly. + % We then want to ensure that the nodedown message arrives + % last ... without garbage after it. + Pid = spawn(fun() -> node_loop_send(Me, NodeMsg, 1) end), + receive {Me, kill_it} -> ok end, halt() end), @@ -860,9 +863,11 @@ monitor_nodes_otp_6481_test(Config, TestType) when is_list(Config) -> %% Verify that '{nodeup, Node}' comes before '{NodeMsg, 1}' (the message %% bringing up the connection). - %%?line no_msgs(500), % Why wait? It fails test sometimes /sverker + ?line no_msgs(500), ?line {nodeup, Node} = receive Msg1 -> Msg1 end, - ?line {NodeMsg, 1} = receive Msg2 -> Msg2 end, + ?line {NodeMsg, 1} = receive Msg2 -> Msg2 end, + % msg stream has begun, kill the node + ?line RemotePid ! {self(), kill_it}, %% Verify that '{nodedown, Node}' comes after the last '{NodeMsg, N}' %% message. @@ -883,6 +888,10 @@ flush_node_msgs(NodeMsg, No) -> OtherMsg -> OtherMsg end. +node_loop_send(Pid, Msg, No) -> + Pid ! {Msg, No}, + node_loop_send(Pid, Msg, No + 1). + monitor_nodes_errors(doc) -> []; monitor_nodes_errors(suite) -> diff --git a/lib/kernel/test/file_SUITE.erl b/lib/kernel/test/file_SUITE.erl index 17c47f871d..47592ddb14 100644 --- a/lib/kernel/test/file_SUITE.erl +++ b/lib/kernel/test/file_SUITE.erl @@ -3268,7 +3268,7 @@ large_file(Config) when is_list(Config) -> {{unix,sunos},{A,B,C}} when A == 5, B == 5, C >= 1; A == 5, B >= 6; A >= 6 -> do_large_file(Config); - {{unix,Unix},_} when Unix =:= linux; Unix =:= darwin -> + {{unix,Unix},_} when Unix =/= sunos -> N = unix_free(Config), io:format("Free: ~w KByte~n", [N]), if N < 5 * (1 bsl 20) -> @@ -3278,7 +3278,7 @@ large_file(Config) when is_list(Config) -> do_large_file(Config) end; _ -> - {skipped,"Only supported on Win32, Linux, or SunOS >= 5.5.1"} + {skipped,"Only supported on Win32, Unix or SunOS >= 5.5.1"} end. unix_free(Config) -> @@ -3290,7 +3290,7 @@ unix_free(Config) -> N. do_large_file(Config) -> - ?line Watchdog = ?t:timetrap(?t:minutes(4)), + ?line Watchdog = ?t:timetrap(?t:minutes(5)), %% ?line Name = filename:join(?config(priv_dir, Config), ?MODULE_STRING ++ "_large_file"), @@ -3329,6 +3329,17 @@ do_large_file(Config) -> ?line {ok,P} = ?FILE_MODULE:position(F, {eof,-L}), ?line {ok,Rs} = ?FILE_MODULE:read(F, L+1), ?line ok = ?FILE_MODULE:close(F), + %% Reopen the file with 'append'; used to fail on Windows causing + %% writes to go to the beginning of the file for files > 4GB. + ?line PL = P + L, + ?line PLL = PL + L, + ?line {ok,F1} = ?FILE_MODULE:open(Name, [raw,read,write,append]), + ?line ok = ?FILE_MODULE:write(F1, R), + ?line {ok,PLL} = ?FILE_MODULE:position(F1, {cur,0}), + ?line {ok,Rs} = ?FILE_MODULE:pread(F1, P, L), + ?line {ok,PL} = ?FILE_MODULE:position(F1, {eof,-L}), + ?line {ok,R} = ?FILE_MODULE:read(F1, L+1), + ?line ok = ?FILE_MODULE:close(F1), %% ?line Mref = erlang:monitor(process, Deleter), ?line Deleter ! {Tester,done}, diff --git a/lib/kernel/test/file_name_SUITE.erl b/lib/kernel/test/file_name_SUITE.erl new file mode 100644 index 0000000000..fea4df8539 --- /dev/null +++ b/lib/kernel/test/file_name_SUITE.erl @@ -0,0 +1,1729 @@ +-module(file_name_SUITE). +%% +%% %CopyrightBegin% +%% +%% Copyright Ericsson AB 1996-2010. All Rights Reserved. +%% +%% The contents of this file are subject to the Erlang Public License, +%% Version 1.1, (the "License"); you may not use this file except in +%% compliance with the License. You should have received a copy of the +%% Erlang Public License along with this software. If not, it can be +%% retrieved online at http://www.erlang.org/. +%% +%% Software distributed under the License is distributed on an "AS IS" +%% basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See +%% the License for the specific language governing rights and limitations +%% under the License. +%% +%% %CopyrightEnd% +%% + +-include("test_server.hrl"). +-include_lib("kernel/include/file.hrl"). + +%% +%% File operations that take filenames as parameters (* not prim_file operation) (** a drive): +%% altname +%% copy (*) +%% del_dir +%% delete +%% get_cwd (**) +%% list_dir +%% make_dir +%% make_link +%% make_symlink +%% open +%% read_file +%% read_file_info +%% read_link +%% read_link_info +%% rename +%% set_cwd +%% write_file +%% write_file_info +%% +%% File operations that opens/uses separate driver port (not connected to file) +%% altname +%% del_dir +%% delete +%% get_cwd +%% list_dir +%% make_dir +%% make_link +%% make_symlink +%% read_file_info +%% read_link +%% read_link_info +%% rename +%% set_cwd +%% write_file_info +%% +%% Operations that use ?FD_DRV in prim_file +%% open +%% read_file +%% write_file +%% +%% +%% Operations that return a filename/path +%% altname +%% get_cwd +%% list_dir +%% read_link + +-export([all/1,init_per_testcase/2, fin_per_testcase/2]). +-export([normal/1,icky/1,very_icky/1,normalize/1]). + + +init_per_testcase(_Func, Config) -> + Dog = test_server:timetrap(test_server:seconds(60)), + [{watchdog,Dog}|Config]. + +fin_per_testcase(_Func, Config) -> + Dog = ?config(watchdog, Config), + test_server:timetrap_cancel(Dog). + + +all(suite) -> + [normal,icky,very_icky,normalize]. + +normalize(suite) -> + []; +normalize(doc) -> + ["Check that filename normalization works"]; +normalize(Config) when is_list(Config) -> + random:seed({1290,431421,830412}), + try + ?line UniMode = file:native_name_encoding() =/= latin1, + if + not UniMode -> + throw(need_unicode_mode); + true -> + ok + end, + ?line Pairs = [rand_comp_decomp(200) || _ <- lists:seq(1,1000)], + case os:type() of + {unix,darwin} -> + ?line [ true = (A =:= prim_file:internal_native2name(B)) || + {A,B} <- Pairs ]; + _ -> + ok + end, + ?line [ true = (A =:= prim_file:internal_normalize_utf8(B)) || + {A,B} <- Pairs ] + + catch + throw:need_unicode_mode -> + io:format("Sorry, can only run in unicode mode.~n"), + {skipped,"VM needs to be started in Unicode filename mode"} + end. + +normal(suite) -> + []; +normal(doc) -> + "Check file operations on normal file names regardless of unicode mode"; +normal(Config) when is_list(Config) -> + {ok,Dir} = file:get_cwd(), + try + Priv = ?config(priv_dir, Config), + file:set_cwd(Priv), + put(file_module,prim_file), + ok = check_normal(prim_file), + put(file_module,file), + ok = check_normal(file) + after + file:set_cwd(Dir) + end. + + +icky(suite) -> + []; +icky(doc) -> + "Check file operations on normal file names regardless of unicode mode"; +icky(Config) when is_list(Config) -> + case hopeless_darwin() of + true -> + {skipped,"This version of darwin does not support icky names at all."}; + false -> + {ok,Dir} = file:get_cwd(), + try + Priv = ?config(priv_dir, Config), + file:set_cwd(Priv), + put(file_module,prim_file), + ok = check_icky(prim_file), + put(file_module,file), + ok = check_icky(file) + after + file:set_cwd(Dir) + end + end. +very_icky(suite) -> + []; +very_icky(doc) -> + "Check file operations on normal file names regardless of unicode mode"; +very_icky(Config) when is_list(Config) -> + case hopeless_darwin() of + true -> + {skipped,"This version of darwin does not support icky names at all."}; + false -> + {ok,Dir} = file:get_cwd(), + try + Priv = ?config(priv_dir, Config), + file:set_cwd(Priv), + put(file_module,prim_file), + case check_very_icky(prim_file) of + need_unicode_mode -> + {skipped,"VM needs to be started in Unicode filename mode"}; + ok -> + put(file_module,file), + ok = check_very_icky(file) + end + after + file:set_cwd(Dir) + end + end. + + +check_normal(Mod) -> + {ok,Dir} = Mod:get_cwd(), + try + ?line make_normal_dir(Mod), + ?line {ok, L0} = Mod:list_dir("."), + ?line L1 = lists:sort(L0), + %erlang:display(L1), + ?line L1 = lists:sort(list(normal_dir())), + ?line {ok,D2} = Mod:get_cwd(), + ?line true = is_list(D2), + ?line case Mod:altname("fil1") of + {error,enotsup} -> + ok; + {ok,LLL} when is_list(LLL) -> + ok + end, + ?line [ true = is_list(El) || El <- L1], + ?line Syms = [ {S,Targ,list_to_binary(get_data(Targ,normal_dir()))} + || {T,S,Targ} <- normal_dir(), T =:= symlink ], + ?line [ {ok, Cont} = Mod:read_file(SymL) || {SymL,_,Cont} <- Syms ], + ?line [ {ok, Targ} = fixlink(Mod:read_link(SymL)) || {SymL,Targ,_} <- Syms ], + ?line chk_cre_dir(Mod,[{directory,"temp_dir",normal_dir()}]), + ?line {ok,BeginAt} = Mod:get_cwd(), + ?line true = is_list(BeginAt), + ?line {error,enoent} = Mod:set_cwd("tmp_dir"), + ?line ok = Mod:set_cwd("temp_dir"), + ?line {ok, NowAt} = Mod:get_cwd(), + ?line true = BeginAt =/= NowAt, + ?line ok = Mod:set_cwd(".."), + ?line {ok,BeginAt} = Mod:get_cwd(), + ?line rm_r(Mod,"temp_dir"), + ?line true = is_list(Dir), + ?line [ true = is_list(FN) || FN <- L0 ], + case has_links() of + true -> + ?line ok = Mod:make_link("fil1","nisse"), + ?line {ok, <<"fil1">>} = Mod:read_file("nisse"), + ?line {ok, #file_info{type = regular}} = Mod:read_link_info("nisse"), + ?line ok = Mod:delete("nisse"), + ?line {ok, <<"fil1">>} = Mod:read_file("fil1"), + ?line {error,enoent} = Mod:read_file("nisse"), + ?line {error,enoent} = Mod:read_link_info("nisse"); + false -> + ok + end, + ?line [ begin + ?line {ok, FD} = Mod:open(Name,[read]), + ?line {ok, Content} = Mod:read(FD,1024), + ?line ok = file:close(FD) + end || {regular,Name,Content} <- normal_dir() ], + ?line [ begin + ?line {ok, FD} = Mod:open(Name,[read,binary]), + ?line BC = list_to_binary(Content), + ?line {ok, BC} = Mod:read(FD,1024), + ?line ok = file:close(FD) + end || {regular,Name,Content} <- normal_dir() ], + ?line Mod:rename("fil1","tmp_fil1"), + ?line {ok, <<"fil1">>} = Mod:read_file("tmp_fil1"), + ?line {error,enoent} = Mod:read_file("fil1"), + ?line Mod:rename("tmp_fil1","fil1"), + ?line {ok, <<"fil1">>} = Mod:read_file("fil1"), + ?line {error,enoent} = Mod:read_file("tmp_fil1"), + ?line {ok,FI} = Mod:read_file_info("fil1"), + ?line NewMode = FI#file_info.mode band (bnot 8#333), + ?line NewMode2 = NewMode bor 8#222, + ?line true = NewMode2 =/= NewMode, + ?line ok = Mod:write_file_info("fil1",FI#file_info{mode = NewMode}), + ?line {ok,#file_info{mode = NewMode}} = Mod:read_file_info("fil1"), + ?line ok = Mod:write_file_info("fil1",FI#file_info{mode = NewMode2}), + ?line {ok,#file_info{mode = NewMode2}} = Mod:read_file_info("fil1"), + ok + after + case Mod:read_file_info("fil1") of + {ok,FII} -> + NewModeI = FII#file_info.mode bor 8#777, + Mod:write_file_info("fil1",FII#file_info{mode = NewModeI}); + _ -> + ok + end, + Mod:set_cwd(Dir), + io:format("Wd now: ~s~n",[Dir]) + end. + +check_icky(Mod) -> + {ok,Dir} = Mod:get_cwd(), + try + ?line true=(length("���") =:= 3), + ?line UniMode = file:native_name_encoding() =/= latin1, + ?line make_icky_dir(Mod), + ?line {ok, L0} = Mod:list_dir("."), + ?line L1 = lists:sort(L0), + io:format("~p ~p~n",[L1,list(icky_dir())]), + ?line L1 = lists:sort(convlist(list(icky_dir()))), + ?line {ok,D2} = Mod:get_cwd(), + ?line true = is_list(D2), +%% Altname only on windows, and there are no non native filenames there +%% ?line case Mod:altname("fil1") of +%% {error,enotsup} -> +%% ok; +%% {ok,LLL} when is_list(LLL) -> +%% ok +%% end, + ?line [ true = ((is_list(El) or (UniMode and is_binary(El)))) || El <- L1], + ?line Syms = [ {S,conv(Targ),list_to_binary(get_data(Targ,icky_dir()))} + || {T,S,Targ} <- icky_dir(), T =:= symlink ], + ?line [ {ok, Cont} = Mod:read_file(SymL) || {SymL,_,Cont} <- Syms ], + ?line [ {ok, Targ} = fixlink(Mod:read_link(SymL)) || {SymL,Targ,_} <- Syms ], + ?line chk_cre_dir(Mod,[{directory,"���_dir",icky_dir()}]), + ?line {ok,BeginAt} = Mod:get_cwd(), + ?line true = is_list(BeginAt), + ?line {error,enoent} = Mod:set_cwd("��_dir"), + ?line ok = Mod:set_cwd("���_dir"), + ?line {ok, NowAt} = Mod:get_cwd(), + ?line true = is_list(NowAt), + ?line true = BeginAt =/= NowAt, + ?line ok = Mod:set_cwd(".."), + ?line {ok,BeginAt} = Mod:get_cwd(), + ?line rm_r2(Mod,"���_dir"), + {OS,TYPE} = os:type(), + % Check that treat_icky really converts to the same as the OS + case UniMode of + true -> + ?line chk_cre_dir(Mod,[{directory,"���_dir",[]}]), + ?line ok = Mod:set_cwd("���_dir"), + ?line ok = Mod:write_file(<<"���">>,<<"hello">>), + ?line Treated = treat_icky(<<"���">>), + ?line {ok,[Treated]} = Mod:list_dir("."), + ?line ok = Mod:delete(<<"���">>), + ?line {ok,[]} = Mod:list_dir("."), + ?line ok = Mod:set_cwd(".."), + ?line rm_r2(Mod,"���_dir"); + false -> + ok + end, + + ?line chk_cre_dir(Mod,[{directory,treat_icky(<<"���_dir">>),icky_dir()}]), + if + UniMode and (OS =/= win32) -> + ?line {error,enoent} = Mod:set_cwd("���_dir"); + true -> + ok + end, + ?line ok = Mod:set_cwd(treat_icky(<<"���_dir">>)), + ?line {ok, NowAt2} = Mod:get_cwd(), + io:format("~p~n",[NowAt2]), + % Cannot create raw unicode-breaking filenames on windows or macos + ?line true = ((((not UniMode) or (OS =:= win32) or (TYPE=:=darwin)) and is_list(NowAt2)) orelse ((UniMode) and is_binary(NowAt2))), + ?line true = BeginAt =/= NowAt2, + ?line ok = Mod:set_cwd(".."), + ?line {ok,BeginAt} = Mod:get_cwd(), + ?line rm_r2(Mod,conv(treat_icky(<<"���_dir">>))), + case has_links() of + true -> + ?line ok = Mod:make_link("fil1","nisse�"), + ?line {ok, <<"fil1">>} = Mod:read_file("nisse�"), + ?line {ok, #file_info{type = regular}} = Mod:read_link_info("nisse�"), + ?line ok = Mod:delete("nisse�"), + ?line ok = Mod:make_link("fil1",treat_icky(<<"nisse�">>)), + ?line {ok, <<"fil1">>} = Mod:read_file(treat_icky(<<"nisse�">>)), + ?line {ok, #file_info{type = regular}} = Mod:read_link_info(treat_icky(<<"nisse�">>)), + ?line ok = Mod:delete(treat_icky(<<"nisse�">>)), + ?line {ok, <<"fil1">>} = Mod:read_file("fil1"), + ?line {error,enoent} = Mod:read_file("nisse�"), + ?line {error,enoent} = Mod:read_link_info("nisse�"), + ?line {error,enoent} = Mod:read_file(treat_icky(<<"nisse�">>)), + ?line {error,enoent} = Mod:read_link_info(treat_icky(<<"nisse�">>)); + false -> + ok + end, + ?line [ begin + ?line {ok, FD} = Mod:open(Name,[read]), + ?line {ok, Content} = Mod:read(FD,1024), + ?line ok = file:close(FD) + end || {regular,Name,Content} <- icky_dir() ], + ?line [ begin + ?line {ok, FD} = Mod:open(Name,[read,binary]), + ?line BC = list_to_binary([Content]), + ?line {ok, BC} = Mod:read(FD,1024), + ?line ok = file:close(FD) + end || {regular,Name,Content} <- icky_dir() ], + ?line Mod:rename("���2","���_fil1"), + ?line {ok, <<"���2">>} = Mod:read_file("���_fil1"), + ?line {error,enoent} = Mod:read_file("���2"), + ?line Mod:rename("���_fil1","���2"), + ?line {ok, <<"���2">>} = Mod:read_file("���2"), + ?line {error,enoent} = Mod:read_file("���_fil1"), + + ?line Mod:rename("���2",treat_icky(<<"���_fil1">>)), + ?line {ok, <<"���2">>} = Mod:read_file(treat_icky(<<"���_fil1">>)), + if + UniMode and (OS =/= win32) -> + {error,enoent} = Mod:read_file("���_fil1"); + true -> + ok + end, + ?line {error,enoent} = Mod:read_file("���2"), + ?line Mod:rename(treat_icky(<<"���_fil1">>),"���2"), + ?line {ok, <<"���2">>} = Mod:read_file("���2"), + ?line {error,enoent} = Mod:read_file("���_fil1"), + ?line {error,enoent} = Mod:read_file(treat_icky(<<"���_fil1">>)), + + ?line {ok,FI} = Mod:read_file_info("���2"), + ?line NewMode = FI#file_info.mode band (bnot 8#333), + ?line NewMode2 = NewMode bor 8#222, + ?line true = NewMode2 =/= NewMode, + ?line ok = Mod:write_file_info("���2",FI#file_info{mode = NewMode}), + ?line {ok,#file_info{mode = NewMode}} = Mod:read_file_info("���2"), + ?line ok = Mod:write_file_info("���2",FI#file_info{mode = NewMode2}), + ?line {ok,#file_info{mode = NewMode2}} = Mod:read_file_info("���2"), + + ?line {ok,FII} = Mod:read_file_info(treat_icky(<<"���5">>)), + ?line true = NewMode2 =/= NewMode, + ?line ok = Mod:write_file_info(treat_icky(<<"���5">>),FII#file_info{mode = NewMode}), + ?line {ok,#file_info{mode = NewMode}} = Mod:read_file_info(treat_icky(<<"���5">>)), + ?line ok = Mod:write_file_info(<<"���5">>,FII#file_info{mode = NewMode2}), + ?line {ok,#file_info{mode = NewMode2}} = Mod:read_file_info(treat_icky(<<"���5">>)), + ok + after + Mod:set_cwd(Dir), + io:format("Wd now: ~s~n",[Dir]) + end. + +check_very_icky(Mod) -> + {ok,Dir} = Mod:get_cwd(), + try + ?line true=(length("���") =:= 3), + ?line UniMode = file:native_name_encoding() =/= latin1, + if + not UniMode -> + throw(need_unicode_mode); + true -> + ok + end, + ?line make_very_icky_dir(Mod), + ?line {ok, L0} = Mod:list_dir("."), + ?line L1 = lists:sort(L0), + ?line L1 = lists:sort(convlist(list(very_icky_dir()))), + ?line {ok,D2} = Mod:get_cwd(), + ?line true = is_list(D2), + ?line [ true = ((is_list(El) or is_binary(El))) || El <- L1], + ?line Syms = [ {S,conv(Targ),list_to_binary(get_data(Targ,very_icky_dir()))} + || {T,S,Targ} <- very_icky_dir(), T =:= symlink ], + ?line [ {ok, Cont} = Mod:read_file(SymL) || {SymL,_,Cont} <- Syms ], + ?line [ {ok, Targ} = fixlink(Mod:read_link(SymL)) || {SymL,Targ,_} <- Syms ], + ?line chk_cre_dir(Mod,[{directory,[1088,1079,1091]++"_dir",very_icky_dir()}]), + ?line {ok,BeginAt} = Mod:get_cwd(), + ?line true = is_list(BeginAt), + ?line {error,enoent} = Mod:set_cwd("��_dir"), + ?line ok = Mod:set_cwd([1088,1079,1091]++"_dir"), + ?line {ok, NowAt} = Mod:get_cwd(), + ?line true = is_list(NowAt), + ?line true = BeginAt =/= NowAt, + ?line ok = Mod:set_cwd(".."), + ?line {ok,BeginAt} = Mod:get_cwd(), + ?line rm_r2(Mod,[1088,1079,1091]++"_dir"), + + case has_links() of + true -> + ?line ok = Mod:make_link("fil1","nisse"++[1088,1079,1091]), + ?line {ok, <<"fil1">>} = + Mod:read_file("nisse"++[1088,1079,1091]), + ?line {ok, #file_info{type = regular}} = + Mod:read_link_info("nisse"++[1088,1079,1091]), + ?line ok = Mod:delete("nisse"++[1088,1079,1091]), + ?line ok = Mod:make_link("fil1",<<"nisse�">>), + ?line {ok, <<"fil1">>} = Mod:read_file(<<"nisse�">>), + ?line {ok, #file_info{type = regular}} = + Mod:read_link_info(<<"nisse�">>), + ?line ok = Mod:delete(<<"nisse�">>), + ?line {ok, <<"fil1">>} = Mod:read_file("fil1"), + ?line {error,enoent} = Mod:read_file("nisse"++[1088,1079,1091]), + ?line {error,enoent} = Mod:read_link_info("nisse"++[1088,1079,1091]), + ?line {error,enoent} = Mod:read_file(<<"nisse�">>), + ?line {error,enoent} = Mod:read_link_info(<<"nisse�">>); + false -> + ok + end, + ?line [ begin + ?line {ok, FD} = Mod:open(Name,[read]), + ?line {ok, Content} = Mod:read(FD,1024), + ?line ok = file:close(FD) + end || {regular,Name,Content} <- very_icky_dir() ], + ?line [ begin + ?line {ok, FD} = Mod:open(Name,[read,binary]), + ?line BC = list_to_binary([Content]), + ?line {ok, BC} = Mod:read(FD,1024), + ?line ok = file:close(FD) + end || {regular,Name,Content} <- very_icky_dir() ], + ?line Mod:rename([956,965,963,954,959,49], + [956,965,963,954,959]++"_fil1"), + ?line {ok, <<"���2">>} = Mod:read_file([956,965,963,954,959]++"_fil1"), + ?line {error,enoent} = Mod:read_file([956,965,963,954,959,49]), + ?line Mod:rename([956,965,963,954,959]++"_fil1",[956,965,963,954,959,49]), + ?line {ok, <<"���2">>} = Mod:read_file([956,965,963,954,959,49]), + ?line {error,enoent} = Mod:read_file([956,965,963,954,959]++"_fil1"), + + ?line {ok,FI} = Mod:read_file_info([956,965,963,954,959,49]), + ?line NewMode = FI#file_info.mode band (bnot 8#333), + ?line NewMode2 = NewMode bor 8#222, + ?line true = NewMode2 =/= NewMode, + ?line ok = Mod:write_file_info([956,965,963,954,959,49], + FI#file_info{mode = NewMode}), + ?line {ok,#file_info{mode = NewMode}} = + Mod:read_file_info([956,965,963,954,959,49]), + ?line ok = Mod:write_file_info([956,965,963,954,959,49], + FI#file_info{mode = NewMode2}), + ?line {ok,#file_info{mode = NewMode2}} = + Mod:read_file_info([956,965,963,954,959,49]), + ?line NumOK0 = case has_links() of + true -> 5; + false -> 3 + end, + ?line NumNOK0 = case has_links() of + true -> 4; + false -> 3 + end, + ?line {NumOK,NumNOK} = case is_binary(treat_icky(<<"foo">>)) of + false -> + {NumOK0+NumNOK0,0}; + true -> + {NumOK0,NumNOK0} + end, + ?line {NumOK,NumNOK} = filelib:fold_files(".",".*",true,fun(_F,{N,M}) when is_list(_F) -> io:format("~ts~n",[_F]),{N+1,M}; (_F,{N,M}) -> io:format("~p~n",[_F]),{N,M+1} end,{0,0}), + ?line ok = filelib:fold_files(".",[1076,1089,1072,124,46,42],true,fun(_F,_) -> ok end,false), + ?line SF3 = unicode:characters_to_binary("���subfil3",file:native_name_encoding()), + ?line Sorted = lists:sort([SF3,<<"���subfil2">>]), + ?line Sorted = lists:sort(filelib:wildcard("*",<<"���subdir2">>)), + ok + catch + throw:need_unicode_mode -> + io:format("Sorry, can only run in unicode mode.~n"), + need_unicode_mode + after + Mod:set_cwd(Dir), + io:format("Wd now: ~s~n",[Dir]) + end. + +%% +%% Utilities +%% + + +rm_rf(Mod,Dir) -> + case Mod:read_link_info(Dir) of + {ok, #file_info{type = directory}} -> + {ok, Content} = Mod:list_dir(Dir), + [ rm_rf(Mod,filename:join(Dir,C)) || C <- Content ], + Mod:del_dir(Dir), + ok; + {ok, #file_info{}} -> + Mod:delete(Dir); + _ -> + ok + end. + +rm_r(Mod,Dir) -> + %erlang:display({rm_r,Dir}), + case Mod:read_link_info(Dir) of + {ok, #file_info{type = directory}} -> + {ok,#file_info{type = directory}} = Mod:read_file_info(Dir), + {ok, Content} = Mod:list_dir(Dir), + [ true = is_list(Part) || Part <- Content ], + [ true = is_list(filename:join(Dir,Part)) || Part <- Content ], + [ rm_r(Mod,filename:join(Dir,C)) || C <- Content ], + ok = Mod:del_dir(Dir), + ok; + {ok, #file_info{type = regular}} -> + {ok,#file_info{type = regular}} = Mod:read_file_info(Dir), + ok = Mod:delete(Dir); + {ok, #file_info{type = symlink}} -> + ok = Mod:delete(Dir) + end. +%% For icky test, allow binaries sometimes +rm_r2(Mod,Dir) -> + %erlang:display({rm_r2,Dir}), + case Mod:read_link_info(Dir) of + {ok, #file_info{type = directory}} -> + {ok,#file_info{type = directory}} = Mod:read_file_info(Dir), + {ok, Content} = Mod:list_dir(Dir), + UniMode = file:native_name_encoding() =/= latin1, + [ true = (is_list(Part) orelse UniMode) || Part <- Content ], + [ true = (is_list(filename:join(Dir,Part)) orelse UniMode) || Part <- Content ], + [ rm_r2(Mod,filename:join(Dir,C)) || C <- Content ], + ok = Mod:del_dir(Dir), + ok; + {ok, #file_info{type = regular}} -> + {ok,#file_info{type = regular}} = Mod:read_file_info(Dir), + ok = Mod:delete(Dir); + {ok, #file_info{type = symlink}} -> + ok = Mod:delete(Dir) + end. +chk_cre_dir(_,[]) -> + ok; +chk_cre_dir(Mod,[{regular,Name,Content}|T]) -> + %io:format("~p~n",[Name]), + ok = Mod:write_file(Name,Content), + chk_cre_dir(Mod,T); +chk_cre_dir(Mod,[{link,Name,Target}|T]) -> + ok = Mod:make_link(Target,Name), + chk_cre_dir(Mod,T); +chk_cre_dir(Mod,[{symlink,Name,Target}|T]) -> + ok = Mod:make_symlink(Target,Name), + chk_cre_dir(Mod,T); +chk_cre_dir(Mod,[{directory,Name,Content}|T]) -> + ok = Mod:make_dir(Name), + %io:format("Content = ~p~n",[Content]), + Content2 = [{Ty,filename:join(Name,N),case Ty of link -> filename:join(Name,C); _ -> C end} || {Ty,N,C} <- Content ], + %io:format("Content2 = ~p~n",[Content2]), + chk_cre_dir(Mod,Content2), + chk_cre_dir(Mod,T). + +has_links() -> + case os:type() of + {win32,_} -> + case os:version() of + {N,NN,_} when (N > 5) andalso (NN >= 1) -> + true; + _ -> + false + end; + _ -> + true + end. + +make_normal_dir(Mod) -> + rm_rf(Mod,"normal_dir"), + Mod:make_dir("normal_dir"), + Mod:set_cwd("normal_dir"), + Mod:write_file("fil1","fil1"), + Mod:write_file("fil2","fil2"), + case has_links() of + true -> + Mod:make_link("fil2","fil3"), + Mod:make_symlink("fil2","fil4"); + _ -> + ok + end, + Mod:make_dir("subdir"), + Mod:write_file(filename:join("subdir","subfil1"),"subfil1"), + ok. + +normal_dir() -> + [{regular,"fil1","fil1"}, + {regular,"fil2","fil2"}] ++ + case has_links() of + true -> + [{regular,"fil3","fil2"}, + {symlink,"fil4","fil2"}]; + false -> + [] + end ++ + [{directory,"subdir", + [{regular,"subfil1","subfil1"}]}]. + +make_icky_dir(Mod) -> + rm_rf(Mod,"icky_dir"), + Icky=icky_dir(), + chk_cre_dir(Mod,[{directory,"icky_dir",linkify([],Icky)}]), + Mod:set_cwd("icky_dir"), + ok. + +linkify(_Passed,[]) -> + []; +linkify(Passed,[{regular,Name,Content}|T]) -> + Regulars = [ {N,C} || {regular,N,C} <- Passed, N =/= Name ], + case lists:keysearch(Content,2,Regulars) of + {value, {Linkto, Content}} -> + [{link,Name,Linkto} | linkify(Passed,T)]; + _ -> + [{regular,Name,Content} | linkify([{regular,Name,Content}|Passed],T)] + end; +linkify(Passed,[{directory, Name, Content}|T]) -> + [{directory,Name, linkify(Content,Content)}|linkify(Passed,T)]; +linkify(Passed,[H|T]) -> + [H|linkify([H|Passed],T)]. + +hopeless_darwin() -> + case {os:type(),os:version()} of + {{unix,darwin},{Major,_,_}} when Major < 9 -> + true; + _ -> + false + end. + +icky_dir() -> + [{regular,"fil1","fil1"}, + {regular,"���2","���2"}] ++ + case has_links() of + true -> + [{regular,"���3","���2"}, + {symlink,"���4","���2"}]; + false -> + [] + end ++ + [{regular,treat_icky(<<"���5">>),"���5"}] ++ + case has_links() of + true -> + [{symlink,treat_icky(<<"���6">>),treat_icky(<<"���5">>)}]; + false -> + [] + end ++ + [{directory,treat_icky(<<"���subdir2">>), + [{regular,treat_icky(<<"���subfil2">>),"���subfil12"}, + {regular,"���subfil3","���subfil13"}]}, + {directory,"���subdir", + [{regular,"���subfil1","���subfil1"}]}]. + +make_very_icky_dir(Mod) -> + rm_rf(Mod,"very_icky_dir"), + Icky=very_icky_dir(), + chk_cre_dir(Mod,[{directory,"very_icky_dir",linkify([],Icky)}]), + Mod:set_cwd("very_icky_dir"), + ok. + +very_icky_dir() -> + [{regular,"fil1","fil1"}, + {regular,[956,965,963,954,959,49],"���2"}] ++ + case has_links() of + true -> + [{regular,[956,965,963,954,959,50],"���2"}, + {symlink,[956,965,963,954,959,51],[956,965,963,954,959,49]}]; + false -> + [] + end ++ + [{regular,treat_icky(<<"���5">>),"���5"}] ++ + case has_links() of + true -> + [{symlink,treat_icky(<<"���6">>),treat_icky(<<"���5">>)}]; + false -> + [] + end ++ + [{directory,treat_icky(<<"���subdir2">>), + [{regular,treat_icky(<<"���subfil2">>),"���subfil12"}, + {regular,"���subfil3","���subfil13"}]}, + {directory,[956,965,963,954,959]++"subdir1", + [{regular,[956,965,963,954,959]++"subfil1","���subfil1"}]}]. + +%% Some OS'es simply do not allow non UTF8 filenames +treat_icky(Bin) -> + case os:type() of + {unix,darwin} -> + binary_to_list(procentify(Bin)); + {win32,_} -> + binary_to_list(Bin); + _ -> + Bin + end. + +% Handle windows having absolute soft link targets. +fixlink({ok,Link}) -> + case os:type() of + {win32,_} -> + {ok,filename:basename(Link)}; + _ -> + {ok,Link} + end; +fixlink(X) -> + X. + +procentify(<<>>) -> + <<>>; +procentify(<<X:8,Rst/binary>>) when X > 127 -> + T=procentify(Rst), + Y = list_to_binary([$% + | io_lib:format("~2.16B",[X])]), + <<Y/binary,T/binary>>; +procentify(<<X:8,Rst/binary>>) -> + T=procentify(Rst), + <<X:8,T/binary>>. + + +list([]) -> + []; +list([{_,Name,_} | T]) -> + [Name | list(T)]. + + +get_data(FN,List) -> + case lists:keysearch(FN,2,List) of + {value,{regular,FN,C}} -> + C; + {value,{symlink,FN,NewFN}} -> + get_data(NewFN,List); + _-> + [] + end. + + +convlist(L) -> + convlist(file:native_name_encoding(),L). +convlist(latin1,[Bin|T]) when is_binary(Bin) -> + %erlang:display('Convert...'), + [binary_to_list(Bin)| convlist(latin1,T)]; +convlist(Any,[H|T]) -> + [H|convlist(Any,T)]; +convlist(_,[]) -> + []. + +conv(L) -> + NoUniMode = file:native_name_encoding() =:= latin1, + if + NoUniMode, is_binary(L) -> + binary_to_list(L); + true -> + L + end. + + +rand_comp_decomp(Max) -> + N = random:uniform(Max), + L = [ rand_decomp() || _ <- lists:seq(1,N) ], + LC = [ A || {A,_} <- L], + LD = lists:flatten([B || {_,B} <- L]), + LB = unicode:characters_to_binary(LD,unicode,utf8), + {LC,LB}. + +rand_decomp() -> + BT = bigtup(), + SZ = tuple_size(BT), + element(random:uniform(SZ),BT). +bigtup() -> + {{192,[65,768]}, + {200,[69,768]}, + {204,[73,768]}, + {210,[79,768]}, + {217,[85,768]}, + {7808,[87,768]}, + {7922,[89,768]}, + {224,[97,768]}, + {232,[101,768]}, + {236,[105,768]}, + {242,[111,768]}, + {249,[117,768]}, + {7809,[119,768]}, + {7923,[121,768]}, + {8173,[168,768]}, + {7846,[65,770,768]}, + {7872,[69,770,768]}, + {7890,[79,770,768]}, + {7847,[97,770,768]}, + {7873,[101,770,768]}, + {7891,[111,770,768]}, + {7700,[69,772,768]}, + {7760,[79,772,768]}, + {7701,[101,772,768]}, + {7761,[111,772,768]}, + {7856,[65,774,768]}, + {7857,[97,774,768]}, + {475,[85,776,768]}, + {476,[117,776,768]}, + {8146,[953,776,768]}, + {8162,[965,776,768]}, + {8074,[913,837,787,768]}, + {8090,[919,837,787,768]}, + {8106,[937,837,787,768]}, + {8066,[945,837,787,768]}, + {8082,[951,837,787,768]}, + {8098,[969,837,787,768]}, + {7946,[913,787,768]}, + {7962,[917,787,768]}, + {7978,[919,787,768]}, + {7994,[921,787,768]}, + {8010,[927,787,768]}, + {8042,[937,787,768]}, + {7938,[945,787,768]}, + {7954,[949,787,768]}, + {7970,[951,787,768]}, + {7986,[953,787,768]}, + {8002,[959,787,768]}, + {8018,[965,787,768]}, + {8034,[969,787,768]}, + {8075,[913,837,788,768]}, + {8091,[919,837,788,768]}, + {8107,[937,837,788,768]}, + {8067,[945,837,788,768]}, + {8083,[951,837,788,768]}, + {8099,[969,837,788,768]}, + {7947,[913,788,768]}, + {7963,[917,788,768]}, + {7979,[919,788,768]}, + {7995,[921,788,768]}, + {8011,[927,788,768]}, + {8027,[933,788,768]}, + {8043,[937,788,768]}, + {7939,[945,788,768]}, + {7955,[949,788,768]}, + {7971,[951,788,768]}, + {7987,[953,788,768]}, + {8003,[959,788,768]}, + {8019,[965,788,768]}, + {8035,[969,788,768]}, + {7900,[79,795,768]}, + {7914,[85,795,768]}, + {7901,[111,795,768]}, + {7915,[117,795,768]}, + {8114,[945,837,768]}, + {8130,[951,837,768]}, + {8178,[969,837,768]}, + {8122,[913,768]}, + {8136,[917,768]}, + {8138,[919,768]}, + {8154,[921,768]}, + {8184,[927,768]}, + {8170,[933,768]}, + {8186,[937,768]}, + {8048,[945,768]}, + {8050,[949,768]}, + {8052,[951,768]}, + {8054,[953,768]}, + {8056,[959,768]}, + {8058,[965,768]}, + {8060,[969,768]}, + {8141,[8127,768]}, + {8157,[8190,768]}, + {193,[65,769]}, + {262,[67,769]}, + {201,[69,769]}, + {500,[71,769]}, + {205,[73,769]}, + {7728,[75,769]}, + {313,[76,769]}, + {7742,[77,769]}, + {323,[78,769]}, + {211,[79,769]}, + {7764,[80,769]}, + {340,[82,769]}, + {346,[83,769]}, + {218,[85,769]}, + {7810,[87,769]}, + {221,[89,769]}, + {377,[90,769]}, + {225,[97,769]}, + {263,[99,769]}, + {233,[101,769]}, + {501,[103,769]}, + {237,[105,769]}, + {7729,[107,769]}, + {314,[108,769]}, + {7743,[109,769]}, + {324,[110,769]}, + {243,[111,769]}, + {7765,[112,769]}, + {341,[114,769]}, + {347,[115,769]}, + {250,[117,769]}, + {7811,[119,769]}, + {253,[121,769]}, + {378,[122,769]}, + {8174,[168,769]}, + {508,[198,769]}, + {510,[216,769]}, + {509,[230,769]}, + {511,[248,769]}, + {7844,[65,770,769]}, + {7870,[69,770,769]}, + {7888,[79,770,769]}, + {7845,[97,770,769]}, + {7871,[101,770,769]}, + {7889,[111,770,769]}, + {7756,[79,771,769]}, + {7800,[85,771,769]}, + {7757,[111,771,769]}, + {7801,[117,771,769]}, + {7702,[69,772,769]}, + {7762,[79,772,769]}, + {7703,[101,772,769]}, + {7763,[111,772,769]}, + {7854,[65,774,769]}, + {7855,[97,774,769]}, + {7726,[73,776,769]}, + {471,[85,776,769]}, + {7727,[105,776,769]}, + {472,[117,776,769]}, + {8147,[953,776,769]}, + {8163,[965,776,769]}, + {506,[65,778,769]}, + {507,[97,778,769]}, + {8076,[913,837,787,769]}, + {8092,[919,837,787,769]}, + {8108,[937,837,787,769]}, + {8068,[945,837,787,769]}, + {8084,[951,837,787,769]}, + {8100,[969,837,787,769]}, + {7948,[913,787,769]}, + {7964,[917,787,769]}, + {7980,[919,787,769]}, + {7996,[921,787,769]}, + {8012,[927,787,769]}, + {8044,[937,787,769]}, + {7940,[945,787,769]}, + {7956,[949,787,769]}, + {7972,[951,787,769]}, + {7988,[953,787,769]}, + {8004,[959,787,769]}, + {8020,[965,787,769]}, + {8036,[969,787,769]}, + {8077,[913,837,788,769]}, + {8093,[919,837,788,769]}, + {8109,[937,837,788,769]}, + {8069,[945,837,788,769]}, + {8085,[951,837,788,769]}, + {8101,[969,837,788,769]}, + {7949,[913,788,769]}, + {7965,[917,788,769]}, + {7981,[919,788,769]}, + {7997,[921,788,769]}, + {8013,[927,788,769]}, + {8029,[933,788,769]}, + {8045,[937,788,769]}, + {7941,[945,788,769]}, + {7957,[949,788,769]}, + {7973,[951,788,769]}, + {7989,[953,788,769]}, + {8005,[959,788,769]}, + {8021,[965,788,769]}, + {8037,[969,788,769]}, + {7898,[79,795,769]}, + {7912,[85,795,769]}, + {7899,[111,795,769]}, + {7913,[117,795,769]}, + {7688,[67,807,769]}, + {7689,[99,807,769]}, + {8116,[945,837,769]}, + {8132,[951,837,769]}, + {8180,[959,837,769]}, + {8123,[913,769]}, + {8137,[917,769]}, + {8139,[919,769]}, + {8155,[921,769]}, + {8185,[927,769]}, + {8171,[933,769]}, + {8187,[937,769]}, + {8049,[945,769]}, + {8051,[949,769]}, + {8053,[951,769]}, + {8055,[953,769]}, + {8057,[959,769]}, + {8059,[965,769]}, + {8061,[969,769]}, + {1027,[1043,769]}, + {1036,[1050,769]}, + {1107,[1075,769]}, + {1116,[1082,769]}, + {8142,[8127,769]}, + {8158,[8190,769]}, + {194,[65,770]}, + {264,[67,770]}, + {202,[69,770]}, + {284,[71,770]}, + {292,[72,770]}, + {206,[73,770]}, + {308,[74,770]}, + {212,[79,770]}, + {348,[83,770]}, + {219,[85,770]}, + {372,[87,770]}, + {374,[89,770]}, + {7824,[90,770]}, + {226,[97,770]}, + {265,[99,770]}, + {234,[101,770]}, + {285,[103,770]}, + {293,[104,770]}, + {238,[105,770]}, + {309,[106,770]}, + {244,[111,770]}, + {349,[115,770]}, + {251,[117,770]}, + {373,[119,770]}, + {375,[121,770]}, + {7825,[122,770]}, + {7852,[65,803,770]}, + {7878,[69,803,770]}, + {7896,[79,803,770]}, + {7853,[97,803,770]}, + {7879,[101,803,770]}, + {7897,[111,803,770]}, + {195,[65,771]}, + {7868,[69,771]}, + {296,[73,771]}, + {209,[78,771]}, + {213,[79,771]}, + {360,[85,771]}, + {7804,[86,771]}, + {7928,[89,771]}, + {227,[97,771]}, + {7869,[101,771]}, + {297,[105,771]}, + {241,[110,771]}, + {245,[111,771]}, + {361,[117,771]}, + {7805,[118,771]}, + {7929,[121,771]}, + {7850,[65,770,771]}, + {7876,[69,770,771]}, + {7894,[79,770,771]}, + {7851,[97,770,771]}, + {7877,[101,770,771]}, + {7895,[111,770,771]}, + {7860,[65,774,771]}, + {7861,[97,774,771]}, + {7904,[79,795,771]}, + {7918,[85,795,771]}, + {7905,[111,795,771]}, + {7919,[117,795,771]}, + {256,[65,772]}, + {274,[69,772]}, + {7712,[71,772]}, + {298,[73,772]}, + {332,[79,772]}, + {362,[85,772]}, + {257,[97,772]}, + {275,[101,772]}, + {7713,[103,772]}, + {299,[105,772]}, + {333,[111,772]}, + {363,[117,772]}, + {482,[198,772]}, + {483,[230,772]}, + {480,[65,775,772]}, + {481,[97,775,772]}, + {478,[65,776,772]}, + {469,[85,776,772]}, + {479,[97,776,772]}, + {470,[117,776,772]}, + {7736,[76,803,772]}, + {7772,[82,803,772]}, + {7737,[108,803,772]}, + {7773,[114,803,772]}, + {492,[79,808,772]}, + {493,[111,808,772]}, + {8121,[913,772]}, + {8153,[921,772]}, + {8169,[933,772]}, + {8113,[945,772]}, + {8145,[953,772]}, + {8161,[965,772]}, + {1250,[1048,772]}, + {1262,[1059,772]}, + {1251,[1080,772]}, + {1263,[1091,772]}, + {258,[65,774]}, + {276,[69,774]}, + {286,[71,774]}, + {300,[73,774]}, + {334,[79,774]}, + {364,[85,774]}, + {259,[97,774]}, + {277,[101,774]}, + {287,[103,774]}, + {301,[105,774]}, + {335,[111,774]}, + {365,[117,774]}, + {7862,[65,803,774]}, + {7863,[97,803,774]}, + {7708,[69,807,774]}, + {7709,[101,807,774]}, + {8120,[913,774]}, + {8152,[921,774]}, + {8168,[933,774]}, + {8112,[945,774]}, + {8144,[953,774]}, + {8160,[965,774]}, + {1232,[1040,774]}, + {1238,[1045,774]}, + {1217,[1046,774]}, + {1049,[1048,774]}, + {1038,[1059,774]}, + {1233,[1072,774]}, + {1239,[1077,774]}, + {1218,[1078,774]}, + {1081,[1080,774]}, + {1118,[1091,774]}, + {7682,[66,775]}, + {266,[67,775]}, + {7690,[68,775]}, + {278,[69,775]}, + {7710,[70,775]}, + {288,[71,775]}, + {7714,[72,775]}, + {304,[73,775]}, + {7744,[77,775]}, + {7748,[78,775]}, + {7766,[80,775]}, + {7768,[82,775]}, + {7776,[83,775]}, + {7786,[84,775]}, + {7814,[87,775]}, + {7818,[88,775]}, + {7822,[89,775]}, + {379,[90,775]}, + {7683,[98,775]}, + {267,[99,775]}, + {7691,[100,775]}, + {279,[101,775]}, + {7711,[102,775]}, + {289,[103,775]}, + {7715,[104,775]}, + {7745,[109,775]}, + {7749,[110,775]}, + {7767,[112,775]}, + {7769,[114,775]}, + {7777,[115,775]}, + {7787,[116,775]}, + {7815,[119,775]}, + {7819,[120,775]}, + {7823,[121,775]}, + {380,[122,775]}, + {7835,[383,775]}, + {7780,[83,769,775]}, + {7781,[115,769,775]}, + {784,[774,775]}, + {7782,[83,780,775]}, + {7783,[115,780,775]}, + {7784,[83,803,775]}, + {7785,[115,803,775]}, + {196,[65,776]}, + {203,[69,776]}, + {7718,[72,776]}, + {207,[73,776]}, + {214,[79,776]}, + {220,[85,776]}, + {7812,[87,776]}, + {7820,[88,776]}, + {376,[89,776]}, + {228,[97,776]}, + {235,[101,776]}, + {7719,[104,776]}, + {239,[105,776]}, + {246,[111,776]}, + {7831,[116,776]}, + {252,[117,776]}, + {7813,[119,776]}, + {7821,[120,776]}, + {255,[121,776]}, + {1242,[399,776]}, + {1258,[415,776]}, + {1243,[601,776]}, + {1259,[629,776]}, + {7758,[79,771,776]}, + {7759,[111,771,776]}, + {7802,[85,772,776]}, + {7803,[117,772,776]}, + {938,[921,776]}, + {939,[933,776]}, + {970,[953,776]}, + {971,[965,776]}, + {980,[978,776]}, + {1031,[1030,776]}, + {1234,[1040,776]}, + {1025,[1045,776]}, + {1244,[1046,776]}, + {1246,[1047,776]}, + {1252,[1048,776]}, + {1254,[1054,776]}, + {1264,[1059,776]}, + {1268,[1063,776]}, + {1272,[1067,776]}, + {1235,[1072,776]}, + {1105,[1077,776]}, + {1245,[1078,776]}, + {1247,[1079,776]}, + {1253,[1080,776]}, + {1255,[1086,776]}, + {1265,[1091,776]}, + {1269,[1095,776]}, + {1273,[1099,776]}, + {1111,[1110,776]}, + {7842,[65,777]}, + {7866,[69,777]}, + {7880,[73,777]}, + {7886,[79,777]}, + {7910,[85,777]}, + {7926,[89,777]}, + {7843,[97,777]}, + {7867,[101,777]}, + {7881,[105,777]}, + {7887,[111,777]}, + {7911,[117,777]}, + {7927,[121,777]}, + {7848,[65,770,777]}, + {7874,[69,770,777]}, + {7892,[79,770,777]}, + {7849,[97,770,777]}, + {7875,[101,770,777]}, + {7893,[111,770,777]}, + {7858,[65,774,777]}, + {7859,[97,774,777]}, + {7902,[79,795,777]}, + {7916,[85,795,777]}, + {7903,[111,795,777]}, + {7917,[117,795,777]}, + {197,[65,778]}, + {366,[85,778]}, + {229,[97,778]}, + {367,[117,778]}, + {7832,[119,778]}, + {7833,[121,778]}, + {336,[79,779]}, + {368,[85,779]}, + {337,[111,779]}, + {369,[117,779]}, + {1266,[1059,779]}, + {1267,[1091,779]}, + {461,[65,780]}, + {268,[67,780]}, + {270,[68,780]}, + {282,[69,780]}, + {486,[71,780]}, + {463,[73,780]}, + {488,[75,780]}, + {317,[76,780]}, + {327,[78,780]}, + {465,[79,780]}, + {344,[82,780]}, + {352,[83,780]}, + {356,[84,780]}, + {467,[85,780]}, + {381,[90,780]}, + {462,[97,780]}, + {269,[99,780]}, + {271,[100,780]}, + {283,[101,780]}, + {487,[103,780]}, + {464,[105,780]}, + {496,[106,780]}, + {489,[107,780]}, + {318,[108,780]}, + {328,[110,780]}, + {466,[111,780]}, + {345,[114,780]}, + {353,[115,780]}, + {357,[116,780]}, + {468,[117,780]}, + {382,[122,780]}, + {494,[439,780]}, + {495,[658,780]}, + {473,[85,776,780]}, + {474,[117,776,780]}, + {901,[168,781]}, + {912,[953,776,781]}, + {944,[965,776,781]}, + {902,[913,781]}, + {904,[917,781]}, + {905,[919,781]}, + {906,[921,781]}, + {908,[927,781]}, + {910,[933,781]}, + {911,[937,781]}, + {940,[945,781]}, + {941,[949,781]}, + {942,[951,781]}, + {943,[953,781]}, + {972,[959,781]}, + {973,[965,781]}, + {974,[969,781]}, + {979,[978,781]}, + {512,[65,783]}, + {516,[69,783]}, + {520,[73,783]}, + {524,[79,783]}, + {528,[82,783]}, + {532,[85,783]}, + {513,[97,783]}, + {517,[101,783]}, + {521,[105,783]}, + {525,[111,783]}, + {529,[114,783]}, + {533,[117,783]}, + {1142,[1140,783]}, + {1143,[1141,783]}, + {514,[65,785]}, + {518,[69,785]}, + {522,[73,785]}, + {526,[79,785]}, + {530,[82,785]}, + {534,[85,785]}, + {515,[97,785]}, + {519,[101,785]}, + {523,[105,785]}, + {527,[111,785]}, + {531,[114,785]}, + {535,[117,785]}, + {8072,[913,837,787]}, + {8088,[919,837,787]}, + {8104,[937,837,787]}, + {8064,[945,837,787]}, + {8080,[951,837,787]}, + {8096,[969,837,787]}, + {7944,[913,787]}, + {7960,[917,787]}, + {7976,[919,787]}, + {7992,[921,787]}, + {8008,[927,787]}, + {8040,[937,787]}, + {7936,[945,787]}, + {7952,[949,787]}, + {7968,[951,787]}, + {7984,[953,787]}, + {8000,[959,787]}, + {8164,[961,787]}, + {8016,[965,787]}, + {8032,[969,787]}, + {8073,[913,837,788]}, + {8089,[919,837,788]}, + {8105,[937,837,788]}, + {8065,[945,837,788]}, + {8081,[951,837,788]}, + {8097,[969,837,788]}, + {7945,[913,788]}, + {7961,[917,788]}, + {7977,[919,788]}, + {7993,[921,788]}, + {8009,[927,788]}, + {8172,[929,788]}, + {8025,[933,788]}, + {8041,[937,788]}, + {7937,[945,788]}, + {7953,[949,788]}, + {7969,[951,788]}, + {7985,[953,788]}, + {8001,[959,788]}, + {8165,[961,788]}, + {8017,[965,788]}, + {8033,[969,788]}, + {416,[79,795]}, + {431,[85,795]}, + {417,[111,795]}, + {432,[117,795]}, + {7840,[65,803]}, + {7684,[66,803]}, + {7692,[68,803]}, + {7864,[69,803]}, + {7716,[72,803]}, + {7882,[73,803]}, + {7730,[75,803]}, + {7734,[76,803]}, + {7746,[77,803]}, + {7750,[78,803]}, + {7884,[79,803]}, + {7770,[82,803]}, + {7778,[83,803]}, + {7788,[84,803]}, + {7908,[85,803]}, + {7806,[86,803]}, + {7816,[87,803]}, + {7924,[89,803]}, + {7826,[90,803]}, + {7841,[97,803]}, + {7685,[98,803]}, + {7693,[100,803]}, + {7865,[101,803]}, + {7717,[104,803]}, + {7883,[105,803]}, + {7731,[107,803]}, + {7735,[108,803]}, + {7747,[109,803]}, + {7751,[110,803]}, + {7885,[111,803]}, + {7771,[114,803]}, + {7779,[115,803]}, + {7789,[116,803]}, + {7909,[117,803]}, + {7807,[118,803]}, + {7817,[119,803]}, + {7925,[121,803]}, + {7827,[122,803]}, + {7906,[79,795,803]}, + {7920,[85,795,803]}, + {7907,[111,795,803]}, + {7921,[117,795,803]}, + {7794,[85,804]}, + {7795,[117,804]}, + {7680,[65,805]}, + {7681,[97,805]}, + {199,[67,807]}, + {7696,[68,807]}, + {290,[71,807]}, + {7720,[72,807]}, + {310,[75,807]}, + {315,[76,807]}, + {325,[78,807]}, + {342,[82,807]}, + {350,[83,807]}, + {354,[84,807]}, + {231,[99,807]}, + {7697,[100,807]}, + {291,[103,807]}, + {7721,[104,807]}, + {311,[107,807]}, + {316,[108,807]}, + {326,[110,807]}, + {343,[114,807]}, + {351,[115,807]}, + {355,[116,807]}, + {260,[65,808]}, + {280,[69,808]}, + {302,[73,808]}, + {490,[79,808]}, + {370,[85,808]}, + {261,[97,808]}, + {281,[101,808]}, + {303,[105,808]}, + {491,[111,808]}, + {371,[117,808]}, + {7698,[68,813]}, + {7704,[69,813]}, + {7740,[76,813]}, + {7754,[78,813]}, + {7792,[84,813]}, + {7798,[85,813]}, + {7699,[100,813]}, + {7705,[101,813]}, + {7741,[108,813]}, + {7755,[110,813]}, + {7793,[116,813]}, + {7799,[117,813]}, + {7722,[72,814]}, + {7723,[104,814]}, + {7706,[69,816]}, + {7724,[73,816]}, + {7796,[85,816]}, + {7707,[101,816]}, + {7725,[105,816]}, + {7797,[117,816]}, + {7686,[66,817]}, + {7694,[68,817]}, + {7732,[75,817]}, + {7738,[76,817]}, + {7752,[78,817]}, + {7774,[82,817]}, + {7790,[84,817]}, + {7828,[90,817]}, + {7687,[98,817]}, + {7695,[100,817]}, + {7830,[104,817]}, + {7733,[107,817]}, + {7739,[108,817]}, + {7753,[110,817]}, + {7775,[114,817]}, + {7791,[116,817]}, + {7829,[122,817]}, + {8129,[168,834]}, + {8151,[953,776,834]}, + {8167,[965,776,834]}, + {8078,[913,837,787,834]}, + {8094,[919,837,787,834]}, + {8110,[937,837,787,834]}, + {8070,[945,837,787,834]}, + {8086,[951,837,787,834]}, + {8102,[969,837,787,834]}, + {7950,[913,787,834]}, + {7982,[919,787,834]}, + {7998,[921,787,834]}, + {8046,[937,787,834]}, + {7942,[945,787,834]}, + {7974,[951,787,834]}, + {7990,[953,787,834]}, + {8022,[965,787,834]}, + {8038,[969,787,834]}, + {8079,[913,837,788,834]}, + {8095,[919,837,788,834]}, + {8111,[937,837,788,834]}, + {8071,[945,837,788,834]}, + {8087,[951,837,788,834]}, + {8103,[969,837,788,834]}, + {7951,[913,788,834]}, + {7983,[919,788,834]}, + {7999,[921,788,834]}, + {8031,[933,788,834]}, + {8047,[937,788,834]}, + {7943,[945,788,834]}, + {7975,[951,788,834]}, + {7991,[953,788,834]}, + {8023,[965,788,834]}, + {8039,[969,788,834]}, + {8119,[945,837,834]}, + {8135,[951,837,834]}, + {8183,[969,837,834]}, + {8118,[945,834]}, + {8134,[951,834]}, + {8150,[953,834]}, + {8166,[965,834]}, + {8182,[969,834]}, + {8143,[8127,834]}, + {8159,[8190,834]}, + {8124,[913,837]}, + {8140,[919,837]}, + {8188,[937,837]}, + {8115,[945,837]}, + {8131,[951,837]}, + {8179,[969,837]}, + {64302,[1488,1463]}, + {64287,[1522,1463]}, + {64303,[1488,1464]}, + {64331,[1493,1465]}, + {64304,[1488,1468]}, + {64305,[1489,1468]}, + {64306,[1490,1468]}, + {64307,[1491,1468]}, + {64308,[1492,1468]}, + {64309,[1493,1468]}, + {64310,[1494,1468]}, + {64312,[1496,1468]}, + {64313,[1497,1468]}, + {64314,[1498,1468]}, + {64315,[1499,1468]}, + {64316,[1500,1468]}, + {64318,[1502,1468]}, + {64320,[1504,1468]}, + {64321,[1505,1468]}, + {64323,[1507,1468]}, + {64324,[1508,1468]}, + {64326,[1510,1468]}, + {64327,[1511,1468]}, + {64328,[1512,1468]}, + {64329,[1513,1468]}, + {64330,[1514,1468]}, + {64332,[1489,1471]}, + {64333,[1499,1471]}, + {64334,[1508,1471]}, + {64300,[1513,1468,1473]}, + {64298,[1513,1473]}, + {64301,[1513,1468,1474]}, + {64299,[1513,1474]}, + {2392,[2325,2364]}, + {2393,[2326,2364]}, + {2394,[2327,2364]}, + {2395,[2332,2364]}, + {2396,[2337,2364]}, + {2397,[2338,2364]}, + {2345,[2344,2364]}, + {2398,[2347,2364]}, + {2399,[2351,2364]}, + {2353,[2352,2364]}, + {2356,[2355,2364]}, + {2524,[2465,2492]}, + {2525,[2466,2492]}, + {2480,[2476,2492]}, + {2527,[2479,2492]}, + {2507,[2503,2494]}, + {2508,[2503,2519]}, + {2649,[2582,2620]}, + {2650,[2583,2620]}, + {2651,[2588,2620]}, + {2652,[2593,2620]}, + {2654,[2603,2620]}, + {2908,[2849,2876]}, + {2909,[2850,2876]}, + {2911,[2863,2876]}, + {2891,[2887,2878]}, + {2888,[2887,2902]}, + {2892,[2887,2903]}, + {3018,[3014,3006]}, + {3019,[3015,3006]}, + {2964,[2962,3031]}, + {3020,[3014,3031]}, + {3144,[3142,3158]}, + {3274,[3270,3266]}, + {3264,[3263,3285]}, + {3275,[3270,3266,3285]}, + {3271,[3270,3285]}, + {3272,[3270,3286]}, + {3402,[3398,3390]}, + {3403,[3399,3390]}, + {3404,[3398,3415]}, + {3635,[3661,3634]}, + {3763,[3789,3762]}, + {3955,[3954,3953]}, + {3957,[3956,3953]}, + {3959,[4018,3968,3953]}, + {3961,[4019,3968,3953]}, + {3958,[4018,3968]}, + {3960,[4019,3968]}, + {3945,[3904,4021]}, + {4025,[3984,4021]}, + {3907,[3906,4023]}, + {3917,[3916,4023]}, + {3922,[3921,4023]}, + {3927,[3926,4023]}, + {3932,[3931,4023]}, + {3987,[3986,4023]}, + {3997,[3996,4023]}, + {4002,[4001,4023]}, + {4007,[4006,4023]}, + {4012,[4011,4023]}, + {12436,[12358,12441]}, + {12364,[12363,12441]}, + {12366,[12365,12441]}, + {12368,[12367,12441]}, + {12370,[12369,12441]}, + {12372,[12371,12441]}, + {12374,[12373,12441]}, + {12376,[12375,12441]}, + {12378,[12377,12441]}, + {12380,[12379,12441]}, + {12382,[12381,12441]}, + {12384,[12383,12441]}, + {12386,[12385,12441]}, + {12389,[12388,12441]}, + {12391,[12390,12441]}, + {12393,[12392,12441]}, + {12400,[12399,12441]}, + {12403,[12402,12441]}, + {12406,[12405,12441]}, + {12409,[12408,12441]}, + {12412,[12411,12441]}, + {12446,[12445,12441]}, + {12532,[12454,12441]}, + {12460,[12459,12441]}, + {12462,[12461,12441]}, + {12464,[12463,12441]}, + {12466,[12465,12441]}, + {12468,[12467,12441]}, + {12470,[12469,12441]}, + {12472,[12471,12441]}, + {12474,[12473,12441]}, + {12476,[12475,12441]}, + {12478,[12477,12441]}, + {12480,[12479,12441]}, + {12482,[12481,12441]}, + {12485,[12484,12441]}, + {12487,[12486,12441]}, + {12489,[12488,12441]}, + {12496,[12495,12441]}, + {12499,[12498,12441]}, + {12502,[12501,12441]}, + {12505,[12504,12441]}, + {12508,[12507,12441]}, + {12535,[12527,12441]}, + {12536,[12528,12441]}, + {12537,[12529,12441]}, + {12538,[12530,12441]}, + {12542,[12541,12441]}, + {12401,[12399,12442]}, + {12404,[12402,12442]}, + {12407,[12405,12442]}, + {12410,[12408,12442]}, + {12413,[12411,12442]}, + {12497,[12495,12442]}, + {12500,[12498,12442]}, + {12503,[12501,12442]}, + {12506,[12504,12442]}, + {12509,[12507,12442]}}. diff --git a/lib/kernel/test/os_SUITE.erl b/lib/kernel/test/os_SUITE.erl index ace9501d18..eacf3c7584 100644 --- a/lib/kernel/test/os_SUITE.erl +++ b/lib/kernel/test/os_SUITE.erl @@ -204,8 +204,9 @@ evil(Config) when is_list(Config) -> evil_loop(Parent, ?EVIL_LOOPS,N) end) end, lists:seq(1, ?EVIL_PROCS)), - Devil = spawn(fun () -> devil(hd(Ps), hd(lists:reverse(Ps))) end), + Devil = spawn_link(fun () -> devil(hd(Ps), hd(lists:reverse(Ps))) end), lists:foreach(fun (P) -> receive {P, done} -> ok end end, Ps), + unlink(Devil), exit(Devil, kill), test_server:timetrap_cancel(Dog), ok. diff --git a/lib/kernel/vsn.mk b/lib/kernel/vsn.mk index 03fe63e385..e33b90a274 100644 --- a/lib/kernel/vsn.mk +++ b/lib/kernel/vsn.mk @@ -1 +1 @@ -KERNEL_VSN = 2.14.2 +KERNEL_VSN = 2.14.3 diff --git a/lib/mnesia/doc/src/notes.xml b/lib/mnesia/doc/src/notes.xml index 2352f11b93..5a6de05c8b 100644 --- a/lib/mnesia/doc/src/notes.xml +++ b/lib/mnesia/doc/src/notes.xml @@ -38,7 +38,36 @@ thus constitutes one section in this document. The title of each section is the version number of Mnesia.</p> - <section><title>Mnesia 4.4.15</title> + <section><title>Mnesia 4.4.16</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Sometimes a 'log_header' record was added to tables when + invoking mnesia:restore/2 with the option + 'recreate_tables'. Thanks Vance Shipley.</p> + <p> + Own Id: OTP-8960</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Compiler warnings were eliminated.</p> + <p> + Own Id: OTP-8855</p> + </item> + </list> + </section> + +</section> + +<section><title>Mnesia 4.4.15</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/mnesia/src/mnesia.appup.src b/lib/mnesia/src/mnesia.appup.src index 47c9bf9979..22ef5178a7 100644 --- a/lib/mnesia/src/mnesia.appup.src +++ b/lib/mnesia/src/mnesia.appup.src @@ -1,7 +1,13 @@ %% -*- erlang -*- {"%VSN%", [ + {"4.4.15",[ + {update, mnesia_dumper, soft, soft_purge, soft_purge, []} + ]} ], [ + {"4.4.15",[ + {update, mnesia_dumper, soft, soft_purge, soft_purge, []} + ]} ] }. diff --git a/lib/mnesia/src/mnesia_dumper.erl b/lib/mnesia/src/mnesia_dumper.erl index f669d009c6..644133cf5d 100644 --- a/lib/mnesia/src/mnesia_dumper.erl +++ b/lib/mnesia/src/mnesia_dumper.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1996-2009. All Rights Reserved. +%% Copyright Ericsson AB 1996-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -643,7 +643,7 @@ insert_op(Tid, _, {op, create_table, TabDef}, InPlace, InitBy) -> true -> ignore; false -> mnesia_log:open_log(temp, - mnesia_log:dcl_log_header(), + mnesia_log:dcd_log_header(), Dcd, false, false, diff --git a/lib/mnesia/vsn.mk b/lib/mnesia/vsn.mk index bce0f7b739..5b52bc6075 100644 --- a/lib/mnesia/vsn.mk +++ b/lib/mnesia/vsn.mk @@ -1 +1 @@ -MNESIA_VSN = 4.4.15 +MNESIA_VSN = 4.4.16 diff --git a/lib/observer/doc/src/notes.xml b/lib/observer/doc/src/notes.xml index 3d7c4fa269..76c13fb3ff 100644 --- a/lib/observer/doc/src/notes.xml +++ b/lib/observer/doc/src/notes.xml @@ -31,6 +31,23 @@ <p>This document describes the changes made to the Observer application.</p> +<section><title>Observer 0.9.8.4</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The multitrace.erl installation example file is now + installed in the examples directory. (Thanks to Peter + Lemenkov.)</p> + <p> + Own Id: OTP-8857</p> + </item> + </list> + </section> + +</section> + <section><title>Observer 0.9.8.3</title> <section><title>Improvements and New Features</title> diff --git a/lib/observer/vsn.mk b/lib/observer/vsn.mk index 499cce6b97..1b72d30eab 100644 --- a/lib/observer/vsn.mk +++ b/lib/observer/vsn.mk @@ -1 +1 @@ -OBSERVER_VSN = 0.9.8.3 +OBSERVER_VSN = 0.9.8.4 diff --git a/lib/odbc/doc/src/notes.xml b/lib/odbc/doc/src/notes.xml index 09d78c3248..7dece7c584 100644 --- a/lib/odbc/doc/src/notes.xml +++ b/lib/odbc/doc/src/notes.xml @@ -31,7 +31,31 @@ <p>This document describes the changes made to the odbc application. </p> - <section><title>ODBC 2.10.8</title> + <section><title>ODBC 2.10.9</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Ipv6 is now supported on Windows as well as on UNIX for + internal socket communication. (ODBC uses sockets instead + of the "Erlang port pipes" as some ODBC-drivers are known + to mess with stdin/stdout.) </p> + <p> + Loopback address constants are used when connecting the + c-side to the erlang-side over local socket API avoiding + getaddrinfo problems, and the {ip, loopback} option is + added as a listen option on the erlang-side. Also cleaned + up the TIME_STAMP contribution.</p> + <p> + Own Id: OTP-8917</p> + </item> + </list> + </section> + +</section> + +<section><title>ODBC 2.10.8</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/odbc/src/odbc.appup.src b/lib/odbc/src/odbc.appup.src index e95e542ff5..f1a370d925 100644 --- a/lib/odbc/src/odbc.appup.src +++ b/lib/odbc/src/odbc.appup.src @@ -1 +1,8 @@ -{"%VSN%", [],[]} +%% -*- erlang -*- +{"%VSN%", + [ + {"2.10.8", [{restart_application, ssl}]} + ], + [ + {"2.10.8", [{restart_application, ssl}]} + ]}. diff --git a/lib/odbc/vsn.mk b/lib/odbc/vsn.mk index fac3f06d4b..aacf3924db 100644 --- a/lib/odbc/vsn.mk +++ b/lib/odbc/vsn.mk @@ -1 +1 @@ -ODBC_VSN = 2.10.8 +ODBC_VSN = 2.10.9 diff --git a/lib/parsetools/doc/src/notes.xml b/lib/parsetools/doc/src/notes.xml index 544850308e..77b3a1a657 100644 --- a/lib/parsetools/doc/src/notes.xml +++ b/lib/parsetools/doc/src/notes.xml @@ -30,6 +30,21 @@ </header> <p>This document describes the changes made to the Parsetools application.</p> +<section><title>Parsetools 2.0.5</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> The formating of Yecc's error messages has been + improved. (Thanks to Joe Armstrong.) </p> + <p> + Own Id: OTP-8919</p> + </item> + </list> + </section> + +</section> + <section><title>Parsetools 2.0.4</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/parsetools/vsn.mk b/lib/parsetools/vsn.mk index 46915baed6..812bf21f03 100644 --- a/lib/parsetools/vsn.mk +++ b/lib/parsetools/vsn.mk @@ -1 +1 @@ -PARSETOOLS_VSN = 2.0.4 +PARSETOOLS_VSN = 2.0.5 diff --git a/lib/public_key/doc/src/notes.xml b/lib/public_key/doc/src/notes.xml index 6e7381eb18..befbd3e586 100644 --- a/lib/public_key/doc/src/notes.xml +++ b/lib/public_key/doc/src/notes.xml @@ -34,6 +34,21 @@ <file>notes.xml</file> </header> +<section><title>Public_Key 0.10</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + Improved dialyzer specs.</p> + <p> + Own Id: OTP-8964</p> + </item> + </list> + </section> + +</section> + <section><title>Public_Key 0.9</title> <section><title>Improvements and New Features</title> diff --git a/lib/public_key/src/pubkey_cert.erl b/lib/public_key/src/pubkey_cert.erl index c8953c6818..fadb993ed9 100644 --- a/lib/public_key/src/pubkey_cert.erl +++ b/lib/public_key/src/pubkey_cert.erl @@ -164,7 +164,7 @@ validate_signature(OtpCert, DerCert, Key, KeyParams, verify_fun(OtpCert, {bad_cert, invalid_signature}, UserState, VerifyFun) end. %%-------------------------------------------------------------------- --spec validate_names(#'OTPCertificate'{}, list(), list(), +-spec validate_names(#'OTPCertificate'{}, no_constraints | list(), list(), term(), term(), fun())-> term(). %% %% Description: Validate Subject Alternative Name. diff --git a/lib/public_key/src/public_key.appup.src b/lib/public_key/src/public_key.appup.src index 0f9f62d2f6..6b6b76d0a5 100644 --- a/lib/public_key/src/public_key.appup.src +++ b/lib/public_key/src/public_key.appup.src @@ -1,6 +1,12 @@ %% -*- erlang -*- {"%VSN%", [ + {"0.9", + [ + {update, public_key, soft, soft_purge, soft_purge, []}, + {update, pubkey_cert, soft, soft_purge, soft_purge, []} + ] + }, {"0.8", [ {update, 'OTP-PUB-KEY', soft, soft_purge, soft_purge, []}, @@ -12,6 +18,12 @@ } ], [ + {"0.9", + [ + {update, public_key, soft, soft_purge, soft_purge, []}, + {update, pubkey_cert, soft, soft_purge, soft_purge, []} + ] + }, {"0.8", [ {update, 'OTP-PUB-KEY', soft, soft_purge, soft_purge, []}, diff --git a/lib/public_key/src/public_key.erl b/lib/public_key/src/public_key.erl index 095a6ff0e0..30398df9cc 100644 --- a/lib/public_key/src/public_key.erl +++ b/lib/public_key/src/public_key.erl @@ -213,10 +213,13 @@ decrypt_private(CipherText, crypto:mpint(D)], Padding). %%-------------------------------------------------------------------- --spec decrypt_public(CipherText :: binary(), rsa_public_key()) -> +-spec decrypt_public(CipherText :: binary(), rsa_public_key() | rsa_private_key()) -> PlainText :: binary(). --spec decrypt_public(CipherText :: binary(), rsa_public_key(), +-spec decrypt_public(CipherText :: binary(), rsa_public_key() | rsa_private_key(), public_crypt_options()) -> PlainText :: binary(). +%% NOTE: The rsa_private_key() is not part of the documented API it is +%% here for testing purposes, in a real situation this is not a relevant +%% thing to do. %% %% Description: Public key decryption using the public key. %%-------------------------------------------------------------------- @@ -232,10 +235,14 @@ decrypt_public(CipherText,#'RSAPrivateKey'{modulus = N, publicExponent = E}, decrypt_public(CipherText, N,E, Options). %%-------------------------------------------------------------------- --spec encrypt_public(PlainText :: binary(), rsa_public_key()) -> +-spec encrypt_public(PlainText :: binary(), rsa_public_key() | rsa_private_key()) -> CipherText :: binary(). --spec encrypt_public(PlainText :: binary(), rsa_public_key(), +-spec encrypt_public(PlainText :: binary(), rsa_public_key() | rsa_private_key(), public_crypt_options()) -> CipherText :: binary(). + +%% NOTE: The rsa_private_key() is not part of the documented API it is +%% here for testing purposes, in a real situation this is not a relevant +%% thing to do. %% %% Description: Public key encryption using the public key. %%-------------------------------------------------------------------- @@ -280,8 +287,8 @@ encrypt_private(PlainText, #'RSAPrivateKey'{modulus = N, sign(PlainText, DigestType, #'RSAPrivateKey'{modulus = N, publicExponent = E, privateExponent = D}) when is_binary(PlainText), - DigestType == md5; - DigestType == sha -> + (DigestType == md5 orelse + DigestType == sha) -> crypto:rsa_sign(DigestType, sized_binary(PlainText), [crypto:mpint(E), crypto:mpint(N), @@ -571,11 +578,9 @@ validate(DerCert, #path_validation_state{working_issuer_name = Issuer, pubkey_cert:prepare_for_next_cert(OtpCert, ValidationState). -sized_binary(Binary) when is_binary(Binary) -> +sized_binary(Binary) -> Size = size(Binary), - <<?UINT32(Size), Binary/binary>>; -sized_binary(List) -> - sized_binary(list_to_binary(List)). + <<?UINT32(Size), Binary/binary>>. %%-------------------------------------------------------------------- %%% Deprecated functions diff --git a/lib/public_key/test/public_key_SUITE.erl b/lib/public_key/test/public_key_SUITE.erl index 81e01f3a02..88cfbcf2b6 100644 --- a/lib/public_key/test/public_key_SUITE.erl +++ b/lib/public_key/test/public_key_SUITE.erl @@ -249,10 +249,8 @@ sign_verify(Config) when is_list(Config) -> true = public_key:pkix_verify(Cert2, {Y, #'Dss-Parms'{p=P, q=Q, g=G}}), %% RSA sign - Msg0 = lists:duplicate(5, "Foo bar 100"), - Msg = list_to_binary(Msg0), + Msg = list_to_binary(lists:duplicate(5, "Foo bar 100")), - RSASign = public_key:sign(Msg0, sha, PrivateRSA), RSASign = public_key:sign(Msg, sha, PrivateRSA), true = public_key:verify(Msg, sha, RSASign, PublicRSA), false = public_key:verify(<<1:8, Msg/binary>>, sha, RSASign, PublicRSA), diff --git a/lib/public_key/vsn.mk b/lib/public_key/vsn.mk index 2810942171..334b9d792e 100644 --- a/lib/public_key/vsn.mk +++ b/lib/public_key/vsn.mk @@ -1 +1 @@ -PUBLIC_KEY_VSN = 0.9 +PUBLIC_KEY_VSN = 0.10 diff --git a/lib/snmp/doc/src/notes.xml b/lib/snmp/doc/src/notes.xml index e9c7bef68f..f84991aa79 100644 --- a/lib/snmp/doc/src/notes.xml +++ b/lib/snmp/doc/src/notes.xml @@ -1,4 +1,4 @@ -<?xml version="1.0" encoding="latin1" ?> +<?xml version="1.0" encoding="iso-8859-1" ?> <!DOCTYPE chapter SYSTEM "chapter.dtd"> <chapter> @@ -35,10 +35,13 @@ <section> <title>SNMP Development Toolkit 4.19</title> <p>Version 4.19 supports code replacement in runtime from/to - version 4.18, 4.17.1, 4.17.</p> + version 4.18, 4.17.1 and 4.17.</p> <section> <title>Improvements and new features</title> +<!-- + <p>-</p> +--> <list type="bulleted"> <item> <p>[compiler] Added support for the textual convention @@ -59,17 +62,42 @@ options. </p> <p>Own Id: OTP-8966</p> </item> + + <item> + <p>[snmpc] Added a MIB compiler escript, + <seealso marker="snmpc">snmpc</seealso>. </p> + <p>Own Id: OTP-9004</p> + </item> </list> </section> <section> - <title>Fixed Bugs and Malfunctions</title> + <title>Reported Fixed Bugs and Malfunctions</title> <p>-</p> +<!-- + <list type="bulleted"> + <item> + <p>When the function FilterMod:accept_recv/2 + returned false the SNMP agent stopped collecting + messages from UDP.</p> + <p>Own Id: OTP-8761</p> + </item> + </list> +--> </section> + <section> + <title>Incompatibilities</title> + <p>-</p> + </section> </section> <!-- 4.19 --> + <section><title>SNMP 4.18</title> + <section> + <title>Improvements and new features</title> + <p>-</p> + </section> <section><title>Fixed Bugs and Malfunctions</title> <list> @@ -79,6 +107,10 @@ </list> </section> + <section> + <title>Incompatibilities</title> + <p>-</p> + </section> </section> <!-- 4.18 --> <section> @@ -96,7 +128,8 @@ <list type="bulleted"> <item> <p>When the function FilterMod:accept_recv/2 - returned false the SNMP agent stopped collecting messages from UDP.</p> + returned false the SNMP agent stopped collecting + messages from UDP.</p> <p>Own Id: OTP-8761</p> </item> </list> diff --git a/lib/snmp/src/compile/Makefile b/lib/snmp/src/compile/Makefile index 4be60e1835..1f1086eae1 100644 --- a/lib/snmp/src/compile/Makefile +++ b/lib/snmp/src/compile/Makefile @@ -20,6 +20,7 @@ include $(ERL_TOP)/make/target.mk EBIN = ../../ebin +BIN = ../../bin include $(ERL_TOP)/make/$(TARGET)/otp.mk @@ -44,9 +45,11 @@ RELSYSDIR = $(RELEASE_PATH)/lib/snmp-$(VSN) include modules.mk +ESCRIPT_BIN = $(ESCRIPT_SRC:%.src=$(BIN)/%) + ERL_FILES = $(MODULES:%=%.erl) -TARGET_FILES = $(MODULES:%=$(EBIN)/%.$(EMULATOR)) +TARGET_FILES = $(MODULES:%=$(EBIN)/%.$(EMULATOR)) $(ESCRIPT_BIN) GENERATED_PARSER = $(PARSER_MODULE:%=%.erl) @@ -97,8 +100,12 @@ info: @echo "" @echo "EBIN: $(EBIN)" @echo "" + @echo "ESCRIPT_SRC: $(ESCRIPT_SRC)" + @echo "ESCRIPT_BIN: $(ESCRIPT_BIN)" + @echo "" @echo "" + # ---------------------------------------------------- # Special Build Targets # ---------------------------------------------------- @@ -107,6 +114,7 @@ parser: $(PARSER_TARGET) $(GENERATED_PARSER): $(PARSER_SRC) + # ---------------------------------------------------- # Release Target # ---------------------------------------------------- @@ -115,9 +123,11 @@ include $(ERL_TOP)/make/otp_release_targets.mk release_spec: opt $(INSTALL_DIR) $(RELSYSDIR)/src $(INSTALL_DIR) $(RELSYSDIR)/src/compiler - $(INSTALL_DATA) $(PARSER_SRC) $(ERL_FILES) $(INTERNAL_HRL_FILES) $(RELSYSDIR)/src/compiler + $(INSTALL_DATA) $(ESCRIPT_SRC) $(PARSER_SRC) $(ERL_FILES) $(INTERNAL_HRL_FILES) $(RELSYSDIR)/src/compiler $(INSTALL_DIR) $(RELSYSDIR)/ebin $(INSTALL_DATA) $(TARGET_FILES) $(RELSYSDIR)/ebin + $(INSTALL_DIR) $(RELSYSDIR)/bin + $(INSTALL_SCRIPT) $(ESCRIPT_BIN) $(RELSYSDIR)/bin release_docs_spec: diff --git a/lib/snmp/src/compile/depend.mk b/lib/snmp/src/compile/depend.mk index 75af1bf293..74eb6e0864 100644 --- a/lib/snmp/src/compile/depend.mk +++ b/lib/snmp/src/compile/depend.mk @@ -44,3 +44,6 @@ $(EBIN)/snmpc_mib_gram.$(EMULATOR): \ ../../include/snmp_types.hrl \ snmpc_mib_gram.erl +$(BIN)/snmpc: snmpc.src + $(PERL) -p -e 's?%VSN%?$(VSN)? ' < $< > $@ + chmod 755 $@ diff --git a/lib/snmp/src/compile/modules.mk b/lib/snmp/src/compile/modules.mk index 6365b0e694..ca78e2e6a9 100644 --- a/lib/snmp/src/compile/modules.mk +++ b/lib/snmp/src/compile/modules.mk @@ -21,6 +21,9 @@ PARSER_SRC = snmpc_mib_gram.yrl PARSER_MODULE = $(PARSER_SRC:%.yrl=%) +ESCRIPT_SRC = \ + snmpc.src + MODULES = \ $(PARSER_MODULE) \ snmpc \ diff --git a/lib/snmp/src/compile/snmpc.src b/lib/snmp/src/compile/snmpc.src new file mode 100644 index 0000000000..5ad88eacc8 --- /dev/null +++ b/lib/snmp/src/compile/snmpc.src @@ -0,0 +1,129 @@ +#!/usr/bin/env escript +%% -*- erlang -*- +%% + +-record(state, + { + version = %VSN%, + mib_file, + outdir = "./", + db = volatile, + include_dirs = ["./"], + include_lib_dirs = [], + deprecated = true, + group_check = true, + description = false, + reference = false, + imports = false, + module_identity = false, + module, + no_defs = false, + relaxed_row_name_assigne_check, + verbosity + }). + +%% --o Dir [defaults to "./"] +%% --i Dir [defaults to "./"] +%% --il Dir +%% --gc +%% --db DB [defaults to volatile] +%% --dep +%% --desc +%% --ref +%% --imp +%% --mi +%% --mod Mod +%% --nd +%% --rrnac +%% --version +%% --verbosity V +main(Args) when is_list(Args) -> + case (catch process_args(Args)) of + ok -> + usage(); + {ok, State} when is_record(State, state) -> + io:format("snmpc: ~p~n", [State]); + {ok, Str} when is_list(Str) -> + io:format("~s~n~n", [Str]), + halt(1); + {error, ReasonStr} -> + usage(ReasonStr) + end; +main(_) -> + usage(). + +process_args([]) -> + {error, lists:flatten(io_lib:format("No MIB-file", []))}; +process_args(Args) -> + %% CWD = "./", + process_args(Args, #state{}). + +process_args([], State) -> + {ok, State}; +process_args(["--help"|_Args], _State) -> + ok; +process_args(["--version"|_Args], State) -> + {ok, lists:flatten(io_lib:format("snmpc ~s", [State#state.version]))}; +process_args(["--verbosity", Verbosity0|Args], #state{verbosity = V} = State) + when (V =:= undefined) -> + Verbosity = list_to_atom(Verbosity0), + case lists:member(Verbosity, [trace,debug,log,info,silence]) of + true -> + process_args(Args, State#state{verbosity = Verbosity}); + false -> + e(lists:flatten(io_lib:format("Unknown verbosity: ~s", [Verbosity0]))) + end; +process_args(["--verbosity"|_Args], #state{verbosity = V}) + when (V =/= undefined) -> + e(lists:flatten(io_lib:format("Verbosity already set to ~w", [V]))); +process_args([Arg|Args], State) -> + io:format("Arg: ~p~n", [Arg]), + process_args(Args, State). + +usage(ReasonStr) -> + io:format("ERROR: ~s~n", [ReasonStr]), + usage(). + +usage() -> + io:format("Usage: snmpc [options] MIB.mib" + "~nOptions:" + "~n --help - Prints this info." + "~n --version - Prints compiler version." + "~n --verbosity <verbosity> - Print debug info." + "~n verbosity = trace | debug | log | info | silence" + "~n Defaults to silence." + "~n --warnings - Print warning messages." + "~n --i <include dir> - Add this dir to the list of dirs that will be" + "~n searched for imported (compiled) MIB files." + "~n The current workin dir will always be included. " + "~n --il <include_dir dir> - Add this dir to the list of dirs that will be" + "~n searched for imported (compiled) MIB files." + "~n It assumes that the first element in the dir name" + "~n correspond to an OTP application. For example snmp/mibs/" + "~n The current workin dir and the <snmp-home>/priv/mibs " + "~n are always listed last the includ path. " + "~n --db <DB> - Database to used for the default instrumentation." + "~n Defaults to volatile." + "~n --deprecated - Keep deprecated definition(s)." + "~n If not specified the compiler will ignore" + "~n deprecated definitions." + "~n --description - The DESCRIPTION field will be included." + "~n --reference - The REFERENCE field will be included." + "~n --imports - The IMPORTS field will be included." + "~n --module_id - The MODULE-IDENTITY field will be included." + "~n --module <module> - The module which implements all the instrumentation" + "~n functions. " + "~n The name of the of all instrumentation functions" + "~n must be the same as the corresponding managed object" + "~n it implements." + "~n --no_defs - The default instrumentation functions will *not* be used" + "~n if a managed object have no instrumentation function. " + "~n Instead this will be reported as an error, and the " + "~n compilation aborts. " + "~n " + "~n", []), + halt(1). + + +e(Reason) -> + throw({error, Reason}). diff --git a/lib/snmp/vsn.mk b/lib/snmp/vsn.mk index 12a9d8ed3c..b55c8718ec 100644 --- a/lib/snmp/vsn.mk +++ b/lib/snmp/vsn.mk @@ -2,7 +2,7 @@ # %CopyrightBegin% # -# Copyright Ericsson AB 1997-2010. All Rights Reserved. +# Copyright Ericsson AB 1997-2011. All Rights Reserved. # # The contents of this file are subject to the Erlang Public License, # Version 1.1, (the "License"); you may not use this file except in diff --git a/lib/ssl/doc/src/notes.xml b/lib/ssl/doc/src/notes.xml index 756c0d1b1f..313f1e59c9 100644 --- a/lib/ssl/doc/src/notes.xml +++ b/lib/ssl/doc/src/notes.xml @@ -31,7 +31,48 @@ <p>This document describes the changes made to the SSL application. </p> - <section><title>SSL 4.1</title> + <section><title>SSL 4.1.1</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> + Correct handling of client certificate verify message + When checking the client certificate verify message the + server used the wrong algorithm identifier to determine + the signing algorithm, causing a function clause error in + the public_key application when the key-exchange + algorithm and the public key algorithm of the client + certificate happen to differ.</p> + <p> + Own Id: OTP-8897</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + For testing purposes ssl now also support some anonymous + cipher suites when explicitly configured to do so.</p> + <p> + Own Id: OTP-8870</p> + </item> + <item> + <p> + Sends an error alert instead of crashing if a crypto + function for the selected cipher suite fails.</p> + <p> + Own Id: OTP-8930 Aux Id: seq11720 </p> + </item> + </list> + </section> + +</section> + +<section><title>SSL 4.1</title> <section><title>Improvements and New Features</title> <list> diff --git a/lib/ssl/doc/src/ssl.xml b/lib/ssl/doc/src/ssl.xml index 511f1e0bb2..ec272379bb 100644 --- a/lib/ssl/doc/src/ssl.xml +++ b/lib/ssl/doc/src/ssl.xml @@ -218,12 +218,12 @@ fun(OtpCert :: #'OTPCertificate'{}, Event :: {bad_cert, Reason :: atom()} | application is encountered. Additionally it will be called when a certificate is considered valid by the path validation to allow access to each certificate in the path to the user - application. Note that the it will differentiate between - the peer certificate and CA certificates by using valid_peer - or valid as the second argument to the verify fun. - See - <seealso marker="public_key:application">public_key(3)</seealso> - for definition of #'OTPCertificate'{} and #'Extension'{}.</p> + application. Note that the it will differentiate between the + peer certificate and CA certificates by using valid_peer or + valid as the second argument to the verify fun. See <seealso + marker="public_key:cert_records">the public_key User's + Guide</seealso> for definition of #'OTPCertificate'{} and + #'Extension'{}.</p> <p>If the verify callback fun returns {fail, Reason}, the verification process is immediately stopped and an alert is diff --git a/lib/ssl/src/ssl.appup.src b/lib/ssl/src/ssl.appup.src index f4e6b59b6d..51c5289bd2 100644 --- a/lib/ssl/src/ssl.appup.src +++ b/lib/ssl/src/ssl.appup.src @@ -1,9 +1,11 @@ %% -*- erlang -*- {"%VSN%", [ + {"4.1", [{restart_application, ssl}]}, {"4.0.1", [{restart_application, ssl}]} ], [ + {"4.1", [{restart_application, ssl}]}, {"4.0.1", [{restart_application, ssl}]} ]}. diff --git a/lib/ssl/src/ssl_certificate.erl b/lib/ssl/src/ssl_certificate.erl index 5571fb01f6..8c0c2bfa5d 100644 --- a/lib/ssl/src/ssl_certificate.erl +++ b/lib/ssl/src/ssl_certificate.erl @@ -28,7 +28,6 @@ -include("ssl_handshake.hrl"). -include("ssl_alert.hrl"). -include("ssl_internal.hrl"). --include("ssl_debug.hrl"). -include_lib("public_key/include/public_key.hrl"). -export([trusted_cert_and_path/2, diff --git a/lib/ssl/src/ssl_cipher.erl b/lib/ssl/src/ssl_cipher.erl index 1c8bbbaf06..72f02a4362 100644 --- a/lib/ssl/src/ssl_cipher.erl +++ b/lib/ssl/src/ssl_cipher.erl @@ -29,7 +29,6 @@ -include("ssl_record.hrl"). -include("ssl_cipher.hrl"). -include("ssl_alert.hrl"). --include("ssl_debug.hrl"). -include_lib("public_key/include/public_key.hrl"). -export([security_parameters/2, suite_definition/1, diff --git a/lib/ssl/src/ssl_connection.erl b/lib/ssl/src/ssl_connection.erl index ce90d22c09..6c9ac65b64 100644 --- a/lib/ssl/src/ssl_connection.erl +++ b/lib/ssl/src/ssl_connection.erl @@ -29,7 +29,6 @@ -behaviour(gen_fsm). --include("ssl_debug.hrl"). -include("ssl_handshake.hrl"). -include("ssl_alert.hrl"). -include("ssl_record.hrl"). diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index 7082a23fd0..c7a1c4965d 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -28,7 +28,6 @@ -include("ssl_cipher.hrl"). -include("ssl_alert.hrl"). -include("ssl_internal.hrl"). --include("ssl_debug.hrl"). -include_lib("public_key/include/public_key.hrl"). -export([master_secret/4, client_hello/5, server_hello/4, hello/4, diff --git a/lib/ssl/src/ssl_handshake.hrl b/lib/ssl/src/ssl_handshake.hrl index 74fba3786c..68a7802ef2 100644 --- a/lib/ssl/src/ssl_handshake.hrl +++ b/lib/ssl/src/ssl_handshake.hrl @@ -26,6 +26,13 @@ -ifndef(ssl_handshake). -define(ssl_handshake, true). +-include_lib("public_key/include/public_key.hrl"). + +-type algo_oid() :: ?'rsaEncryption' | ?'id-dsa'. +-type public_key() :: #'RSAPublicKey'{} | integer(). +-type public_key_params() :: #'Dss-Parms'{} | term(). +-type public_key_info() :: {algo_oid(), public_key(), public_key_params()}. + -record(session, { session_id, peer_certificate, diff --git a/lib/ssl/src/ssl_internal.hrl b/lib/ssl/src/ssl_internal.hrl index 4148032cb7..43a85c2d9d 100644 --- a/lib/ssl/src/ssl_internal.hrl +++ b/lib/ssl/src/ssl_internal.hrl @@ -25,6 +25,24 @@ -include_lib("public_key/include/public_key.hrl"). +-type reason() :: term(). +-type reply() :: term(). +-type msg() :: term(). +-type from() :: term(). +-type host() :: string() | tuple(). +-type port_num() :: integer(). +-type session_id() :: 0 | binary(). +-type tls_version() :: {integer(), integer()}. +-type tls_atom_version() :: sslv3 | tlsv1. +-type cache_ref() :: term(). +-type certdb_ref() :: term(). +-type key_algo() :: null | rsa | dhe_rsa | dhe_dss | dh_anon. +-type der_cert() :: binary(). +-type private_key() :: #'RSAPrivateKey'{} | #'DSAPrivateKey'{}. +-type issuer() :: tuple(). +-type serialnumber() :: integer(). +-type cert_key() :: {reference(), integer(), issuer()}. + %% basic binary constructors -define(BOOLEAN(X), X:8/unsigned-big-integer). -define(BYTE(X), X:8/unsigned-big-integer). @@ -93,28 +111,6 @@ active = true }). --type reason() :: term(). --type reply() :: term(). --type msg() :: term(). --type from() :: term(). --type host() :: string() | tuple(). --type port_num() :: integer(). --type session_id() :: 0 | binary(). --type tls_version() :: {integer(), integer()}. --type tls_atom_version() :: sslv3 | tlsv1. --type cache_ref() :: term(). --type certdb_ref() :: term(). --type key_algo() :: null | rsa | dhe_rsa | dhe_dss | dh_anon. --type oid() :: tuple(). --type public_key() :: #'RSAPublicKey'{} | integer(). --type public_key_params() :: #'Dss-Parms'{} | term(). --type public_key_info() :: {oid(), public_key(), public_key_params()}. --type der_cert() :: binary(). --type private_key() :: #'RSAPrivateKey'{} | #'DSAPrivateKey'{}. --type issuer() :: tuple(). --type serialnumber() :: integer(). --type cert_key() :: {reference(), integer(), issuer()}. - -endif. % -ifdef(ssl_internal). diff --git a/lib/ssl/src/ssl_record.erl b/lib/ssl/src/ssl_record.erl index e7a2d8ecf1..f1c0073965 100644 --- a/lib/ssl/src/ssl_record.erl +++ b/lib/ssl/src/ssl_record.erl @@ -30,7 +30,6 @@ -include("ssl_alert.hrl"). -include("ssl_handshake.hrl"). -include("ssl_cipher.hrl"). --include("ssl_debug.hrl"). %% Connection state handling -export([init_connection_states/1, diff --git a/lib/ssl/src/ssl_ssl3.erl b/lib/ssl/src/ssl_ssl3.erl index 18c3f4ed3d..c49f9f1e6d 100644 --- a/lib/ssl/src/ssl_ssl3.erl +++ b/lib/ssl/src/ssl_ssl3.erl @@ -25,7 +25,6 @@ -module(ssl_ssl3). -include("ssl_cipher.hrl"). --include("ssl_debug.hrl"). -include("ssl_internal.hrl"). -include("ssl_record.hrl"). % MD5 and SHA diff --git a/lib/ssl/src/ssl_tls1.erl b/lib/ssl/src/ssl_tls1.erl index 88394b23ba..3784483e9c 100644 --- a/lib/ssl/src/ssl_tls1.erl +++ b/lib/ssl/src/ssl_tls1.erl @@ -27,7 +27,6 @@ -include("ssl_cipher.hrl"). -include("ssl_internal.hrl"). -include("ssl_record.hrl"). --include("ssl_debug.hrl"). -export([master_secret/3, finished/3, certificate_verify/2, mac_hash/7, setup_keys/6, suites/0]). diff --git a/lib/ssl/test/ssl_basic_SUITE.erl b/lib/ssl/test/ssl_basic_SUITE.erl index ea84b3c9d1..8f9554f3ce 100644 --- a/lib/ssl/test/ssl_basic_SUITE.erl +++ b/lib/ssl/test/ssl_basic_SUITE.erl @@ -31,6 +31,7 @@ -define('24H_in_sec', 86400). -define(TIMEOUT, 60000). +-define(LONG_TIMEOUT, 600000). -define(EXPIRE, 10). -define(SLEEP, 500). @@ -45,7 +46,7 @@ %% variable, but should NOT alter/remove any existing entries. %%-------------------------------------------------------------------- init_per_suite(Config0) -> - Dog = ssl_test_lib:timetrap(?TIMEOUT *2), + Dog = ssl_test_lib:timetrap(?LONG_TIMEOUT *2), crypto:start(), application:start(public_key), ssl:start(), diff --git a/lib/ssl/test/ssl_session_cache_SUITE.erl b/lib/ssl/test/ssl_session_cache_SUITE.erl index 7a441e4599..0f39759d97 100644 --- a/lib/ssl/test/ssl_session_cache_SUITE.erl +++ b/lib/ssl/test/ssl_session_cache_SUITE.erl @@ -28,6 +28,7 @@ -define(SLEEP, 500). -define(TIMEOUT, 60000). +-define(LONG_TIMEOUT, 600000). -behaviour(ssl_session_cache_api). %% For the session cache tests @@ -45,7 +46,7 @@ %% variable, but should NOT alter/remove any existing entries. %%-------------------------------------------------------------------- init_per_suite(Config0) -> - Dog = ssl_test_lib:timetrap(?TIMEOUT *2), + Dog = ssl_test_lib:timetrap(?LONG_TIMEOUT *2), crypto:start(), application:start(public_key), ssl:start(), @@ -119,7 +120,7 @@ end_per_testcase(session_cache_process_list, Config) -> end_per_testcase(session_cache_process_mnesia, Config) -> application:unset_env(ssl, session_cb), application:unset_env(ssl, session_cb_init_args), - mnesia:stop(), + mnesia:kill(), ssl:stop(), ssl:start(), end_per_testcase(default_action, Config); diff --git a/lib/ssl/test/ssl_to_openssl_SUITE.erl b/lib/ssl/test/ssl_to_openssl_SUITE.erl index 55a0100b1e..afedeaf099 100644 --- a/lib/ssl/test/ssl_to_openssl_SUITE.erl +++ b/lib/ssl/test/ssl_to_openssl_SUITE.erl @@ -27,6 +27,7 @@ -include("test_server.hrl"). -define(TIMEOUT, 120000). +-define(LONG_TIMEOUT, 600000). -define(SLEEP, 1000). -define(OPENSSL_RENEGOTIATE, "r\n"). -define(OPENSSL_QUIT, "Q\n"). @@ -44,7 +45,7 @@ %% variable, but should NOT alter/remove any existing entries. %%-------------------------------------------------------------------- init_per_suite(Config0) -> - Dog = ssl_test_lib:timetrap(?TIMEOUT *2), + Dog = ssl_test_lib:timetrap(?LONG_TIMEOUT *2), case os:find_executable("openssl") of false -> {skip, "Openssl not found"}; diff --git a/lib/ssl/vsn.mk b/lib/ssl/vsn.mk index 30a0a3b3f7..ee692adb3b 100644 --- a/lib/ssl/vsn.mk +++ b/lib/ssl/vsn.mk @@ -1,2 +1,2 @@ -SSL_VSN = 4.1 +SSL_VSN = 4.1.1 diff --git a/lib/stdlib/doc/src/filelib.xml b/lib/stdlib/doc/src/filelib.xml index 4ff3b22f32..47d64f245c 100644 --- a/lib/stdlib/doc/src/filelib.xml +++ b/lib/stdlib/doc/src/filelib.xml @@ -36,14 +36,23 @@ <description> <p>This module contains utilities on a higher level than the <c>file</c> module.</p> + <p>The module supports Unicode file names, so that it will match against regular expressions given in Unicode and that it will find and process raw file names (i.e. files named in a way that does not confirm to the expected encoding).</p> + <p>If the VM operates in Unicode file naming mode on a machine with transparent file naming, the <c>fun()</c> provided to <c>fold_files/5</c> needs to be prepared to handle binary file names.</p> + <p>For more information about raw file names, see the <seealso marker="kernel:file">file</seealso> module.</p> </description> <section> <title>DATA TYPES</title> <code type="none"> -filename() = string() | atom() | DeepList -dirname() = filename() -DeepList = [char() | atom() | DeepList]</code> +filename() = = string() | atom() | DeepList | RawFilename + DeepList = [char() | atom() | DeepList] + RawFilename = binary() + If VM is in unicode filename mode, string() and char() are allowed to be > 255. + RawFilename is a filename not subject to Unicode translation, meaning that it + can contain characters not conforming to the Unicode encoding expected from the + filesystem (i.e. non-UTF-8 characters although the VM is started in Unicode + filename mode). +dirname() = filename()</code> </section> <funcs> @@ -90,6 +99,18 @@ DeepList = [char() | atom() | DeepList]</code> If <c>Recursive</c> is true all sub-directories to <c>Dir</c> are processed. The regular expression matching is done on just the filename without the directory part.</p> + + <p>If Unicode file name translation is in effect and the file + system is completely transparent, file names that cannot be + interpreted as Unicode may be encountered, in which case the + <c>fun()</c> must be prepared to handle raw file names + (i.e. binaries). If the regular expression contains + codepoints beyond 255, it will not match file names that do + not conform to the expected character encoding (i.e. are not + encoded in valid UTF-8).</p> + + <p>For more information about raw file names, see the + <seealso marker="kernel:file">file</seealso> module.</p> </desc> </func> <func> diff --git a/lib/stdlib/doc/src/filename.xml b/lib/stdlib/doc/src/filename.xml index fe6c6f898e..cdee6e4a81 100644 --- a/lib/stdlib/doc/src/filename.xml +++ b/lib/stdlib/doc/src/filename.xml @@ -4,7 +4,7 @@ <erlref> <header> <copyright> - <year>1997</year><year>2009</year> + <year>1997</year><year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -43,13 +43,22 @@ only, even if the arguments contain back slashes. Use <c>join/1</c> to normalize a file name by removing redundant directory separators.</p> + <p>The module supports raw file names in the way that if a binary is present, or the file name cannot be interpreted according to the return value of + <seealso marker="kernel:file#native_name_encoding/0">file:native_name_encoding/0</seealso>, a raw file name will also be returned. For example filename:join/1 provided with a path component being a binary (and also not being possible to interpret under the current native file name encoding) will result in a raw file name being returned (the join operation will have been performed of course). For more information about raw file names, see the <seealso marker="kernel:file">file</seealso> module.</p> </description> <section> <title>DATA TYPES</title> <code type="none"> -name() = string() | atom() | DeepList -DeepList = [char() | atom() | DeepList]</code> +name() = string() | atom() | DeepList | RawFilename + DeepList = [char() | atom() | DeepList] + RawFilename = binary() + If VM is in unicode filename mode, string() and char() are allowed to be > 255. + RawFilename is a filename not subject to Unicode translation, meaning that it + can contain characters not conforming to the Unicode encoding expected from the + filesystem (i.e. non-UTF-8 characters although the VM is started in Unicode + filename mode). + </code> </section> <funcs> <func> diff --git a/lib/stdlib/doc/src/notes.xml b/lib/stdlib/doc/src/notes.xml index 6c618bc798..940044d0a8 100644 --- a/lib/stdlib/doc/src/notes.xml +++ b/lib/stdlib/doc/src/notes.xml @@ -30,6 +30,126 @@ </header> <p>This document describes the changes made to the STDLIB application.</p> +<section><title>STDLIB 1.17.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> When several clients accessed a Dets table + simultaneously, one of them calling + <c>dets:insert_new/2</c>, the Dets server could crash. + Alternatively, under the same conditions, <c>ok</c> was + sometimes returned instead of <c>true</c>. (Thanks to + John Hughes.) </p> + <p> + Own Id: OTP-8856</p> + </item> + <item> + <p> When several clients accessed a Dets table + simultaneously, inserted or updated objects were + sometimes lost due to the Dets file being truncated. + (Thanks to John Hughes.) </p> + <p> + Own Id: OTP-8898</p> + </item> + <item> + <p> When several clients accessed a Dets table + simultaneously, modifications of the Dets server's + internal state were sometimes thrown away. The symptoms + are diverse: error with reason <c>bad_object</c>; + inserted objects not returned by <c>lookup()</c>; et + cetera. (Thanks to John Hughes.) </p> + <p> + Own Id: OTP-8899</p> + </item> + <item> + <p> If a Dets table was closed after calling + <c>bchunk/2</c>, <c>match/1,3</c>, + <c>match_object/1,3</c>, or <c>select/1,3</c> and then + opened again, a subsequent call using the returned + continuation would normally return a reply. This bug has + fixed; now the call fails with reason <c>badarg</c>. </p> + <p> + Own Id: OTP-8903</p> + </item> + <item> + <p> Cover did not collect coverage data for files such as + Yecc parses containing include directives. The bug has + been fixed by modifying <c>epp</c>, the Erlang Code + Preprocessor. </p> + <p> + Own Id: OTP-8911</p> + </item> + <item> + <p> If a Dets table with fewer slots than keys was opened + and then closed after just a lookup, the contents were no + longer well-formed. This bug has been fixed. (Thanks to + Matthew Evans.) </p> + <p> + Own Id: OTP-8923</p> + </item> + <item> + <p> + In a supervisor, when it terminates a child, if that + child happens to have exited fractionally early, with + normal, the supervisor reports this as an error. This + should not be reported as an error.</p> + <p> + *** POTENTIAL INCOMPATIBILITY ***</p> + <p> + Own Id: OTP-8938 Aux Id: seq11615 </p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> + The documentation filelib:wildcard/1,2 now describes the + character set syntax for wildcards.</p> + <p> + Own Id: OTP-8879 Aux Id: seq11683 </p> + </item> + <item> + <p>Buffer overflows have been prevented in <c>erlc</c>, + <c>dialyzer</c>, <c>typer</c>, <c>run_test</c>, + <c>heart</c>, <c>escript</c>, and <c>erlexec</c>.</p> + (Thanks to Michael Santos.) + <p> + Own Id: OTP-8892</p> + </item> + <item> + <p> + Using a float for the number of copies for + <c>string:copies/2</c> resulted in an infinite loop. Now + it will fail with an exception instead. (Thanks to + Michael Santos.)</p> + <p> + Own Id: OTP-8915</p> + </item> + <item> + <p> + New ETS option <c>compressed</c>, to enable a more + compact storage format at the expence of heavier table + operations. For test and evaluation, <c>erl +ec</c> can + be used to force compression on all ETS tables.</p> + <p> + Own Id: OTP-8922 Aux Id: seq11658 </p> + </item> + <item> + <p> The default maximum number of slots of a Dets table + has been changed as to be equal to the maximum number of + slots. (Thanks to Richard Carlsson.) </p> + <p> + Own Id: OTP-8959</p> + </item> + </list> + </section> + +</section> + <section><title>STDLIB 1.17.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/stdlib/doc/src/unicode_usage.xml b/lib/stdlib/doc/src/unicode_usage.xml index f1b0659ea2..416df1f02c 100644 --- a/lib/stdlib/doc/src/unicode_usage.xml +++ b/lib/stdlib/doc/src/unicode_usage.xml @@ -5,7 +5,7 @@ <header> <copyright> <year>1999</year> - <year>2009</year> + <year>2010</year> <holder>Ericsson AB. All Rights Reserved.</holder> </copyright> <legalnotice> @@ -168,6 +168,48 @@ Eshell V5.7 (abort with ^G) <image file="ushell2.gif"><icaption>Unicode characters in allowed and disallowed context</icaption></image> </section> <section> +<title>Unicode file names</title> +<p>Most modern operating systems support Unicode file names in some way or another. There are several different ways to do this and Erlang by default treats the different approaches differently:</p> +<taglist> +<tag>Mandatory Unicode file naming</tag> +<item> +<p>Windows and, for most common uses, MacOSX enforces Unicode support for file names. All files created in the filesystem have names that can consistently be interpreted. In MacOSX, all file names are retrieved in UTF-8 encoding, while Windows has selected an approach where each system call handling file names has a special Unicode aware variant, giving much the same effect. There are no file names on these systems that are not Unicode file names, why the default behavior of the Erlang VM is to work in "Unicode file name translation mode", meaning that a file name can be given as a Unicode list and that will be automatically translated to the proper name encoding for the underlying operating and file system.</p> +<p>Doing i.e. a <c>file:list_dir/1</c> on one of these systems may return Unicode lists with codepoints beyond 255, depending on the content of the actual filesystem.</p> +<p>As the feature is fairly new, you may still stumble upon non core applications that cannot handle being provided with file names containing characters with codepoints larger than 255, but the core Erlang system should have no problems with Unicode file names.</p> +</item> +<tag>Transparent file naming</tag> +<item> +<p>Most Unix operating systems have adopted a simpler approach, namely that Unicode file naming is not enforced, but by convention. Those systems usually use UTF-8 encoding for Unicode file names, but do not enforce it. On such a system, a file name containing characters having codepoints between 128 and 255 may be named either as plain ISO-latin-1 or using UTF-8 encoding. As no consistency is enforced, the Erlang VM can do no consistent translation of all file names. If the VM would automatically select encoding based on heuristics, one could get unexpected behavior on these systems, therefore file names not being encoded in UTF-8 are returned as "raw file names" if Unicode file naming support is turned on.</p> +<p>A raw file name is not a list, but a binary. Many non core applications still do not handle file names given as binaries, why such raw names are avoided by default. This means that systems having implemented Unicode file naming through transparent file systems and an UTF-8 convention, do not by default have Unicode file naming turned on. Explicitly turning Unicode file name handling on for these types of systems is considered experimental.</p> +</item> +</taglist> +<p>The Unicode file naming support was introduced with OTP release R14B01. A VM operating in Unicode file mode can work with files having names in any language or character set (as long as it's supported by the underlying OS and file system). The Unicode character list is used to denote file or directory names and if the file system content is listed, you will also be able to get Unicode lists as return value. The support lies in the kernel and stdlib modules, why most applications (that does not explicitly require the file names to be in the ISO-latin-1 range) will benefit from the Unicode support without change.</p> + +<p>On Operating systems with mandatory Unicode file names, this means that you more easily conform to the file names of other (non Erlang) applications, and you can also process file names that, at least on Windows, were completely inaccessible (due to having names that could not be represented in ISO-latin-1). Also you will avoid creating incomprehensible file names on MacOSX as the vfs layer of the OS will accept all your file names as UTF-8 and will not rewrite them.</p> + +<p>For most systems, turning on Unicode file name translation is no problem even if it uses transparent file naming. Very few systems have mixed file name encodings. A consistent UTF-8 named system will work perfectly in Unicode file name mode. It is still however considered experimental in R14B01. Unicode file name translation is turned on with the <c>+fnu</c> switch to the <c>erl</c> program. If the VM is started in Unicode file name translation mode, <c>file:native_name_encoding/0</c> will return the atom <c>utf8</c>.</p> + +<p>In Unicode file name mode, file names given to the BIF <c>open_port/2</c> with the option <c>{spawn_executable,...}</c> are also interpreted as Unicode. So is the parameter list given in the <c>args</c> option available when using <c>spawn_executable</c>. The UTF-8 translation of arguments can be avoided using binaries, see the discussion about raw file names below.</p> + +<p>It is worth noting that the file <c>encoding</c> options given when opening a file has nothing to do with the file <em>name</em> encoding convention. You can very well open files containing UTF-8 but having file names in ISO-latin-1 or vice versa.</p> + +<note>Erlang drivers and NIF shared objects still can not be named with names containing codepoints beyond 127. This is a known limitation to be removed in a future release. Erlang modules however can, but it is definitely not a good idea and is still considered experimental.</note> + +<section> +<title>Notes about raw file names and automatic file name conversion</title> +<p>Raw file names is introduced together with Unicode file name support in erts-5.8.2 (OTP R14B01). The reason "raw file names" is introduced in the system is to be able to consistently represent file names given in different encodings on the same system. Having the VM automatically translate a file name that is not in UTF-8 to a list of Unicode characters might seem practical, but this would open up for both duplicate file names and other inconsistent behavior. Consider a directory containing a file named "bj�rn" in ISO-latin-1, while the Erlang VM is operating in Unicode file name mode (and therefore expecting UTF-8 file naming). The ISO-latin-1 name is not valid UTF-8 and one could be tempted to think that automatic conversion in for example <c>file:list_dir/1</c> is a good idea. But what would happen if we later tried to open the file and have the name as a Unicode list (magically converted from the ISO-latin-1 file name)? The VM will convert the file name given to UTF-8, as this is the encoding expected. Effectively this means trying to open the file named <<"bj�rn"/utf8>>. This file does not exist, and even if it existed it would not be the same file as the one that was listed. We could even create two files named "bj�rn", one named in the UTF-8 encoding and one not. If <c>file:list_dir/1</c> would automatically convert the ISO-latin-1 file name to a list, we would get two identical file names as the result. To avoid this, we need to differentiate between file names being properly encoded according to the Unicode file naming convention (i.e. UTF-8) and file names being invalid under the encoding. This is done by representing invalid encoding as "raw" file names, i.e. as binaries.</p> +<p>The core system of Erlang (kernel and stdlib) accepts raw file names except for loadable drivers and executables invoked using <c>open_port({spawn, ...} ...)</c>. <c>open_port({spawn_executable, ...} ...)</c> however does accept them. As mentioned earlier, the arguments given in the option list to <c>open_port({spawn_executable, ...} ...)</c> undergo the same conversion as the file names, meaning that the executable will be provided with arguments in UTF-8 as well. This translation is avoided consistently with how the file names are treated, by giving the argument as a binary.</p> +<p>To force Unicode file name translation mode on systems where this is not the default is considered experimental in OTP R14B01 due to the raw file names possibly being a new experience to the programmer and that the non core applications of OTP are not tested for compliance with raw file names yet. Unicode file name translation is expected to be default in future releases.</p> +<p>If working with raw file names, one can still conform to the encoding convention of the Erlang VM by using the <c>file:native_name_encoding/0</c> function, which returns either the atom <c>latin1</c> or the atom <c>utf8</c> depending on the file name translation mode. On Linux, a VM started without explicitly stating the file name translation mode will default to <c>latin1</c> as the native file name encoding, why file names on the disk encoded as UTF-8 will be returned as a list of the names interpreted as ISO-latin-1. The "UTF-8 list" is not a practical type for displaying or operating on in Erlang, but it is backward compatible and usable in all functions requiring a file name. On Windows and MacOSX, the default behavior is that of file name translation, why the <c>file:native_name_encoding/0</c> by default returns <c>utf8</c> on those systems (the fact that Windows actually does not use UTF-8 on the file system level can safely be ignored by the Erlang programmer). The default behavior can be changed using the <c>+fnu</c> or <c>+fnl</c> options to the VM, see the <c>erl</c> command manual page.</p> +<p>Even if you are operating without Unicode file naming translation automatically done by the VM, you can access and create files with names in UTF-8 encoding by using raw file names encoded as UTF-8. Enforcing the UTF-8 encoding regardless of the mode the Erlang VM is started in might, in some circumstances be a good idea, as the convention of using UTF-8 file names is spreading.</p> +</section> +<section> +<title>Notes about MacOSX</title> +<p>MacOSXs vfs layer enforces UTF-8 file names in a quite aggressive way. Older versions did this by simply refusing to create non UTF-8 conforming file names, while newer versions replace offending bytes with the sequence "%HH", where HH is the original character in hexadecimal notation. As Unicode translation is enabled by default on MacOSX, the only way to come up against this is to either start the VM with the <c>+fnl</c> flag or to use a raw file name in <c>latin1</c> encoding. In that case, the file can not be opened with the same name as the one used to create this. The problem is by design in newer versions of MacOSX.</p> +<p>MacOSX also reorganizes the names of files so that the representation of accents etc is denormalized, i.e. the character <c>�</c> is represented as the codepoints [111,776], where 111 is the character <c>o</c> and 776 is a special accent character. This type of denormalized Unicode is otherwise very seldom used and Erlang normalizes those file names on retrieval, so that denormalized file names is not passed up to the Erlang application. In Erlang the file name "bj�rn" is retrieved as [98,106,246,114,110], not as [98,106,117,776,114,110], even though the file system might think differently.</p> +</section> +</section> +<section> <title>Unicode-aware modules</title> <p>Most of the modules in Erlang/OTP are of course Unicode-unaware in the sense that they have no notion of Unicode and really shouldn't have. Typically they handle non-textual or byte-oriented data (like <c>gen_tcp</c> etc).</p> <p>Modules that actually handle textual data (like <c>io_lib</c>, <c>string</c> etc) are sometimes subject to conversion or extension to be able to handle Unicode characters.</p> diff --git a/lib/stdlib/src/c.erl b/lib/stdlib/src/c.erl index 6d50a575eb..d04d8f191f 100644 --- a/lib/stdlib/src/c.erl +++ b/lib/stdlib/src/c.erl @@ -659,7 +659,7 @@ portformat(Name, Id, Cmd) -> pwd() -> case file:get_cwd() of {ok, Str} -> - ok = io:format("~s\n", [Str]); + ok = io:format("~ts\n", [fixup_one_bin(Str)]); {error, _} -> ok = io:format("Cannot determine current directory\n") end. @@ -684,11 +684,27 @@ ls() -> ls(Dir) -> case file:list_dir(Dir) of {ok, Entries} -> - ls_print(sort(Entries)); + ls_print(sort(fixup_bin(Entries))); {error,_E} -> format("Invalid directory\n") end. +fixup_one_bin(X) when is_binary(X) -> + L = binary_to_list(X), + [ if + El > 127 -> + $?; + true -> + El + end || El <- L]; +fixup_one_bin(X) -> + X. +fixup_bin([H|T]) -> + [fixup_one_bin(H) | fixup_bin(T)]; +fixup_bin([]) -> + []. + + ls_print([]) -> ok; ls_print(L) -> Width = min([max(lengths(L, [])), 40]) + 5, @@ -698,7 +714,7 @@ ls_print(X, Width, Len) when Width + Len >= 80 -> io:nl(), ls_print(X, Width, 0); ls_print([H|T], Width, Len) -> - io:format("~-*s",[Width,H]), + io:format("~-*ts",[Width,H]), ls_print(T, Width, Len+Width); ls_print([], _, _) -> io:nl(). diff --git a/lib/stdlib/src/filelib.erl b/lib/stdlib/src/filelib.erl index d5ddf9ed7e..c845b61204 100644 --- a/lib/stdlib/src/filelib.erl +++ b/lib/stdlib/src/filelib.erl @@ -47,14 +47,14 @@ wildcard(Pattern) when is_list(Pattern) -> ?HANDLE_ERROR(do_wildcard(Pattern, file)). -spec wildcard(file:name(), file:name() | atom()) -> [file:filename()]. -wildcard(Pattern, Cwd) when is_list(Pattern), is_list(Cwd) -> +wildcard(Pattern, Cwd) when is_list(Pattern), (is_list(Cwd) or is_binary(Cwd)) -> ?HANDLE_ERROR(do_wildcard(Pattern, Cwd, file)); wildcard(Pattern, Mod) when is_list(Pattern), is_atom(Mod) -> ?HANDLE_ERROR(do_wildcard(Pattern, Mod)). -spec wildcard(file:name(), file:name(), atom()) -> [file:filename()]. wildcard(Pattern, Cwd, Mod) - when is_list(Pattern), is_list(Cwd), is_atom(Mod) -> + when is_list(Pattern), (is_list(Cwd) or is_binary(Cwd)), is_atom(Mod) -> ?HANDLE_ERROR(do_wildcard(Pattern, Cwd, Mod)). -spec is_dir(file:name()) -> boolean(). @@ -118,7 +118,7 @@ do_wildcard_comp({compiled_wildcard,{exists,File}}, Mod) -> do_wildcard_comp({compiled_wildcard,[Base|Rest]}, Mod) -> do_wildcard_1([Base], Rest, Mod). -do_wildcard(Pattern, Cwd, Mod) when is_list(Pattern), is_list(Cwd) -> +do_wildcard(Pattern, Cwd, Mod) when is_list(Pattern), (is_list(Cwd) or is_binary(Cwd)) -> do_wildcard_comp(do_compile_wildcard(Pattern), Cwd, Mod). do_wildcard_comp({compiled_wildcard,{exists,File}}, Cwd, Mod) -> @@ -127,9 +127,18 @@ do_wildcard_comp({compiled_wildcard,{exists,File}}, Cwd, Mod) -> _ -> [] end; do_wildcard_comp({compiled_wildcard,[current|Rest]}, Cwd0, Mod) -> - Cwd = filename:join([Cwd0]), %Slash away redundant slashes. - PrefixLen = length(Cwd)+1, - [lists:nthtail(PrefixLen, N) || N <- do_wildcard_1([Cwd], Rest, Mod)]; + {Cwd,PrefixLen} = case filename:join([Cwd0]) of + Bin when is_binary(Bin) -> {Bin,byte_size(Bin)+1}; + Other -> {Other,length(Other)+1} + end, %Slash away redundant slashes. + [ + if + is_binary(N) -> + <<_:PrefixLen/binary,Res/binary>> = N, + Res; + true -> + lists:nthtail(PrefixLen, N) + end || N <- do_wildcard_1([Cwd], Rest, Mod)]; do_wildcard_comp({compiled_wildcard,[Base|Rest]}, _Cwd, Mod) -> do_wildcard_1([Base], Rest, Mod). @@ -166,36 +175,44 @@ do_is_regular(File, Mod) -> %% If <Recursive> is true all sub-directories to <Dir> are processed do_fold_files(Dir, RegExp, Recursive, Fun, Acc, Mod) -> - {ok, Re1} = re:compile(RegExp), - do_fold_files1(Dir, Re1, Recursive, Fun, Acc, Mod). + {ok, Re1} = re:compile(RegExp,[unicode]), + do_fold_files1(Dir, Re1, RegExp, Recursive, Fun, Acc, Mod). -do_fold_files1(Dir, RegExp, Recursive, Fun, Acc, Mod) -> +do_fold_files1(Dir, RegExp, OrigRE, Recursive, Fun, Acc, Mod) -> case eval_list_dir(Dir, Mod) of - {ok, Files} -> do_fold_files2(Files, Dir, RegExp, Recursive, Fun, Acc, Mod); + {ok, Files} -> do_fold_files2(Files, Dir, RegExp, OrigRE, + Recursive, Fun, Acc, Mod); {error, _} -> Acc end. -do_fold_files2([], _Dir, _RegExp, _Recursive, _Fun, Acc, _Mod) -> +%% OrigRE is not to be compiled as it's for non conforming filenames, +%% i.e. for filenames that does not comply to the current encoding, which should +%% be very rare. We use it only in those cases and do not want to precompile. +do_fold_files2([], _Dir, _RegExp, _OrigRE, _Recursive, _Fun, Acc, _Mod) -> Acc; -do_fold_files2([File|T], Dir, RegExp, Recursive, Fun, Acc0, Mod) -> +do_fold_files2([File|T], Dir, RegExp, OrigRE, Recursive, Fun, Acc0, Mod) -> FullName = filename:join(Dir, File), case do_is_regular(FullName, Mod) of true -> - case re:run(File, RegExp, [{capture,none}]) of + case (catch re:run(File, if is_binary(File) -> OrigRE; + true -> RegExp end, + [{capture,none}])) of match -> Acc = Fun(FullName, Acc0), - do_fold_files2(T, Dir, RegExp, Recursive, Fun, Acc, Mod); + do_fold_files2(T, Dir, RegExp, OrigRE, Recursive, Fun, Acc, Mod); + {'EXIT',_} -> + do_fold_files2(T, Dir, RegExp, OrigRE, Recursive, Fun, Acc0, Mod); nomatch -> - do_fold_files2(T, Dir, RegExp, Recursive, Fun, Acc0, Mod) + do_fold_files2(T, Dir, RegExp, OrigRE, Recursive, Fun, Acc0, Mod) end; false -> case Recursive andalso do_is_dir(FullName, Mod) of true -> - Acc1 = do_fold_files1(FullName, RegExp, Recursive, + Acc1 = do_fold_files1(FullName, RegExp, OrigRE, Recursive, Fun, Acc0, Mod), - do_fold_files2(T, Dir, RegExp, Recursive, Fun, Acc1, Mod); + do_fold_files2(T, Dir, RegExp, OrigRE, Recursive, Fun, Acc1, Mod); false -> - do_fold_files2(T, Dir, RegExp, Recursive, Fun, Acc0, Mod) + do_fold_files2(T, Dir, RegExp, OrigRE, Recursive, Fun, Acc0, Mod) end end. @@ -268,6 +285,13 @@ do_wildcard_3(Base, [Pattern|Rest], Result, Mod) -> do_wildcard_3(Base, [], Result, _Mod) -> [Base|Result]. +wildcard_4(Pattern, [File|Rest], Base, Result) when is_binary(File) -> + case wildcard_5(Pattern, binary_to_list(File)) of + true -> + wildcard_4(Pattern, Rest, Base, [join(Base, File)|Result]); + false -> + wildcard_4(Pattern, Rest, Base, Result) + end; wildcard_4(Pattern, [File|Rest], Base, Result) -> case wildcard_5(Pattern, File) of true -> diff --git a/lib/stdlib/src/filename.erl b/lib/stdlib/src/filename.erl index 01c06e4596..e38b8957f2 100644 --- a/lib/stdlib/src/filename.erl +++ b/lib/stdlib/src/filename.erl @@ -41,6 +41,9 @@ -include_lib("kernel/include/file.hrl"). +-define(IS_DRIVELETTER(Letter),(((Letter >= $A) andalso (Letter =< $Z)) orelse + ((Letter >= $a) andalso (Letter =< $z)))). + %% Converts a relative filename to an absolute filename %% or the filename itself if it already is an absolute filename %% Note that no attempt is made to create the most beatiful @@ -57,12 +60,18 @@ %% (for Unix) : absname("/") -> "/" %% (for WIN32): absname("/") -> "D:/" --spec absname(file:name()) -> string(). + +-spec absname(file:name()) -> file:filename(). absname(Name) -> {ok, Cwd} = file:get_cwd(), absname(Name, Cwd). --spec absname(file:name(), string()) -> string(). +-spec absname(file:name(), file:filename()) -> file:filename(). +absname(Name, AbsBase) when is_binary(Name), is_list(AbsBase) -> + absname(Name,filename_string_to_binary(AbsBase)); +absname(Name, AbsBase) when is_list(Name), is_binary(AbsBase) -> + absname(filename_string_to_binary(Name),AbsBase); + absname(Name, AbsBase) -> case pathtype(Name) of relative -> @@ -77,6 +86,20 @@ absname(Name, AbsBase) -> %% Handles volumerelative names (on Windows only). +absname_vr([<<"/">>|Rest1], [Volume|_], _AbsBase) -> + %% Absolute path on current drive. + join([Volume|Rest1]); +absname_vr([<<X, $:>>|Rest1], [<<X,_/binary>>|_], AbsBase) -> + %% Relative to current directory on current drive. + absname(join(Rest1), AbsBase); +absname_vr([<<X, $:>>|Name], _, _AbsBase) -> + %% Relative to current directory on another drive. + Dcwd = + case file:get_cwd([X, $:]) of + {ok, Dir} -> filename_string_to_binary(Dir); + {error, _} -> <<X, $:, $/>> + end, + absname(join(Name), Dcwd); absname_vr(["/"|Rest1], [Volume|_], _AbsBase) -> %% Absolute path on current drive. join([Volume|Rest1]); @@ -92,41 +115,13 @@ absname_vr([[X, $:]|Name], _, _AbsBase) -> end, absname(join(Name), Dcwd). -%% Joins a relative filename to an absolute base. For VxWorks the -%% resulting name is fixed to minimize the length by collapsing -%% ".." directories. -%% For other systems this is just a join/2, but assumes that +%% Joins a relative filename to an absolute base. +%% This is just a join/2, but assumes that %% AbsBase must be absolute and Name must be relative. --spec absname_join(string(), file:name()) -> string(). +-spec absname_join(file:filename(), file:name()) -> file:filename(). absname_join(AbsBase, Name) -> - case major_os_type() of - vxworks -> - absname_pretty(AbsBase, split(Name), lists:reverse(split(AbsBase))); - _Else -> - join(AbsBase, flatten(Name)) - end. - -%% Handles absolute filenames for VxWorks - these are 'pretty-printed', -%% since a C function call chdir("/erlang/lib/../bin") really sets -%% cwd to '/erlang/lib/../bin' which also works, but the long term -%% effect is potentially not so good ... -%% -%% absname_pretty("../bin", "/erlang/lib") -> "/erlang/bin" -%% absname_pretty("../../../..", "/erlang") -> "/erlang" - -absname_pretty(Abspath, Relpath, []) -> - %% AbsBase _must_ begin with a vxworks device name - {device, _Rest, Dev} = vxworks_first(Abspath), - absname_pretty(Abspath, Relpath, [lists:reverse(Dev)]); -absname_pretty(_Abspath, [], AbsBase) -> - join(lists:reverse(AbsBase)); -absname_pretty(Abspath, [[$.]|Rest], AbsBase) -> - absname_pretty(Abspath, Rest, AbsBase); -absname_pretty(Abspath, [[$.,$.]|Rest], [_|AbsRest]) -> - absname_pretty(Abspath, Rest, AbsRest); -absname_pretty(Abspath, [First|Rest], AbsBase) -> - absname_pretty(Abspath, Rest, [First|AbsBase]). + join(AbsBase, flatten(Name)). %% Returns the part of the filename after the last directory separator, %% or the filename itself if it has no separators. @@ -136,12 +131,36 @@ absname_pretty(Abspath, [First|Rest], AbsBase) -> %% basename("/usr/foo/") -> "foo" (trailing slashes ignored) %% basename("/") -> [] --spec basename(file:name()) -> string(). +-spec basename(file:name()) -> file:filename(). +basename(Name) when is_binary(Name) -> + case os:type() of + {win32,_} -> + win_basenameb(Name); + _ -> + basenameb(Name,[<<"/">>]) + end; + basename(Name0) -> Name = flatten(Name0), {DirSep2, DrvSep} = separators(), basename1(skip_prefix(Name, DrvSep), [], DirSep2). +win_basenameb(<<Letter,$:,Rest/binary>>) when ?IS_DRIVELETTER(Letter) -> + basenameb(Rest,[<<"/">>,<<"\\">>]); +win_basenameb(O) -> + basenameb(O,[<<"/">>,<<"\\">>]). +basenameb(Bin,Sep) -> + Parts = [ X || X <- binary:split(Bin,Sep,[global]), + X =/= <<>> ], + if + Parts =:= [] -> + <<>>; + true -> + lists:last(Parts) + end. + + + basename1([$/|[]], Tail, DirSep2) -> basename1([], Tail, DirSep2); basename1([$/|Rest], _Tail, DirSep2) -> @@ -155,26 +174,11 @@ basename1([Char|Rest], Tail, DirSep2) when is_integer(Char) -> basename1([], Tail, _DirSep2) -> lists:reverse(Tail). -skip_prefix(Name, false) -> % No prefix for unix, but for VxWorks. - case major_os_type() of - vxworks -> - case vxworks_first(Name) of - {device, Rest, _Device} -> - Rest; - {not_device, _Rest, _First} -> - Name - end; - _Else -> - Name - end; -skip_prefix(Name, DrvSep) -> - skip_prefix1(Name, DrvSep). - -skip_prefix1([L, DrvSep|Name], DrvSep) when is_integer(L) -> +skip_prefix(Name, false) -> Name; -skip_prefix1([L], _) when is_integer(L) -> - [L]; -skip_prefix1(Name, _) -> +skip_prefix([L, DrvSep|Name], DrvSep) when ?IS_DRIVELETTER(L) -> + Name; +skip_prefix(Name, _) -> Name. %% Returns the last component of the filename, with the given @@ -190,7 +194,29 @@ skip_prefix1(Name, _) -> %% rootname(basename("xxx.jam")) -> "xxx" %% rootname(basename("xxx.erl")) -> "xxx" --spec basename(file:name(), file:name()) -> string(). +-spec basename(file:name(), file:name()) -> file:filename(). +basename(Name, Ext) when is_binary(Name), is_list(Ext) -> + basename(Name,filename_string_to_binary(Ext)); +basename(Name, Ext) when is_list(Name), is_binary(Ext) -> + basename(filename_string_to_binary(Name),Ext); +basename(Name, Ext) when is_binary(Name), is_binary(Ext) -> + BName = basename(Name), + LAll = byte_size(Name), + LN = byte_size(BName), + LE = byte_size(Ext), + case LN - LE of + Neg when Neg < 0 -> + BName; + Pos -> + StartLen = LAll - Pos - LE, + case Name of + <<_:StartLen/binary,Part:Pos/binary,Ext/binary>> -> + Part; + _Other -> + BName + end + end; + basename(Name0, Ext0) -> Name = flatten(Name0), Ext = flatten(Ext0), @@ -204,7 +230,7 @@ basename([$/|[]], Ext, Tail, DrvSep2) -> basename([], Ext, Tail, DrvSep2); basename([$/|Rest], Ext, _Tail, DrvSep2) -> basename(Rest, Ext, [], DrvSep2); -basename([$\\|Rest], Ext, Tail, DirSep2) when is_integer(DirSep2) -> +basename([DirSep2|Rest], Ext, Tail, DirSep2) when is_integer(DirSep2) -> basename([$/|Rest], Ext, Tail, DirSep2); basename([Char|Rest], Ext, Tail, DrvSep2) when is_integer(Char) -> basename(Rest, Ext, [Char|Tail], DrvSep2); @@ -216,21 +242,43 @@ basename([], _Ext, Tail, _DrvSep2) -> %% Example: dirname("/usr/src/kalle.erl") -> "/usr/src", %% dirname("kalle.erl") -> "." --spec dirname(file:name()) -> string(). +-spec dirname(file:name()) -> file:filename(). +dirname(Name) when is_binary(Name) -> + {Dsep,Drivesep} = separators(), + SList = case Dsep of + Sep when is_integer(Sep) -> + [ <<Sep>> ]; + _ -> + [] + end, + {XPart0,Dirs} = case Drivesep of + X when is_integer(X) -> + case Name of + <<DL,X,Rest/binary>> when ?IS_DRIVELETTER(DL) -> + {<<DL,X>>,Rest}; + _ -> + {<<>>,Name} + end; + _ -> + {<<>>,Name} + end, + Parts0 = binary:split(Dirs,[<<"/">>|SList],[global]), + %% Fairly short lists of parts, OK to reverse twice... + Parts = case Parts0 of + [] -> []; + _ -> lists:reverse(fstrip(tl(lists:reverse(Parts0)))) + end, + XPart = case {Parts,XPart0} of + {[],<<>>} -> + <<".">>; + _ -> + XPart0 + end, + dirjoin(Parts,XPart,<<"/">>); + dirname(Name0) -> Name = flatten(Name0), - case os:type() of - vxworks -> - {Devicep, Restname, FirstComp} = vxworks_first(Name), - case Devicep of - device -> - dirname(Restname, FirstComp, [], separators()); - _ -> - dirname(Name, [], [], separators()) - end; - _ -> - dirname(Name, [], [], separators()) - end. + dirname(Name, [], [], separators()). dirname([[_|_]=List|Rest], Dir, File, Seps) -> dirname(List++Rest, Dir, File, Seps); @@ -258,6 +306,26 @@ dirname([], [DrvSep,Dl], File, {_,DrvSep}) -> end; dirname([], Dir, _, _) -> lists:reverse(Dir). + +%% Compatibility with lists variant, remove trailing slashes +fstrip([<<>>,X|Y]) -> + fstrip([X|Y]); +fstrip(A) -> + A. + + +dirjoin([<<>>|T],Acc,Sep) -> + dirjoin1(T,<<Acc/binary,"/">>,Sep); +dirjoin(A,B,C) -> + dirjoin1(A,B,C). + +dirjoin1([],Acc,_) -> + Acc; +dirjoin1([One],Acc,_) -> + <<Acc/binary,One/binary>>; +dirjoin1([H|T],Acc,Sep) -> + dirjoin(T,<<Acc/binary,H/binary,Sep/binary>>,Sep). + %% Given a filename string, returns the file extension, %% including the period. Returns an empty list if there @@ -268,7 +336,31 @@ dirname([], Dir, _, _) -> %% %% On Windows: fn:dirname("\\usr\\src/kalle.erl") -> "/usr/src" --spec extension(file:name()) -> string(). +-spec extension(file:name()) -> file:filename(). +extension(Name) when is_binary(Name) -> + {Dsep,_} = separators(), + SList = case Dsep of + Sep when is_integer(Sep) -> + [ <<Sep>> ]; + _ -> + [] + end, + case binary:matches(Name,[<<".">>]) of + nomatch -> % Bug in binary workaround :( + <<>>; + [] -> + <<>>; + List -> + {Pos,_} = lists:last(List), + <<_:Pos/binary,Part/binary>> = Name, + case binary:match(Part,[<<"/">>|SList]) of + nomatch -> + Part; + _ -> + <<>> + end + end; + extension(Name0) -> Name = flatten(Name0), extension(Name, [], major_os_type()). @@ -281,8 +373,6 @@ extension([$/|Rest], _Result, OsType) -> extension(Rest, [], OsType); extension([$\\|Rest], _Result, win32) -> extension(Rest, [], win32); -extension([$\\|Rest], _Result, vxworks) -> - extension(Rest, [], vxworks); extension([Char|Rest], Result, OsType) when is_integer(Char) -> extension(Rest, [Char|Result], OsType); extension([], Result, _OsType) -> @@ -290,23 +380,36 @@ extension([], Result, _OsType) -> %% Joins a list of filenames with directory separators. --spec join([string()]) -> string(). +-spec join([file:filename()]) -> file:filename(). join([Name1, Name2|Rest]) -> join([join(Name1, Name2)|Rest]); join([Name]) when is_list(Name) -> join1(Name, [], [], major_os_type()); +join([Name]) when is_binary(Name) -> + join1b(Name, <<>>, [], major_os_type()); join([Name]) when is_atom(Name) -> join([atom_to_list(Name)]). %% Joins two filenames with directory separators. --spec join(string(), string()) -> string(). +-spec join(file:filename(), file:filename()) -> file:filename(). join(Name1, Name2) when is_list(Name1), is_list(Name2) -> OsType = major_os_type(), case pathtype(Name2) of relative -> join1(Name1, Name2, [], OsType); _Other -> join1(Name2, [], [], OsType) end; +join(Name1, Name2) when is_binary(Name1), is_list(Name2) -> + join(Name1,filename_string_to_binary(Name2)); +join(Name1, Name2) when is_list(Name1), is_binary(Name2) -> + join(filename_string_to_binary(Name1),Name2); +join(Name1, Name2) when is_binary(Name1), is_binary(Name2) -> + OsType = major_os_type(), + case pathtype(Name2) of + relative -> join1b(Name1, Name2, [], OsType); + _Other -> join1b(Name2, <<>>, [], OsType) + end; + join(Name1, Name2) when is_atom(Name1) -> join(atom_to_list(Name1), Name2); join(Name1, Name2) when is_atom(Name2) -> @@ -321,8 +424,6 @@ when is_integer(UcLetter), UcLetter >= $A, UcLetter =< $Z -> join1(Rest, RelativeName, [$:, UcLetter+$a-$A], win32); join1([$\\|Rest], RelativeName, Result, win32) -> join1([$/|Rest], RelativeName, Result, win32); -join1([$\\|Rest], RelativeName, Result, vxworks) -> - join1([$/|Rest], RelativeName, Result, vxworks); join1([$/|Rest], RelativeName, [$., $/|Result], OsType) -> join1(Rest, RelativeName, [$/|Result], OsType); join1([$/|Rest], RelativeName, [$/|Result], OsType) -> @@ -344,6 +445,26 @@ join1([Char|Rest], RelativeName, Result, OsType) when is_integer(Char) -> join1([Atom|Rest], RelativeName, Result, OsType) when is_atom(Atom) -> join1(atom_to_list(Atom)++Rest, RelativeName, Result, OsType). +join1b(<<UcLetter, $:, Rest/binary>>, RelativeName, [], win32) +when is_integer(UcLetter), UcLetter >= $A, UcLetter =< $Z -> + join1b(Rest, RelativeName, [$:, UcLetter+$a-$A], win32); +join1b(<<$\\,Rest/binary>>, RelativeName, Result, win32) -> + join1b(<<$/,Rest/binary>>, RelativeName, Result, win32); +join1b(<<$/,Rest/binary>>, RelativeName, [$., $/|Result], OsType) -> + join1b(Rest, RelativeName, [$/|Result], OsType); +join1b(<<$/,Rest/binary>>, RelativeName, [$/|Result], OsType) -> + join1b(Rest, RelativeName, [$/|Result], OsType); +join1b(<<>>, <<>>, Result, OsType) -> + list_to_binary(maybe_remove_dirsep(Result, OsType)); +join1b(<<>>, RelativeName, [$:|Rest], win32) -> + join1b(RelativeName, <<>>, [$:|Rest], win32); +join1b(<<>>, RelativeName, [$/|Result], OsType) -> + join1b(RelativeName, <<>>, [$/|Result], OsType); +join1b(<<>>, RelativeName, Result, OsType) -> + join1b(RelativeName, <<>>, [$/|Result], OsType); +join1b(<<Char,Rest/binary>>, RelativeName, Result, OsType) when is_integer(Char) -> + join1b(Rest, RelativeName, [Char|Result], OsType). + maybe_remove_dirsep([$/, $:, Letter], win32) -> [Letter, $:, $/]; maybe_remove_dirsep([$/], _) -> @@ -357,7 +478,7 @@ maybe_remove_dirsep(Name, _) -> %% a given base directory, which is is assumed to be normalised %% by a previous call to join/{1,2}. --spec append(string(), file:name()) -> string(). +-spec append(file:filename(), file:name()) -> file:filename(). append(Dir, Name) -> Dir ++ [$/|Name]. @@ -376,19 +497,14 @@ append(Dir, Name) -> -spec pathtype(file:name()) -> 'absolute' | 'relative' | 'volumerelative'. pathtype(Atom) when is_atom(Atom) -> pathtype(atom_to_list(Atom)); -pathtype(Name) when is_list(Name) -> +pathtype(Name) when is_list(Name) or is_binary(Name) -> case os:type() of {unix, _} -> unix_pathtype(Name); - {win32, _} -> win32_pathtype(Name); - vxworks -> case vxworks_first(Name) of - {device, _Rest, _Dev} -> - absolute; - _ -> - relative - end; - {ose,_} -> unix_pathtype(Name) + {win32, _} -> win32_pathtype(Name) end. +unix_pathtype(<<$/,_/binary>>) -> + absolute; unix_pathtype([$/|_]) -> absolute; unix_pathtype([List|Rest]) when is_list(List) -> @@ -404,6 +520,15 @@ win32_pathtype([Atom|Rest]) when is_atom(Atom) -> win32_pathtype(atom_to_list(Atom)++Rest); win32_pathtype([Char, List|Rest]) when is_list(List) -> win32_pathtype([Char|List++Rest]); +win32_pathtype(<<$/, $/, _/binary>>) -> absolute; +win32_pathtype(<<$\\, $/, _/binary>>) -> absolute; +win32_pathtype(<<$/, $\\, _/binary>>) -> absolute; +win32_pathtype(<<$\\, $\\, _/binary>>) -> absolute; +win32_pathtype(<<$/, _/binary>>) -> volumerelative; +win32_pathtype(<<$\\, _/binary>>) -> volumerelative; +win32_pathtype(<<_Letter, $:, $/, _/binary>>) -> absolute; +win32_pathtype(<<_Letter, $:, $\\, _/binary>>) -> absolute; +win32_pathtype(<<_Letter, $:, _/binary>>) -> volumerelative; win32_pathtype([$/, $/|_]) -> absolute; win32_pathtype([$\\, $/|_]) -> absolute; win32_pathtype([$/, $\\|_]) -> absolute; @@ -422,7 +547,9 @@ win32_pathtype(_) -> relative. %% Examples: rootname("/jam.src/kalle") -> "/jam.src/kalle" %% rootname("/jam.src/foo.erl") -> "/jam.src/foo" --spec rootname(file:name()) -> string(). +-spec rootname(file:name()) -> file:filename(). +rootname(Name) when is_binary(Name) -> + list_to_binary(rootname(binary_to_list(Name))); % No need to handle unicode, . is < 128 rootname(Name0) -> Name = flatten(Name0), rootname(Name, [], [], major_os_type()). @@ -431,8 +558,6 @@ rootname([$/|Rest], Root, Ext, OsType) -> rootname(Rest, [$/]++Ext++Root, [], OsType); rootname([$\\|Rest], Root, Ext, win32) -> rootname(Rest, [$/]++Ext++Root, [], win32); -rootname([$\\|Rest], Root, Ext, vxworks) -> - rootname(Rest, [$/]++Ext++Root, [], vxworks); rootname([$.|Rest], Root, [], OsType) -> rootname(Rest, Root, ".", OsType); rootname([$.|Rest], Root, Ext, OsType) -> @@ -451,7 +576,13 @@ rootname([], Root, _Ext, _OsType) -> %% Examples: rootname("/jam.src/kalle.jam", ".erl") -> "/jam.src/kalle.jam" %% rootname("/jam.src/foo.erl", ".erl") -> "/jam.src/foo" --spec rootname(file:name(), file:name()) -> string(). +-spec rootname(file:name(), file:name()) -> file:filename(). +rootname(Name, Ext) when is_binary(Name), is_binary(Ext) -> + list_to_binary(rootname(binary_to_list(Name),binary_to_list(Ext))); +rootname(Name, Ext) when is_binary(Name) -> + rootname(Name,filename_string_to_binary(Ext)); +rootname(Name, Ext) when is_binary(Ext) -> + rootname(filename_string_to_binary(Name),Ext); rootname(Name0, Ext0) -> Name = flatten(Name0), Ext = flatten(Ext0), @@ -471,27 +602,55 @@ rootname2([Char|Rest], Ext, Result) when is_integer(Char) -> %% split("foo/bar") -> ["foo", "bar"] %% split("a:\\msdev\\include") -> ["a:/", "msdev", "include"] --spec split(file:name()) -> [string()]. +-spec split(file:name()) -> [file:filename()]. +split(Name) when is_binary(Name) -> + case os:type() of + {win32, _} -> win32_splitb(Name); + _ -> unix_splitb(Name) + end; + split(Name0) -> Name = flatten(Name0), case os:type() of - {unix, _} -> unix_split(Name); {win32, _} -> win32_split(Name); - vxworks -> vxworks_split(Name); - {ose,_} -> unix_split(Name) + _ -> unix_split(Name) end. -%% If a VxWorks filename starts with '[/\].*[^/\]' '[/\].*:' or '.*:' -%% that part of the filename is considered a device. -%% The rest of the name is interpreted exactly as for win32. -%% XXX - dirty solution to make filename:split([]) return the same thing on -%% VxWorks as on unix and win32. -vxworks_split([]) -> - []; -vxworks_split(L) -> - {_Devicep, Rest, FirstComp} = vxworks_first(L), - split(Rest, [], [lists:reverse(FirstComp)], win32). +unix_splitb(Name) -> + L = binary:split(Name,[<<"/">>],[global]), + LL = case L of + [<<>>|Rest] -> + [<<"/">>|Rest]; + _ -> + L + end, + [ X || X <- LL, X =/= <<>>]. + + +fix_driveletter(Letter0) -> + if + Letter0 >= $A, Letter0 =< $Z -> + Letter0+$a-$A; + true -> + Letter0 + end. +win32_splitb(<<Letter0,$:, Slash, Rest/binary>>) when (((Slash =:= $\\) orelse (Slash =:= $/)) andalso + ?IS_DRIVELETTER(Letter0)) -> + Letter = fix_driveletter(Letter0), + L = binary:split(Rest,[<<"/">>,<<"\\">>],[global]), + [<<Letter,$:,$/>> | [ X || X <- L, X =/= <<>> ]]; +win32_splitb(<<Letter0,$:,Rest/binary>>) when ?IS_DRIVELETTER(Letter0) -> + Letter = fix_driveletter(Letter0), + L = binary:split(Rest,[<<"/">>,<<"\\">>],[global]), + [<<Letter,$:>> | [ X || X <- L, X =/= <<>> ]]; +win32_splitb(<<Slash,Rest/binary>>) when ((Slash =:= $\\) orelse (Slash =:= $/)) -> + L = binary:split(Rest,[<<"/">>,<<"\\">>],[global]), + [<<$/>> | [ X || X <- L, X =/= <<>> ]]; +win32_splitb(Name) -> + L = binary:split(Name,[<<"/">>,<<"\\">>],[global]), + [ X || X <- L, X =/= <<>> ]. + unix_split(Name) -> split(Name, [], unix). @@ -502,8 +661,6 @@ win32_split([X, $\\|Rest]) when is_integer(X) -> win32_split([X, $/|Rest]); win32_split([X, Y, $\\|Rest]) when is_integer(X), is_integer(Y) -> win32_split([X, Y, $/|Rest]); -win32_split([$/, $/|Rest]) -> - split(Rest, [], [[$/, $/]]); win32_split([UcLetter, $:|Rest]) when UcLetter >= $A, UcLetter =< $Z -> win32_split([UcLetter+$a-$A, $:|Rest]); win32_split([Letter, $:, $/|Rest]) -> @@ -540,7 +697,7 @@ split([], Comp, Components, OsType) -> %% will be converted to backslashes. On all platforms, the %% name will be normalized as done by join/1. --spec nativename(string()) -> string(). +-spec nativename(file:filename()) -> file:filename(). nativename(Name0) -> Name = join([Name0]), %Normalize. case os:type() of @@ -557,13 +714,12 @@ win32_nativename([]) -> separators() -> case os:type() of - {unix, _} -> {false, false}; {win32, _} -> {$\\, $:}; - vxworks -> {$\\, false}; - {ose,_} -> {false, false} + _ -> {false, false} end. + %% find_src(Module) -- %% find_src(Module, Rules) -- %% @@ -733,45 +889,12 @@ major_os_type() -> OsT -> OsT end. -%% Need to take care of the first pathname component separately -%% due to VxWorks less than good device naming rules. -%% (i.e. this is VxWorks specific ...) -%% The following four all starts with device names -%% elrond:/foo -> elrond: -%% elrond:\\foo.bar -> elrond: -%% /DISK1:foo -> /DISK1: -%% /usr/include -> /usr -%% This one doesn't: -%% foo/bar - -vxworks_first([]) -> - {not_device, [], []}; -vxworks_first([$/|T]) -> - vxworks_first2(device, T, [$/]); -vxworks_first([$\\|T]) -> - vxworks_first2(device, T, [$/]); -vxworks_first([H|T]) when is_list(H) -> - vxworks_first(H++T); -vxworks_first([H|T]) -> - vxworks_first2(not_device, T, [H]). - -vxworks_first2(Devicep, [], FirstComp) -> - {Devicep, [], FirstComp}; -vxworks_first2(Devicep, [$/|T], FirstComp) -> - {Devicep, [$/|T], FirstComp}; -vxworks_first2(Devicep, [$\\|T], FirstComp) -> - {Devicep, [$/|T], FirstComp}; -vxworks_first2(_Devicep, [$:|T], FirstComp)-> - {device, T, [$:|FirstComp]}; -vxworks_first2(Devicep, [H|T], FirstComp) when is_list(H) -> - vxworks_first2(Devicep, H++T, FirstComp); -vxworks_first2(Devicep, [H|T], FirstComp) -> - vxworks_first2(Devicep, T, [H|FirstComp]). - %% flatten(List) %% Flatten a list, also accepting atoms. --spec flatten(file:name()) -> string(). +-spec flatten(file:name()) -> file:filename(). +flatten(Bin) when is_binary(Bin) -> + Bin; flatten(List) -> do_flatten(List, []). @@ -785,3 +908,12 @@ do_flatten([], Tail) -> Tail; do_flatten(Atom, Tail) when is_atom(Atom) -> atom_to_list(Atom) ++ flatten(Tail). + +filename_string_to_binary(List) -> + case unicode:characters_to_binary(flatten(List),unicode,file:native_name_encoding()) of + {error,_,_} -> + erlang:error(badarg); + Bin when is_binary(Bin) -> + Bin + end. + diff --git a/lib/stdlib/test/binary_module_SUITE.erl b/lib/stdlib/test/binary_module_SUITE.erl index 16ed9a2c26..e4cdcf6125 100644 --- a/lib/stdlib/test/binary_module_SUITE.erl +++ b/lib/stdlib/test/binary_module_SUITE.erl @@ -186,7 +186,7 @@ badargs(Config) when is_list(Config) -> binary:match(<<1,2,3>>, {ac,ets:match_spec_compile([{'_',[],['$_']}])}, [{scope,{0,1}}])), - ?line nomatch = + ?line [] = ?MASK_ERROR(binary:matches(<<1,2,3>>,<<1>>,[{scope,{0,0}}])), ?line badarg = ?MASK_ERROR(binary:matches(<<1,2,3>>,{bm,<<>>},[{scope,{0,1}}])), diff --git a/lib/stdlib/test/ets_SUITE.erl b/lib/stdlib/test/ets_SUITE.erl index 2d8eb08105..620848003c 100644 --- a/lib/stdlib/test/ets_SUITE.erl +++ b/lib/stdlib/test/ets_SUITE.erl @@ -5231,8 +5231,6 @@ smp_select_delete(Config) when is_list(Config) -> types(doc) -> ["Test different types"]; types(Config) when is_list(Config) -> init_externals(), - io:format("ets:i() before test:\n",[]), - ets:i(), % SVERK: trouble shooting repeat_for_opts(types_do,[[set,ordered_set],compressed]). types_do(Opts) -> @@ -5328,6 +5326,10 @@ my_tab_to_list(Ts,Key, Acc) -> my_tab_to_list(Ts,ets:next(Ts,Key),[ets:lookup(Ts, Key)| Acc]). etsmem() -> + AllTabs = lists:map(fun(T) -> {T,ets:info(T,name),ets:info(T,size), + ets:info(T,memory),ets:info(T,type)} + end, ets:all()), + Mem = {try erlang:memory(ets) catch error:notsup -> notsup end, case erlang:system_info({allocator,ets_alloc}) of false -> undefined; @@ -5346,12 +5348,13 @@ etsmem() -> {value,{_,BlSz,_,_}} = lists:keysearch(blocks_size, 1, L), {Bl0+Bl,BlSz0+BlSz} end, {0,0}, MSBCS) - end}. + end}, + {Mem,AllTabs}. -verify_etsmem(MemInfo) -> +verify_etsmem({MemInfo,AllTabs}) -> wait_for_test_procs(), case etsmem() of - MemInfo -> + {MemInfo,_} -> io:format("Ets mem info: ~p", [MemInfo]), case MemInfo of {ErlMem,EtsAlloc} when ErlMem == notsup; EtsAlloc == undefined -> @@ -5360,13 +5363,15 @@ verify_etsmem(MemInfo) -> _ -> ok end; - Other -> + {MemInfo2, AllTabs2} -> io:format("Expected: ~p", [MemInfo]), - io:format("Actual: ~p", [Other]), - ets:i(), + io:format("Actual: ~p", [MemInfo2]), + io:format("Changed tables before: ~p\n",[AllTabs -- AllTabs2]), + io:format("Changed tables after: ~p\n", [AllTabs2 -- AllTabs]), ?t:fail() end. + start_loopers(N, Prio, Fun, State) -> lists:map(fun (_) -> my_spawn_opt(fun () -> looper(Fun, State) end, @@ -5683,6 +5688,7 @@ make_ext_ref() -> Ref. init_externals() -> + SysDistSz = ets:info(sys_dist,size), ?line Pa = filename:dirname(code:which(?MODULE)), ?line {ok, Node} = test_server:start_node(plopp, slave, [{args, " -pa " ++ Pa}]), ?line Res = case rpc:call(Node, ?MODULE, rpc_externals, []) of @@ -5691,6 +5697,17 @@ init_externals() -> R -> R end, ?line test_server:stop_node(Node), + + %% Wait for table 'sys_dist' to stabilize + repeat_while(fun() -> + case ets:info(sys_dist,size) of + SysDistSz -> false; + Sz -> + io:format("Waiting for sys_dist to revert size from ~p to size ~p\n", + [Sz, SysDistSz]), + receive after 1000 -> true end + end + end), put(externals, Res). rpc_externals() -> diff --git a/lib/stdlib/test/filelib_SUITE.erl b/lib/stdlib/test/filelib_SUITE.erl index d54741051f..5a279609c6 100644 --- a/lib/stdlib/test/filelib_SUITE.erl +++ b/lib/stdlib/test/filelib_SUITE.erl @@ -53,8 +53,11 @@ wildcard_one(Config) when is_list(Config) -> wildcard_two(Config) when is_list(Config) -> ?line Dir = filename:join(?config(priv_dir, Config), "wildcard_two"), + ?line DirB = unicode:characters_to_binary(Dir, file:native_name_encoding()), ?line ok = file:make_dir(Dir), - ?line do_wildcard_1(Dir, fun(Wc) -> filelib:wildcard(Wc, Dir) end), + ?line do_wildcard_1(Dir, fun(Wc) -> io:format("~p~n",[{Wc,Dir, X = filelib:wildcard(Wc, Dir)}]),X end), + ?line do_wildcard_1(Dir, fun(Wc) -> io:format("~p~n",[{Wc,DirB, X = filelib:wildcard(Wc, DirB)}]), + [unicode:characters_to_list(Y,file:native_name_encoding()) || Y <- X] end), ?line do_wildcard_1(Dir, fun(Wc) -> filelib:wildcard(Wc, Dir++"/") end), case os:type() of {win32,_} -> @@ -253,5 +256,7 @@ ensure_dir_eexist(Config) when is_list(Config) -> %% There already is a file with the name of the directory %% we want to create. ?line NeedFile = filename:join(Name, "file"), + ?line NeedFileB = filename:join(Name, <<"file">>), ?line {error, eexist} = filelib:ensure_dir(NeedFile), + ?line {error, eexist} = filelib:ensure_dir(NeedFileB), ok. diff --git a/lib/stdlib/test/filename_SUITE.erl b/lib/stdlib/test/filename_SUITE.erl index ab6521f37b..dbce93600b 100644 --- a/lib/stdlib/test/filename_SUITE.erl +++ b/lib/stdlib/test/filename_SUITE.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 1997-2009. All Rights Reserved. +%% Copyright Ericsson AB 1997-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -22,12 +22,19 @@ basename_1/1, basename_2/1, dirname/1, extension/1, join/1, t_nativename/1]). -export([pathtype/1,rootname/1,split/1,find_src/1]). +-export([absname_bin/1, absname_bin_2/1, + basename_bin_1/1, basename_bin_2/1, + dirname_bin/1, extension_bin/1, join_bin/1]). +-export([pathtype_bin/1,rootname_bin/1,split_bin/1]). -include("test_server.hrl"). all(suite) -> [absname, absname_2, basename_1, basename_2, dirname, extension, - join, pathtype, rootname, split, t_nativename, find_src]. + join, pathtype, rootname, split, t_nativename, find_src, + absname_bin, absname_bin_2, basename_bin_1, basename_bin_2, dirname_bin, + extension_bin, + join_bin, pathtype_bin, rootname_bin, split_bin]. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -457,3 +464,307 @@ find_src(Config) when is_list(Config) -> %% Try to find the source for a preloaded module. ?line {error,{preloaded,init}} = filename:find_src(init), ok. + +%% +%% +%% With binaries +%% +%% + +absname_bin(Config) when is_list(Config) -> + case os:type() of + {win32, _} -> + ?line [Drive|_] = ?config(priv_dir, Config), + ?line Temp = filename:join([Drive|":/"], "temp"), + ?line case file:make_dir(Temp) of + ok -> ok; + {error,eexist} -> ok + end, + ?line {ok,Cwd} = file:get_cwd(), + ?line ok = file:set_cwd(Temp), + ?line <<Drive:8,":/temp/foo">> = filename:absname(<<"foo">>), + ?line <<Drive:8,":/temp/../ebin">> = filename:absname(<<"../ebin">>), + ?line <<Drive:8,":/erlang">> = filename:absname(<<"/erlang">>), + ?line <<Drive:8,":/erlang/src">> = filename:absname(<<"/erlang/src">>), + ?line <<Drive:8,":/erlang/src">> = filename:absname(<<"\\erlang\\src">>), + ?line <<Drive:8,":/temp/erlang">> = filename:absname(<<Drive:8,":erlang">>), + ?line <<Drive:8,":/temp/erlang/src">> = + filename:absname(<<Drive:8,":erlang/src">>), + ?line <<Drive:8,":/temp/erlang/src">> = + filename:absname(<<Drive:8,":erlang\\src\\">>), + ?line <<"a:/erlang">> = filename:absname(<<"a:erlang">>), + + ?line file:set_cwd(<<Drive:8,":/">>), + ?line <<Drive:8,":/foo">> = filename:absname(<<"foo">>), + ?line <<Drive:8,":/../ebin">> = filename:absname(<<"../ebin">>), + ?line <<Drive:8,":/erlang">> = filename:absname(<<"/erlang">>), + ?line <<Drive:8,":/erlang/src">> = filename:absname(<<"/erlang/src">>), + ?line <<Drive:8,":/erlang/src">> = filename:absname(<<"\\erlang\\\\src">>), + ?line <<Drive:8,":/erlang">> = filename:absname(<<Drive:8,":erlang">>), + ?line <<Drive:8,":/erlang/src">> = filename:absname(<<Drive:8,":erlang/src">>), + ?line <<"a:/erlang">> = filename:absname(<<"a:erlang">>), + + ?line file:set_cwd(Cwd), + ok; + {unix, _} -> + ?line ok = file:set_cwd(<<"/usr">>), + ?line <<"/usr/foo">> = filename:absname(<<"foo">>), + ?line <<"/usr/../ebin">> = filename:absname(<<"../ebin">>), + + ?line file:set_cwd(<<"/">>), + ?line <<"/foo">> = filename:absname(<<"foo">>), + ?line <<"/../ebin">> = filename:absname(<<"../ebin">>), + ?line <<"/erlang">> = filename:absname(<<"/erlang">>), + ?line <<"/erlang/src">> = filename:absname(<<"/erlang/src">>), + ?line <<"/erlang/src">> = filename:absname(<<"/erlang///src">>), + ok + end. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +absname_bin_2(Config) when is_list(Config) -> + case os:type() of + {win32, _} -> + ?line [Drive|_] = ?config(priv_dir, Config), + ?line <<Drive:8,":/temp/foo">> = filename:absname(<<"foo">>, <<Drive:8,":/temp">>), + ?line <<Drive:8,":/temp/../ebin">> = filename:absname(<<"../ebin">>, + <<Drive:8,":/temp">>), + ?line <<Drive:8,":/erlang">> = filename:absname(<<"/erlang">>, <<Drive:8,":/temp">>), + ?line <<Drive:8,":/erlang/src">> = filename:absname(<<"/erlang/src">>, + <<Drive:8,":/temp">>), + ?line <<Drive:8,":/erlang/src">> = filename:absname(<<"\\erlang\\src">>, + <<Drive:8,":/temp">>), + ?line <<Drive:8,":/temp/erlang">> = filename:absname(<<Drive:8,":erlang">>, + <<Drive:8,":/temp">>), + ?line <<Drive:8,":/temp/erlang/src">> = filename:absname(<<Drive:8,":erlang/src">>, + <<Drive:8,":/temp">>), + ?line <<Drive:8,":/temp/erlang/src">> = + filename:absname(<<Drive:8,":erlang\\src\\">>, <<Drive:8,":/temp">>), + ?line <<"a:/erlang">> = filename:absname(<<"a:erlang">>, <<Drive:8,":/temp">>), + + ?line file:set_cwd(<<Drive:8,":/">>), + ?line <<Drive:8,":/foo">> = filename:absname(foo, <<Drive:8,":/">>), + ?line <<Drive:8,":/foo">> = filename:absname(<<"foo">>, <<Drive:8,":/">>), + ?line <<Drive:8,":/../ebin">> = filename:absname(<<"../ebin">>, <<Drive:8,":/">>), + ?line <<Drive:8,":/erlang">> = filename:absname(<<"/erlang">>, <<Drive:8,":/">>), + ?line <<Drive:8,":/erlang/src">> = filename:absname(<<"/erlang/src">>, + <<Drive:8,":/">>), + ?line <<Drive:8,":/erlang/src">> = filename:absname(<<"\\erlang\\\\src">>, + <<Drive:8,":/">>), + ?line <<Drive:8,":/erlang">> = filename:absname(<<Drive:8,":erlang">>, + <<Drive:8,":/">>), + ?line <<Drive:8,":/erlang/src">> = filename:absname(<<Drive:8,":erlang/src">>, + <<Drive:8,":/">>), + ?line <<"a:/erlang">> = filename:absname(<<"a:erlang">>, <<Drive:8,":/">>), + + ok; + {unix, _} -> + ?line <<"/usr/foo">> = filename:absname(<<"foo">>, <<"/usr">>), + ?line <<"/usr/../ebin">> = filename:absname(<<"../ebin">>, <<"/usr">>), + + ?line <<"/foo">> = filename:absname(<<"foo">>, <<"/">>), + ?line <<"/../ebin">> = filename:absname(<<"../ebin">>, <<"/">>), + ?line <<"/erlang">> = filename:absname(<<"/erlang">>, <<"/">>), + ?line <<"/erlang/src">> = filename:absname(<<"/erlang/src">>, <<"/">>), + ?line <<"/erlang/src">> = filename:absname(<<"/erlang///src">>, <<"/">>), + ok + end. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +basename_bin_1(Config) when is_list(Config) -> + ?line Dog = test_server:timetrap(test_server:seconds(10)), + ?line <<".">> = filename:basename(<<".">>), + ?line <<"foo">> = filename:basename(<<"foo">>), + ?line <<"foo">> = filename:basename(<<"/usr/foo">>), + ?line <<"foo.erl">> = filename:basename(<<"A:usr/foo.erl">>), + ?line case os:type() of + {win32, _} -> + ?line <<"foo">> = filename:basename(<<"A:\\usr\\foo">>), + ?line <<"foo">> = filename:basename(<<"A:foo">>); + {unix, _} -> + ?line <<"strange\\but\\true">> = + filename:basename(<<"strange\\but\\true">>) + end, + ?line test_server:timetrap_cancel(Dog), + ok. + +basename_bin_2(Config) when is_list(Config) -> + ?line Dog = test_server:timetrap(test_server:seconds(10)), + ?line <<".">> = filename:basename(<<".">>, <<".erl">>), + ?line <<"foo">> = filename:basename(<<"foo.erl">>, <<".erl">>), + ?line <<"foo.erl">> = filename:basename(<<"/usr/foo.erl">>, <<".hrl">>), + ?line <<"foo.erl">> = filename:basename(<<"/usr.hrl/foo.erl">>, <<".hrl">>), + ?line <<"foo">> = filename:basename(<<"/usr.hrl/foo">>, <<".hrl">>), + ?line <<"foo">> = filename:basename(<<"usr/foo/">>, <<".erl">>), + ?line <<"foo.erl">> = filename:basename(<<"usr/foo.erl/">>, <<".erl">>), + ?line case os:type() of + {win32, _} -> + ?line <<"foo">> = filename:basename(<<"A:foo">>, <<".erl">>), + ?line <<"foo.erl">> = filename:basename(<<"a:\\usr\\foo.erl">>, + <<".hrl">>), + ?line <<"foo.erl">> = filename:basename(<<"c:\\usr.hrl\\foo.erl">>, + <<".hrl">>), + ?line <<"foo">> = filename:basename(<<"A:\\usr\\foo">>, <<".hrl">>); + {unix, _} -> + ?line <<"strange\\but\\true">> = + filename:basename(<<"strange\\but\\true.erl">>, <<".erl">>), + ?line <<"strange\\but\\true">> = + filename:basename(<<"strange\\but\\true">>, <<".erl">>) + end, + ?line test_server:timetrap_cancel(Dog), + ok. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +dirname_bin(Config) when is_list(Config) -> + case os:type() of + {win32,_} -> + ?line <<"A:/usr">> = filename:dirname(<<"A:/usr/foo.erl">>), + ?line <<"A:usr">> = filename:dirname(<<"A:usr/foo.erl">>), + ?line <<"/usr">> = filename:dirname(<<"\\usr\\foo.erl">>), + ?line <<"/">> = filename:dirname(<<"\\usr">>), + ?line <<"A:">> = filename:dirname(<<"A:">>); + vxworks -> + ?line <<"net:/usr">> = filename:dirname(<<"net:/usr/foo.erl">>), + ?line <<"/disk0:/usr">> = filename:dirname(<<"/disk0:/usr/foo.erl">>), + ?line <<"/usr">> = filename:dirname(<<"\\usr\\foo.erl">>), + ?line <<"/usr">> = filename:dirname(<<"\\usr">>), + ?line <<"net:">> = filename:dirname(<<"net:">>); + _ -> true + end, + ?line <<"usr">> = filename:dirname(<<"usr///foo.erl">>), + ?line <<".">> = filename:dirname(<<"foo.erl">>), + ?line <<".">> = filename:dirname(<<".">>), + case os:type() of + vxworks -> + ?line <<"/">> = filename:dirname(<<"/">>), + ?line <<"/usr">> = filename:dirname(<<"/usr">>); + _ -> + ?line <<"/">> = filename:dirname(<<"/">>), + ?line <<"/">> = filename:dirname(<<"/usr">>) + end, + ok. + + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +extension_bin(Config) when is_list(Config) -> + ?line <<".erl">> = filename:extension(<<"A:/usr/foo.erl">>), + ?line <<".erl">> = filename:extension(<<"A:/usr/foo.nisse.erl">>), + ?line <<".erl">> = filename:extension(<<"A:/usr.bar/foo.nisse.erl">>), + ?line <<"">> = filename:extension(<<"A:/usr.bar/foo">>), + ?line <<"">> = filename:extension(<<"A:/usr/foo">>), + ?line case os:type() of + {win32, _} -> + ?line <<"">> = filename:extension(<<"A:\\usr\\foo">>), + ?line <<".erl">> = + filename:extension(<<"A:/usr.bar/foo.nisse.erl">>), + ?line <<"">> = filename:extension(<<"A:/usr.bar/foo">>), + ok; + vxworks -> + ?line <<"">> = filename:extension(<<"/disk0:\\usr\\foo">>), + ?line <<".erl">> = + filename:extension(<<"net:/usr.bar/foo.nisse.erl">>), + ?line <<"">> = filename:extension(<<"net:/usr.bar/foo">>), + ok; + _ -> ok + end. + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +join_bin(Config) when is_list(Config) -> + ?line <<"/">> = filename:join([<<"/">>]), + ?line <<"/">> = filename:join([<<"//">>]), + ?line <<"usr/foo.erl">> = filename:join(<<"usr">>,<<"foo.erl">>), + ?line <<"/src/foo.erl">> = filename:join(usr, <<"/src/foo.erl">>), + ?line <<"/src/foo.erl">> = filename:join([<<"/src/">>,'foo.erl']), + ?line <<"/src/foo.erl">> = filename:join(<<"usr">>, ["/sr", 'c/foo.erl']), + ?line <<"/src/foo.erl">> = filename:join(<<"usr">>, <<"/src/foo.erl">>), + + %% Make sure that redundant slashes work too. + ?line <<"a/b/c/d/e/f/g">> = filename:join([<<"a//b/c/////d//e/f/g">>]), + ?line <<"a/b/c/d/e/f/g">> = filename:join([<<"a//b/c/">>, <<"d//e/f/g">>]), + ?line <<"a/b/c/d/e/f/g">> = filename:join([<<"a//b/c">>, <<"d//e/f/g">>]), + ?line <<"/d/e/f/g">> = filename:join([<<"a//b/c">>, <<"/d//e/f/g">>]), + ?line <<"/d/e/f/g">> = filename:join([<<"a//b/c">>, <<"//d//e/f/g">>]), + + ?line <<"foo/bar">> = filename:join([$f,$o,$o,$/,[]], <<"bar">>), + + ?line case os:type() of + {win32, _} -> + ?line <<"d:/">> = filename:join([<<"D:/">>]), + ?line <<"d:/">> = filename:join([<<"D:\\">>]), + ?line <<"d:/abc">> = filename:join([<<"D:/">>, <<"abc">>]), + ?line <<"d:abc">> = filename:join([<<"D:">>, <<"abc">>]), + ?line <<"a/b/c/d/e/f/g">> = + filename:join([<<"a//b\\c//\\/\\d/\\e/f\\g">>]), + ?line <<"a:usr/foo.erl">> = + filename:join([<<"A:">>,<<"usr">>,<<"foo.erl">>]), + ?line <<"/usr/foo.erl">> = + filename:join([<<"A:">>,<<"/usr">>,<<"foo.erl">>]), + ?line <<"c:usr">> = filename:join(<<"A:">>,<<"C:usr">>), + ?line <<"a:usr">> = filename:join(<<"A:">>,<<"usr">>), + ?line <<"c:/usr">> = filename:join(<<"A:">>, <<"C:/usr">>), + ?line <<"c:/usr/foo.erl">> = + filename:join([<<"A:">>,<<"C:/usr">>,<<"foo.erl">>]), + ?line <<"c:usr/foo.erl">> = + filename:join([<<"A:">>,<<"C:usr">>,<<"foo.erl">>]), + ?line <<"d:/foo">> = filename:join([$D, $:, $/, []], <<"foo">>), + ok; + {unix, _} -> + ok + end. + +pathtype_bin(Config) when is_list(Config) -> + ?line relative = filename:pathtype(<<"..">>), + ?line relative = filename:pathtype(<<"foo">>), + ?line relative = filename:pathtype(<<"foo/bar">>), + ?line relative = filename:pathtype('foo/bar'), + case os:type() of + {win32, _} -> + ?line volumerelative = filename:pathtype(<<"/usr/local/bin">>), + ?line volumerelative = filename:pathtype(<<"A:usr/local/bin">>), + ok; + {unix, _} -> + ?line absolute = filename:pathtype(<<"/">>), + ?line absolute = filename:pathtype(<<"/usr/local/bin">>), + ok + end. + +rootname_bin(Config) when is_list(Config) -> + ?line <<"/jam.src/kalle">> = filename:rootname(<<"/jam.src/kalle">>), + ?line <<"/jam.src/foo">> = filename:rootname(<<"/jam.src/foo.erl">>), + ?line <<"/jam.src/foo">> = filename:rootname(<<"/jam.src/foo.erl">>, <<".erl">>), + ?line <<"/jam.src/foo.jam">> = filename:rootname(<<"/jam.src/foo.jam">>, <<".erl">>), + ?line <<"/jam.src/foo.jam">> = filename:rootname(["/jam.sr",'c/foo.j',"am"],<<".erl">>), + ?line <<"/jam.src/foo.jam">> = filename:rootname(["/jam.sr",'c/foo.j'|am],<<".erl">>), + ok. + +split_bin(Config) when is_list(Config) -> + case os:type() of + vxworks -> + ?line [<<"/usr">>,<<"local">>,<<"bin">>] = filename:split(<<"/usr/local/bin">>); + _ -> + ?line [<<"/">>,<<"usr">>,<<"local">>,<<"bin">>] = filename:split(<<"/usr/local/bin">>) + end, + ?line [<<"foo">>,<<"bar">>]= filename:split(<<"foo/bar">>), + ?line [<<"foo">>, <<"bar">>, <<"hello">>]= filename:split(<<"foo////bar//hello">>), + case os:type() of + {win32,_} -> + ?line [<<"a:/">>,<<"msdev">>,<<"include">>] = + filename:split(<<"a:/msdev/include">>), + ?line [<<"a:/">>,<<"msdev">>,<<"include">>] = + filename:split(<<"A:/msdev/include">>), + ?line [<<"msdev">>,<<"include">>] = + filename:split(<<"msdev\\include">>), + ?line [<<"a:/">>,<<"msdev">>,<<"include">>] = + filename:split(<<"a:\\msdev\\include">>), + ?line [<<"a:">>,<<"msdev">>,<<"include">>] = + filename:split(<<"a:msdev\\include">>), + ok; + _ -> + ok + end. + diff --git a/lib/stdlib/vsn.mk b/lib/stdlib/vsn.mk index db7954af04..ac02e1f359 100644 --- a/lib/stdlib/vsn.mk +++ b/lib/stdlib/vsn.mk @@ -1 +1 @@ -STDLIB_VSN = 1.17.2 +STDLIB_VSN = 1.17.3 diff --git a/lib/syntax_tools/doc/src/notes.xml b/lib/syntax_tools/doc/src/notes.xml index fca93a27d9..3f5eb7231e 100644 --- a/lib/syntax_tools/doc/src/notes.xml +++ b/lib/syntax_tools/doc/src/notes.xml @@ -31,6 +31,20 @@ <p>This document describes the changes made to the Syntax_Tools application.</p> +<section><title>Syntax_Tools 1.6.7</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Miscellaneous updates</p> + <p> + Own Id: OTP-8976</p> + </item> + </list> + </section> + +</section> + <section><title>Syntax_Tools 1.6.6</title> <section><title>Improvements and New Features</title> diff --git a/lib/syntax_tools/vsn.mk b/lib/syntax_tools/vsn.mk index 6051fb8e39..2e23f6aef9 100644 --- a/lib/syntax_tools/vsn.mk +++ b/lib/syntax_tools/vsn.mk @@ -1 +1 @@ -SYNTAX_TOOLS_VSN = 1.6.6 +SYNTAX_TOOLS_VSN = 1.6.7 diff --git a/lib/test_server/doc/src/notes.xml b/lib/test_server/doc/src/notes.xml index e0c4c28e44..ab329c399b 100644 --- a/lib/test_server/doc/src/notes.xml +++ b/lib/test_server/doc/src/notes.xml @@ -32,6 +32,20 @@ <file>notes.xml</file> </header> +<section><title>Test_Server 3.4.2</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Miscellaneous updates</p> + <p> + Own Id: OTP-8976</p> + </item> + </list> + </section> + +</section> + <section><title>Test_Server 3.4.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/test_server/vsn.mk b/lib/test_server/vsn.mk index 4c3df28814..4e293b76a7 100644 --- a/lib/test_server/vsn.mk +++ b/lib/test_server/vsn.mk @@ -1,2 +1,2 @@ -TEST_SERVER_VSN = 3.4.1 +TEST_SERVER_VSN = 3.4.2 diff --git a/lib/tools/doc/src/notes.xml b/lib/tools/doc/src/notes.xml index 4a71993da9..5f9cecd6e4 100644 --- a/lib/tools/doc/src/notes.xml +++ b/lib/tools/doc/src/notes.xml @@ -30,6 +30,26 @@ </header> <p>This document describes the changes made to the Tools application.</p> +<section><title>Tools 2.6.6.2</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p>eprof: API sort mismatch has now been fixed. </p> + <p> + Own Id: OTP-8853</p> + </item> + <item> + <p> + eprof: fix division by zero in statistics</p> + <p> + Own Id: OTP-8963</p> + </item> + </list> + </section> + +</section> + <section><title>Tools 2.6.6.1</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/tools/src/eprof.erl b/lib/tools/src/eprof.erl index 8e5c0ec46b..87fdc1fa34 100644 --- a/lib/tools/src/eprof.erl +++ b/lib/tools/src/eprof.erl @@ -136,7 +136,7 @@ handle_call({analyze, procs, Opts}, _, #state{ bpd = #bpd{ p = Ps, us = Tus} = B lists:foreach(fun ({Pid, Mfas}) -> {Pn, Pus} = sum_bp_total_n_us(Mfas), - format(Fd, "~n****** Process ~w -- ~s % of profiled time *** ~n", [Pid, s("~.2f", [100.0*(Pus/Tus)])]), + format(Fd, "~n****** Process ~w -- ~s % of profiled time *** ~n", [Pid, s("~.2f", [100.0*divide(Pus,Tus)])]), print_bp_mfa(Mfas, {Pn,Pus}, Fd, Opts), ok end, gb_trees:to_list(Ps)), @@ -443,8 +443,8 @@ string_bp_mfa([{Mfa, {Count, Time}}|Mfas], Tus, {MfaW, CountW, PercW, TimeW, TpC Smfa = s(Mfa), Scount = s(Count), Stime = s(Time), - Sperc = s("~.2f", [100*(Time/Tus)]), - Stpc = s("~.2f", [Time/Count]), + Sperc = s("~.2f", [100*divide(Time,Tus)]), + Stpc = s("~.2f", [divide(Time,Count)]), string_bp_mfa(Mfas, Tus, { erlang:max(MfaW, length(Smfa)), @@ -484,3 +484,6 @@ format(Fd, Format, Strings) -> io:format(Fd, Format, Strings), io:format(Format, Strings), ok. + +divide(_,0) -> 0.0; +divide(T,N) -> T/N. diff --git a/lib/tools/vsn.mk b/lib/tools/vsn.mk index 77b5254eaa..0c89b18065 100644 --- a/lib/tools/vsn.mk +++ b/lib/tools/vsn.mk @@ -1 +1 @@ -TOOLS_VSN = 2.6.6.1 +TOOLS_VSN = 2.6.6.2 diff --git a/lib/wx/doc/src/notes.xml b/lib/wx/doc/src/notes.xml index 8414028106..1493501ea4 100644 --- a/lib/wx/doc/src/notes.xml +++ b/lib/wx/doc/src/notes.xml @@ -31,6 +31,25 @@ <p>This document describes the changes made to the wxErlang application.</p> +<section><title>Wx 0.98.8</title> + + <section><title>Improvements and New Features</title> + <list> + <item> + <p>Add wxSystemSettings which was missing in the previous + release, despite previous comments.</p> <p>Fix an + external loop when stopping erlang nicely.</p> + <p>Separate OpenGL to it's own dynamic loaded library, so + other graphic libraries can reuse the gl module and it + will not waste memory if not used.</p> + <p> + Own Id: OTP-8951</p> + </item> + </list> + </section> + +</section> + <section><title>Wx 0.98.7</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/wx/vsn.mk b/lib/wx/vsn.mk index c0cc302317..dea0678063 100644 --- a/lib/wx/vsn.mk +++ b/lib/wx/vsn.mk @@ -1 +1 @@ -WX_VSN = 0.98.7 +WX_VSN = 0.98.8 diff --git a/lib/xmerl/doc/src/notes.xml b/lib/xmerl/doc/src/notes.xml index d67a622481..aa66cbec77 100644 --- a/lib/xmerl/doc/src/notes.xml +++ b/lib/xmerl/doc/src/notes.xml @@ -31,6 +31,40 @@ <p>This document describes the changes made to the Xmerl application.</p> +<section><title>Xmerl 1.2.7</title> + + <section><title>Fixed Bugs and Malfunctions</title> + <list> + <item> + <p> An empty element declared as simpleContent was not + properly validated. </p> + <p> + Own Id: OTP-8599</p> + </item> + <item> + <p> Fix format_man_pages so it handles all man sections + and remove warnings/errors in various man pages. </p> + <p> + Own Id: OTP-8600</p> + </item> + </list> + </section> + + + <section><title>Improvements and New Features</title> + <list> + <item> + <p> Fix entity checking so there are no fatal errors for + undefined entities when option skip_external_dtd is used. + </p> + <p> + Own Id: OTP-8947</p> + </item> + </list> + </section> + +</section> + <section><title>Xmerl 1.2.6</title> <section><title>Fixed Bugs and Malfunctions</title> diff --git a/lib/xmerl/src/xmerl_sax_parser_base.erlsrc b/lib/xmerl/src/xmerl_sax_parser_base.erlsrc index 9d184152d1..3b9eaa309c 100644 --- a/lib/xmerl/src/xmerl_sax_parser_base.erlsrc +++ b/lib/xmerl/src/xmerl_sax_parser_base.erlsrc @@ -1,7 +1,7 @@ %%-*-erlang-*- %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2008-2009. All Rights Reserved. +%% Copyright Ericsson AB 2008-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -934,6 +934,13 @@ parse_att_value(?STRING_REST("&", Rest), State, Stop, Acc) -> parse_att_value(Rest1, State2, Stop, ParsedValue ++ Acc); {external_general, Name, _} -> ?fatal_error(State1, "External parsed entity reference in attribute value: " ++ Name); + {not_found, Name} -> + case State#xmerl_sax_parser_state.skip_external_dtd of + false -> + ?fatal_error(State1, "Entity not declared: " ++ Name); %%VC: Entity Declared + true -> + parse_att_value(Rest1, State1, Stop, ";" ++ lists:reverse(Name) ++ "&" ++ Acc) + end; {unparsed, Name, _} -> ?fatal_error(State1, "Unparsed entity reference in attribute value: " ++ Name) end; @@ -1098,6 +1105,13 @@ parse_content(?STRING_REST("&", Rest), State, Acc, _IgnorableWS) -> {external_general, _, {PubId, SysId}} -> State2 = parse_external_entity(State1, PubId, SysId), parse_content(Rest1, State2, Acc, false); + {not_found, Name} -> + case State#xmerl_sax_parser_state.skip_external_dtd of + false -> + ?fatal_error(State1, "Entity not declared: " ++ Name); %%VC: Entity Declared + true -> + parse_content(Rest1, State1, ";" ++ lists:reverse(Name) ++ "&" ++ Acc, false) + end; {unparsed, Name, _} -> ?fatal_error(State1, "Unparsed entity reference in content: " ++ Name) end; @@ -1357,7 +1371,7 @@ look_up_reference(Name, HaveToExist, State) -> yes -> ?fatal_error(State, "Entity not declared: " ++ Name); %%WFC: Entity Declared no -> - ?fatal_error(State, "Entity not declared: " ++ Name) %%VC: Entity Declared + {not_found, Name} %%VC: Entity Declared end; false -> {not_found, Name} @@ -1869,7 +1883,14 @@ parse_doctype_decl(?STRING_REST("%", Rest), State) -> parse_doctype_decl(?APPEND_STRING(IValue, Rest1), State1); {external_parameter, _, {PubId, SysId}} -> State2 = parse_external_entity(State1#xmerl_sax_parser_state{file_type = entity}, PubId, SysId), - parse_doctype_decl(Rest1, State2) + parse_doctype_decl(Rest1, State2); + {not_found, Name} -> + case State#xmerl_sax_parser_state.skip_external_dtd of + false -> + ?fatal_error(State1, "Entity not declared: " ++ Name); %%WFC: Entity Declared + true -> + parse_doctype_decl(Rest1, State1) + end end; parse_doctype_decl(?STRING_REST("<!", Rest1), State) -> parse_doctype_decl_1(Rest1, State); @@ -2443,7 +2464,7 @@ parse_ndata(Bytes, State) -> %% Acc = string() %% Result : {Value, Rest, State} %% Value = string() -%% Description: Parse an attribute value +%% Description: Parse an entity value %%---------------------------------------------------------------------- parse_entity_value(?STRING_EMPTY, State, undefined, Acc) -> {Acc, [], State}; %% stop clause when parsing references @@ -2473,7 +2494,7 @@ parse_entity_value(?STRING_REST("&", Rest), State, Stop, Acc) -> {external_general, Name, _} -> parse_entity_value(Rest1, State1, Stop, ";" ++ lists:reverse(Name) ++ "&" ++ Acc); {not_found, Name} -> - parse_entity_value(Rest1, State1, Stop, ";" ++ lists:reverse(Name) ++ "&" ++ Acc); + parse_entity_value(Rest1, State1, Stop, ";" ++ lists:reverse(Name) ++ "&" ++ Acc); {unparsed, Name, _} -> ?fatal_error(State1, "Unparsed entity reference in entity value: " ++ Name) end; @@ -2490,7 +2511,15 @@ parse_entity_value(?STRING_REST("%", Rest), #xmerl_sax_parser_state{file_type=Ty IValue = ?TO_INPUT_FORMAT(" " ++ RefValue ++ " "), parse_entity_value(?APPEND_STRING(IValue, Rest1), State1, Stop, Acc); {external_parameter, _, {_PubId, _SysId}} -> - ?fatal_error(State1, "Parameter references in entity value not supported yet.") + ?fatal_error(State1, "Parameter references in entity value not supported yet."); + {not_found, Name} -> + case State#xmerl_sax_parser_state.skip_external_dtd of + false -> + ?fatal_error(State1, "Entity not declared: " ++ Name); %%VC: Entity Declared + true -> + parse_entity_value(Rest1, State1, Stop, ";" ++ lists:reverse(Name) ++ "&" ++ Acc) + end + end end; parse_entity_value(?STRING_UNBOUND_REST(Stop, Rest), State, Stop, Acc) -> diff --git a/lib/xmerl/src/xmerl_scan.erl b/lib/xmerl/src/xmerl_scan.erl index e2e6f95c4a..e07d495fc7 100644 --- a/lib/xmerl/src/xmerl_scan.erl +++ b/lib/xmerl/src/xmerl_scan.erl @@ -34,7 +34,9 @@ %% See also <a href="xmerl_examples.html">tutorial</a> on customization %% functions. %% </p> +%% <p> %% Possible options are: +%% </p> %% <dl> %% <dt><code>{acc_fun, Fun}</code></dt> %% <dd>Call back function to accumulate contents of entity.</dd> diff --git a/lib/xmerl/src/xmerl_xpath.erl b/lib/xmerl/src/xmerl_xpath.erl index 182a186d2c..e654a8ef1d 100644 --- a/lib/xmerl/src/xmerl_xpath.erl +++ b/lib/xmerl/src/xmerl_xpath.erl @@ -1,7 +1,7 @@ %% %% %CopyrightBegin% %% -%% Copyright Ericsson AB 2003-2009. All Rights Reserved. +%% Copyright Ericsson AB 2003-2010. All Rights Reserved. %% %% The contents of this file are subject to the Erlang Public License, %% Version 1.1, (the "License"); you may not use this file except in @@ -57,7 +57,9 @@ %% @type option_list(). <p>Options allows to customize the behaviour of the %% XPath scanner. %% </p> +%% <p> %% Possible options are: +%% </p> %% <dl> %% <dt><code>{namespace, #xmlNamespace}</code></dt> %% <dd>Set namespace nodes, from XmlNamspace, in xmlContext</dd> diff --git a/lib/xmerl/src/xmerl_xsd.erl b/lib/xmerl/src/xmerl_xsd.erl index 1aedc9e270..c0923520c2 100644 --- a/lib/xmerl/src/xmerl_xsd.erl +++ b/lib/xmerl/src/xmerl_xsd.erl @@ -29,7 +29,9 @@ %% @type option_list(). <p>Options allow to customize the behaviour of the %% validation. %% </p> +%% <p> %% Possible options are : +%% </p> %% <dl> %% <dt><code>{tab2file,boolean()}</code></dt> %% <dd>Enables saving of abstract structure on file for debugging @@ -46,6 +48,7 @@ %% <dd>It is possible by this option to provide a state with process %% information from an earlier validation.</dd> %% </dl> +%% @end %%%------------------------------------------------------------------- -module(xmerl_xsd). diff --git a/lib/xmerl/vsn.mk b/lib/xmerl/vsn.mk index d85a57f447..a4d7efaee3 100644 --- a/lib/xmerl/vsn.mk +++ b/lib/xmerl/vsn.mk @@ -1 +1 @@ -XMERL_VSN = 1.2.6 +XMERL_VSN = 1.2.7 diff --git a/make/emd2exml.in b/make/emd2exml.in index ea1085cf71..16c38379d9 100644 --- a/make/emd2exml.in +++ b/make/emd2exml.in @@ -381,6 +381,8 @@ put_text(S, [$>|Cs], CTag, EmTag, Acc) when CTag /= no -> put_text(S, Cs, CTag, EmTag, [">"|Acc]); put_text(S, [$&|Cs], CTag, EmTag, Acc) when CTag /= no -> put_text(S, Cs, CTag, EmTag, ["&"|Acc]); +put_text(S, [$&, $ |Cs], CTag, EmTag, Acc) -> %Workaround for INSTALL-WIN32.md + put_text(S, Cs, CTag, EmTag, ["& "|Acc]); put_text(S, [$'|Cs], CTag, EmTag, Acc)when CTag /= no -> put_text(S, Cs, CTag, EmTag, ["'"|Acc]); put_text(S, [$<|Cs], no, EmTag, Acc) -> diff --git a/make/otp.mk.in b/make/otp.mk.in index 6ae7c5b456..bf287be416 100644 --- a/make/otp.mk.in +++ b/make/otp.mk.in @@ -209,6 +209,8 @@ MAN9DIR = $(DOCDIR)/man9 TEXDIR = . +SPECDIR = $(DOCDIR)/specs + # HTML & GIF files that always are generated and must be delivered SGML_COLL_FILES = $(SGML_APPLICATION_FILES) $(SGML_PART_FILES) XML_COLL_FILES = $(XML_APPLICATION_FILES) $(XML_PART_FILES) @@ -237,41 +239,52 @@ FOP = @FOP@ DOCGEN=$(ERL_TOP)/lib/erl_docgen +SPECS_ESRC = ../../src +# Extract specifications and types from Erlang source files (-spec, -type) +$(SPECDIR)/specs_%.xml: $(SPECS_ESRC)/%.erl + escript $(DOCGEN)/priv/bin/specs_gen.escript $(SPECS_FLAGS) -o$(dir $@) $< -$(MAN1DIR)/%.1:: %.xml +$(MAN1DIR)/%.1: %.xml date=`date +"%B %e %Y"`; \ xsltproc --output "$@" --stringparam company "Ericsson AB" --stringparam docgen "$(DOCGEN)" --stringparam gendate "$$date" --stringparam appname "$(APPLICATION)" --stringparam appver "$(VSN)" --xinclude -path $(DOCGEN)/priv/docbuilder_dtd -path $(DOCGEN)/priv/dtd_man_entities $(DOCGEN)/priv/xsl/db_man.xsl $< -$(MAN2DIR)/%.2:: %.xml +$(MAN2DIR)/%.2: %.xml date=`date +"%B %e %Y"`; \ xsltproc --output "$@" --stringparam company "Ericsson AB" --stringparam docgen "$(DOCGEN)" --stringparam gendate "$$date" --stringparam appname "$(APPLICATION)" --stringparam appver "$(VSN)" --xinclude -path $(DOCGEN)/priv/docbuilder_dtd -path $(DOCGEN)/priv/dtd_man_entities $(DOCGEN)/priv/xsl/db_man.xsl $< -$(MAN3DIR)/%.3:: %.xml +ifneq ($(wildcard $(SPECDIR)),) +$(MAN3DIR)/%.3: %.xml $(SPECDIR)/specs_%.xml + date=`date +"%B %e %Y"`; \ + specs_file=`pwd`/$(SPECDIR)/specs_$*.xml; \ + xsltproc --output "$@" --stringparam company "Ericsson AB" --stringparam docgen "$(DOCGEN)" --stringparam gendate "$$date" --stringparam appname "$(APPLICATION)" --stringparam appver "$(VSN)" --stringparam specs_file "$$specs_file" --xinclude -path $(DOCGEN)/priv/docbuilder_dtd -path $(DOCGEN)/priv/dtd_man_entities $(DOCGEN)/priv/xsl/db_man.xsl $< +else +$(MAN3DIR)/%.3: %.xml date=`date +"%B %e %Y"`; \ xsltproc --output "$@" --stringparam company "Ericsson AB" --stringparam docgen "$(DOCGEN)" --stringparam gendate "$$date" --stringparam appname "$(APPLICATION)" --stringparam appver "$(VSN)" --xinclude -path $(DOCGEN)/priv/docbuilder_dtd -path $(DOCGEN)/priv/dtd_man_entities $(DOCGEN)/priv/xsl/db_man.xsl $< +endif # left for compatability -$(MAN4DIR)/%.4:: %.xml +$(MAN4DIR)/%.4: %.xml date=`date +"%B %e %Y"`; \ xsltproc --output "$@" --stringparam company "Ericsson AB" --stringparam docgen "$(DOCGEN)" --stringparam gendate "$$date" --stringparam appname "$(APPLICATION)" --stringparam appver "$(VSN)" --xinclude -path $(DOCGEN)/priv/docbuilder_dtd -path $(DOCGEN)/priv/dtd_man_entities $(DOCGEN)/priv/xsl/db_man.xsl $< -$(MAN4DIR)/%.5:: %.xml +$(MAN4DIR)/%.5: %.xml date=`date +"%B %e %Y"`; \ xsltproc --output "$@" --stringparam company "Ericsson AB" --stringparam docgen "$(DOCGEN)" --stringparam gendate "$$date" --stringparam appname "$(APPLICATION)" --stringparam appver "$(VSN)" --xinclude -path $(DOCGEN)/priv/docbuilder_dtd -path $(DOCGEN)/priv/dtd_man_entities $(DOCGEN)/priv/xsl/db_man.xsl $< # left for compatability -$(MAN6DIR)/%.6:: %_app.xml +$(MAN6DIR)/%.6: %_app.xml date=`date +"%B %e %Y"`; \ xsltproc --output "$@" --stringparam company "Ericsson AB" --stringparam docgen "$(DOCGEN)" --stringparam gendate "$$date" --stringparam appname "$(APPLICATION)" --stringparam appver "$(VSN)" --xinclude -path $(DOCGEN)/priv/docbuilder_dtd -path $(DOCGEN)/priv/dtd_man_entities $(DOCGEN)/priv/xsl/db_man.xsl $< -$(MAN6DIR)/%.7:: %_app.xml +$(MAN6DIR)/%.7: %_app.xml date=`date +"%B %e %Y"`; \ xsltproc --output "$@" --stringparam company "Ericsson AB" --stringparam docgen "$(DOCGEN)" --stringparam gendate "$$date" --stringparam appname "$(APPLICATION)" --stringparam appver "$(VSN)" --xinclude -path $(DOCGEN)/priv/docbuilder_dtd -path $(DOCGEN)/priv/dtd_man_entities $(DOCGEN)/priv/xsl/db_man.xsl $< -$(MAN9DIR)/%.9:: %.xml +$(MAN9DIR)/%.9: %.xml date=`date +"%B %e %Y"`; \ xsltproc --output "$@" --stringparam company "Ericsson AB" --stringparam docgen "$(DOCGEN)" --stringparam gendate "$$date" --stringparam appname "$(APPLICATION)" --stringparam appver "$(VSN)" --xinclude -path $(DOCGEN)/priv/docbuilder_dtd -path $(DOCGEN)/priv/dtd_man_entities $(DOCGEN)/priv/xsl/db_man.xsl $< diff --git a/make/otp_release_targets.mk b/make/otp_release_targets.mk index 7d433e738c..8058e634d4 100644 --- a/make/otp_release_targets.mk +++ b/make/otp_release_targets.mk @@ -21,13 +21,30 @@ # Targets for the new documentation support # ---------------------------------------------------- +ifneq ($(TOP_SPECS_FILE),) +TOP_SPECS_PARAM = --stringparam specs_file "`pwd`/$(TOP_SPECS_FILE)" +endif + +MOD2APP = $(ERL_TOP)/make/$(TARGET)/mod2app.xml +ifneq ($(wildcard $(MOD2APP)),) +MOD2APP_PARAM = --stringparam mod2app_file "$(MOD2APP)" +endif + ifeq ($(TOPDOC),) -$(HTMLDIR)/index.html: $(XML_FILES) +$(HTMLDIR)/index.html: $(XML_FILES) $(SPECS_FILES) date=`date +"%B %e %Y"`; \ - $(XSLTPROC) --noout --stringparam outdir $(HTMLDIR) --stringparam docgen "$(DOCGEN)" --stringparam topdocdir "$(TOPDOCDIR)" \ - --stringparam pdfdir "$(PDFDIR)" \ - --stringparam gendate "$$date" --stringparam appname "$(APPLICATION)" --stringparam appver "$(VSN)" --xinclude \ - -path $(DOCGEN)/priv/docbuilder_dtd -path $(DOCGEN)/priv/dtd_html_entities $(DOCGEN)/priv/xsl/db_html.xsl book.xml + $(XSLTPROC) --noout \ + --stringparam outdir $(HTMLDIR) \ + --stringparam docgen "$(DOCGEN)" \ + --stringparam topdocdir "$(TOPDOCDIR)" \ + --stringparam pdfdir "$(PDFDIR)" \ + --xinclude $(TOP_SPECS_PARAM) $(MOD2APP_PARAM) \ + --stringparam gendate "$$date" \ + --stringparam appname "$(APPLICATION)" \ + --stringparam appver "$(VSN)" \ + -path $(DOCGEN)/priv/docbuilder_dtd \ + -path $(DOCGEN)/priv/dtd_html_entities \ + $(DOCGEN)/priv/xsl/db_html.xsl book.xml endif $(HTMLDIR)/users_guide.html: $(XML_FILES) @@ -37,14 +54,17 @@ $(HTMLDIR)/users_guide.html: $(XML_FILES) --stringparam gendate "$$date" --stringparam appname "$(APPLICATION)" --stringparam appver "$(VSN)" --xinclude \ -path $(DOCGEN)/priv/docbuilder_dtd -path $(DOCGEN)/priv/dtd_html_entities $(DOCGEN)/priv/xsl/db_html.xsl book.xml - -%.fo: $(XML_FILES) +%.fo: $(XML_FILES) $(SPECS_FILES) date=`date +"%B %e %Y"`; \ - $(XSLTPROC) --stringparam docgen "$(DOCGEN)" --stringparam gendate "$$date" --stringparam appname "$(APPLICATION)" \ - --stringparam appver "$(VSN)" --xinclude \ - -path $(DOCGEN)/priv/docbuilder_dtd -path $(DOCGEN)/priv/dtd_html_entities $(DOCGEN)/priv/xsl/db_pdf.xsl book.xml > $@ - - + $(XSLTPROC) \ + --stringparam docgen "$(DOCGEN)" \ + --stringparam gendate "$$date" \ + --stringparam appname "$(APPLICATION)" \ + --stringparam appver "$(VSN)" \ + --xinclude $(TOP_SPECS_PARAM) \ + -path $(DOCGEN)/priv/docbuilder_dtd \ + -path $(DOCGEN)/priv/dtd_html_entities \ + $(DOCGEN)/priv/xsl/db_pdf.xsl book.xml > $@ # ------------------------------------------------------------------------ # The following targets just exist in the documentation directory |