aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator
AgeCommit message (Collapse)Author
2013-01-10Merge branch 'fdm/fix_fd_leak_async_thread_pool/OTP-10677'Björn-Egil Dahlberg
* fdm/fix_fd_leak_async_thread_pool/OTP-10677: Fix fd leak when using async thread pool
2013-01-09Fix fd leak when using async thread poolFilipe David Borba Manana
When using the async thread pool, if an erlang process asks to open a file and it gets shutdown/killed while the file:open/2 call hasn't returned, it's possible to leak a file descriptor against the target file. This happens because when the file driver is stopped (file_stop() function is called), an async thread is executing, about to execute, or executed already the invoke_open() function. After file_stop() is called, the file_async_ready() function will not run, and this function is responsible for setting desc->fd with the file descriptor that invoke_open() got. The file_stop() call closes desc->fd if it refers to a valid file descriptor, which is not the case here, because this function was called before file_async_ready() could run. This leak is easily reproducile in a GNU/Linux system using the following test code: -module(t). -export([t/1]). t(N) -> Pid = spawn_link(fun() -> process_flag(trap_exit, true), loop(N) end), Ref = erlang:monitor(process, Pid), receive {'DOWN', Ref, _, _, _} -> ok end. loop(0) -> ok; loop(N) -> Name = integer_to_list(N), Server = self(), Pid = spawn(fun() -> Server ! continue, {ok, FdW} = file:open(Name, [raw, write]), {ok, FdR} = file:open(Name, [raw, read]), % Optional close calls, with or without them % it makes no difference. %ok = file:close(FdW), %ok = file:close(FdR), ok end), receive continue -> ok end, exit(Pid, shutdown), loop(N - 1). Running this code with a few iterations is enough to very often notice, with the lsof command, that the beam.smp process is holding forever file descriptors open. This issue doesn't happen if the async thread pool is not used. Example: $ erl +A 4 Erlang R15B03 (erts-5.9.3) [source] [64-bit] [smp:4:4] [async-threads:4] [hipe] [kernel-poll:false] Eshell V5.9.3 (abort with ^G) 1> c(t). {ok,t} 2> os:getpid(). "31975" 3> t:t(20). ok In a separate shell: $ lsof -p 31975 COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME beam.smp 31975 fdmanana cwd DIR 8,18 22736896 32563204 /home/fdmanana/git/hub/otp/tmp beam.smp 31975 fdmanana rtd DIR 8,1 4096 2 / beam.smp 31975 fdmanana txt REG 8,1 7600263 1835126 /opt/r15b03/lib/erlang/erts-5.9.3/bin/beam.smp beam.smp 31975 fdmanana mem REG 8,1 7220736 2497283 /usr/lib/locale/locale-archive beam.smp 31975 fdmanana mem REG 8,1 10280 2505021 /usr/lib/libsctp.so.1.0.11 beam.smp 31975 fdmanana mem REG 8,1 1811128 917795 /lib/x86_64-linux-gnu/libc-2.15.so beam.smp 31975 fdmanana mem REG 8,1 31752 917803 /lib/x86_64-linux-gnu/librt-2.15.so beam.smp 31975 fdmanana mem REG 8,1 135366 917799 /lib/x86_64-linux-gnu/libpthread-2.15.so beam.smp 31975 fdmanana mem REG 8,1 159200 921249 /lib/x86_64-linux-gnu/libtinfo.so.5.9 beam.smp 31975 fdmanana mem REG 8,1 1030512 917962 /lib/x86_64-linux-gnu/libm-2.15.so beam.smp 31975 fdmanana mem REG 8,1 14768 917702 /lib/x86_64-linux-gnu/libdl-2.15.so beam.smp 31975 fdmanana mem REG 8,1 149280 917974 /lib/x86_64-linux-gnu/ld-2.15.so beam.smp 31975 fdmanana 0u CHR 136,1 4 /dev/pts/1 beam.smp 31975 fdmanana 1u CHR 136,1 4 /dev/pts/1 beam.smp 31975 fdmanana 2u CHR 136,1 4 /dev/pts/1 beam.smp 31975 fdmanana 3r FIFO 0,8 1298297 pipe beam.smp 31975 fdmanana 4w FIFO 0,8 1298297 pipe beam.smp 31975 fdmanana 5r FIFO 0,8 1298298 pipe beam.smp 31975 fdmanana 6w FIFO 0,8 1298298 pipe beam.smp 31975 fdmanana 7w REG 8,18 0 32564173 /home/fdmanana/git/hub/otp/tmp/20 beam.smp 31975 fdmanana 8w REG 8,18 0 32564176 /home/fdmanana/git/hub/otp/tmp/16 beam.smp 31975 fdmanana 9w REG 8,18 0 32564177 /home/fdmanana/git/hub/otp/tmp/15 beam.smp 31975 fdmanana 10w REG 8,18 0 32564179 /home/fdmanana/git/hub/otp/tmp/12 beam.smp 31975 fdmanana 11w REG 8,18 0 32564180 /home/fdmanana/git/hub/otp/tmp/11 beam.smp 31975 fdmanana 12w REG 8,18 0 32564205 /home/fdmanana/git/hub/otp/tmp/10 beam.smp 31975 fdmanana 13w REG 8,18 0 32564182 /home/fdmanana/git/hub/otp/tmp/8 beam.smp 31975 fdmanana 14w REG 8,18 0 32564183 /home/fdmanana/git/hub/otp/tmp/7 beam.smp 31975 fdmanana 15w REG 8,18 0 32564186 /home/fdmanana/git/hub/otp/tmp/3
2013-01-09erts: Remove the packages aliases for BIFsBjörn Gustavsson
2013-01-08Merge branch 'sverk/hipe-smp-independence'Sverker Eriksson
* sverk/hipe-smp-independence: erts,hipe: Make hipe compiler ask running emulator about process struct offsets
2012-12-20Teach VM not to dump core on faulty seq_trace in msPatrik Nyblom
2012-12-20Fix emulator +fna optionPatrik Nyblom
2012-12-20Add small patch for debugging of dist messagesPatrik Nyblom
Thanks to Ádám Gólya <[email protected]>
2012-12-20erts,hipe: Make hipe compiler ask running emulator about process struct offsetsSverker Eriksson
Changes in "struct process" has made native code more depenedent on if the executing emulator is smp enabled or not. This commit will make a default built hipe compiler (without -xcomp) to ask the running emulator about process struct offsets. This will make native code work (again) as long as the same emulator is used for compilation as well as execution of the native code.
2012-12-18Merge branch 'rickard/r16/port-optimizations/OTP-10336'Rickard Green
* rickard/r16/port-optimizations/OTP-10336: Fix driver_monitor_process() ASSERT Fix scheduled port link operation Fix aborts of port tasks when terminating ports
2012-12-17Fix driver_monitor_process() ASSERTRickard Green
2012-12-14tests: Update tuple_SUITE for insert and deleteBjörn-Egil Dahlberg
* Test erlang:insert_element/3 BIF * Test erlang:delete_element/2 BIF
2012-12-14Add insert_element/3 and delete_element/2Björn-Egil Dahlberg
* erlang:insert_element/3 - extend a tuple at an index * erlang:delete_element/2 - remove an element at an index
2012-12-14Merge branch 'sverk/egil/r16/new-alloc-header-scheme/OTP-10273'Björn-Egil Dahlberg
OTP-10415 * sverk/egil/r16/new-alloc-header-scheme/OTP-10273: (42 commits) erts: Make ll main mbc fit into 2pow size erts: Clear entire mseg cache upon request erts: Reduce max heap sizes tests: Refactor away ?line macro in beam_SUITE tests: Fix heap_sizes check tests: Refactor away ?line macro in process_SUITE tests: Use new correct min_bin_vheap_size in test erts: Set super alignment (256kb) and limits for sbct (8Mb) and lmbcs (128Mb) erts: Do not cache segments that are misaligned erts: Add mseg cache for large sbc segments erts: Reintroduce mseg options amcbf and rmcbf erts: Optimize erl_alloc_util.c by substitute MBC_BLK_SZ erts: Fix bug when allocating size near sbc_threshold erts: Make gc sizes fit into MB Carrier blocks erts: Force sbmbc to be disabled in a crude way erts: Fix new header scheme for win64 erts: Fix mseg cache. Malplaced NULL pointer erts: Remove unused mseg options amcbf and rmcbf erts: Use aligned bits as constant in mseg_alloc erts: Don't let zero be considered a power of two ... Conflicts: erts/emulator/test/process_SUITE.erl
2012-12-14erts: Make ll main mbc fit into 2pow sizeBjörn-Egil Dahlberg
2012-12-14erts: Clear entire mseg cache upon requestBjörn-Egil Dahlberg
2012-12-14erts: Reduce max heap sizesBjörn-Egil Dahlberg
Reduces max heap sizes to max addressable memory space. In reality this could be even less since we need at least twice as much adressable memory to do a garbage collection. The rest of the system also uses memory thus further constraining heap memory space and in the 64 bit case we really only have 48 bits mappable memory.
2012-12-14tests: Refactor away ?line macro in beam_SUITEBjörn-Egil Dahlberg
2012-12-14tests: Fix heap_sizes checkBjörn-Egil Dahlberg
2012-12-14tests: Refactor away ?line macro in process_SUITEBjörn-Egil Dahlberg
2012-12-14tests: Use new correct min_bin_vheap_size in testBjörn-Egil Dahlberg
2012-12-14erts: Set super alignment (256kb) and limits for sbct (8Mb) and lmbcs (128Mb)Sverker Eriksson
2012-12-14erts: Do not cache segments that are misalignedBjörn-Egil Dahlberg
* SBC may realloc carriers to misaligned addresses which is perfectly fine. However, those segments may not be cached because MBC allocations might find them and MBC's *needs* correct alignment.
2012-12-14erts: Add mseg cache for large sbc segmentsBjörn-Egil Dahlberg
* Not a power of two (unpowered) segements
2012-12-14erts: Reintroduce mseg options amcbf and rmcbfBjörn-Egil Dahlberg
Used with new sbc cache
2012-12-14erts: Optimize erl_alloc_util.c by substitute MBC_BLK_SZSverker Eriksson
with either MBC_ABLK_SZ or MBC_FBLK_SZ in all cases when we already know what kind of block it is.
2012-12-14erts: Fix bug when allocating size near sbc_thresholdSverker Eriksson
A block larger than sbc_threshold can be allocated in MBC if the subsequent "residue block" is smaller than min_block_size. Solved by lowering sbc_threshold to ("hard limit" - min_block_size).
2012-12-14erts: Make gc sizes fit into MB Carrier blocksBjörn-Egil Dahlberg
* Account for block header size in gc-sizes * Also slow down growth to 20% instead of 25% when size threshold is reached
2012-12-14erts: Force sbmbc to be disabled in a crude waySverker Eriksson
2012-12-14erts: Fix new header scheme for win64Sverker Eriksson
2012-12-14erts: Fix mseg cache. Malplaced NULL pointerBjörn-Egil Dahlberg
2012-12-14erts: Remove unused mseg options amcbf and rmcbfBjörn-Egil Dahlberg
2012-12-14erts: Use aligned bits as constant in mseg_allocBjörn-Egil Dahlberg
HAVE_SUPER_ALIGNED_MB_CARRIERS is always true with mmap and thus aligned bits is a constant and so is "page" size for mmap. Conflicts: erts/emulator/sys/common/erl_mseg.h
2012-12-14Fix scheduled port link operationRickard Green
2012-12-14Fix aborts of port tasks when terminating portsRickard Green
2012-12-12Merge branch 'egil/enforce-tuple-specification-size/OTP-10633'Björn-Egil Dahlberg
* egil/enforce-tuple-specification-size/OTP-10633: erts: Use memcpy instead of while in setelement/3 test: Refactor away ?line macro in tuple_SUITE erts: Enforce tuple max size on BIFs erts: Define max tuple size to 24 bits
2012-12-11Fix bit set op of non-atomic fallback in atomic APIRickard Green
2012-12-11Fix debug build on threads disabled buildRickard Green
2012-12-10Fix thread progress management of unmanaged threadsRickard Green
2012-12-07Merge branch 'rickard/port-optimizations/OTP-10336' into ↵Rickard Green
rickard/r16/port-optimizations/OTP-10336 * rickard/port-optimizations/OTP-10336: Change annotate level for emacs-22 in cerl Update etp-commands Add documentation on communication in Erlang Add support for busy port message queue Add driver callback epilogue Implement true asynchronous signaling between processes and ports Add erl_drv_[send|output]_term Move busy port flag Use rwlock for driver list Optimize management of port tasks Improve configuration of process and port tables Remove R9 compatibility features Use ptab functionality also for ports Prepare for use of ptab functionality also for ports Atomic port state Generalize process table implementation Implement functionality for delaying thread progress from unmanaged threads Conflicts: erts/doc/src/erl_driver.xml erts/doc/src/erlang.xml erts/emulator/beam/beam_bif_load.c erts/emulator/beam/beam_bp.c erts/emulator/beam/beam_emu.c erts/emulator/beam/bif.c erts/emulator/beam/copy.c erts/emulator/beam/erl_alloc.c erts/emulator/beam/erl_alloc.types erts/emulator/beam/erl_bif_info.c erts/emulator/beam/erl_bif_port.c erts/emulator/beam/erl_bif_trace.c erts/emulator/beam/erl_init.c erts/emulator/beam/erl_message.c erts/emulator/beam/erl_port_task.c erts/emulator/beam/erl_process.c erts/emulator/beam/erl_process.h erts/emulator/beam/erl_process_lock.c erts/emulator/beam/erl_trace.c erts/emulator/beam/export.h erts/emulator/beam/global.h erts/emulator/beam/io.c erts/emulator/sys/unix/sys.c erts/emulator/sys/vxworks/sys.c erts/emulator/test/port_SUITE.erl erts/etc/unix/cerl.src erts/preloaded/ebin/erlang.beam erts/preloaded/ebin/prim_inet.beam erts/preloaded/src/prim_inet.erl lib/hipe/cerl/erl_bif_types.erl lib/kernel/doc/src/inet.xml lib/kernel/src/inet.erl
2012-12-07Add support for busy port message queueRickard Green
2012-12-07Add driver callback epilogueRickard Green
2012-12-07Implement true asynchronous signaling between processes and portsRickard Green
2012-12-06Merge branch 'maint'Ingela Anderton Andin
Conflicts: erts/emulator/sys/vxworks/sys.c erts/vsn.mk lib/ssl/src/ssl_connection.erl lib/ssl/test/ssl_basic_SUITE.erl
2012-12-06Merge branch 'bjorn/remove-tuple-funs/OTP-10170'Björn Gustavsson
* bjorn/remove-tuple-funs/OTP-10170: erl_expand_records: Remove stale support for literal tuple funs Teach is_function/2 that tuples are not funs
2012-12-05Merge branch 'sverk/bs_construct_SUITE-outofmem'Sverker Eriksson
* sverk/bs_construct_SUITE-outofmem: erts: Limit memory usage of bs_construct_SUITE:huge_binary
2012-12-04Merge branch 'ta/docsmaster'Henrik Nord
* ta/docsmaster: Fix various doc typos for R16 Fix various typos in HOWTO/ for R16 Fix various comment typos for R16 OTP-10611
2012-12-03Add erl_drv_[send|output]_termRickard Green
2012-12-03Move busy port flagRickard Green
2012-12-03Use rwlock for driver listRickard Green
Conflicts: erts/emulator/beam/io.c
2012-12-03Optimize management of port tasksRickard Green
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-07 22:07:00 +0000