aboutsummaryrefslogtreecommitdiffstats
path: root/erts/emulator/beam/erl_process.c
AgeCommit message (Collapse)Author
2015-05-08Optimized timer implementationRickard Green
2015-03-20Merge branch 'rickard/time_api/OTP-11997'Rickard Green
* rickard/time_api/OTP-11997: (22 commits) Update primary bootstrap inets: Suppress deprecated warning on erlang:now/0 inets: Cleanup of multiple copies of functions Add inets_lib with common functions used by multiple modules inets: Update comments Suppress deprecated warning on erlang:now/0 Use new time API and be back-compatible in inets Remove unused functions and removed redundant test asn1 test SUITE: Eliminate use of now/0 Disable deprecated warning on erlang:now/0 in diameter_lib Use new time API and be back-compatible in ssh Replace all calls to now/0 in CT with new time API functions test_server: Replace usage of erlang:now() with usage of new API Replace usage of erlang:now() with usage of new API Replace usage of erlang:now() with usage of new API Replace usage of erlang:now() with usage of new API Replace usage of erlang:now() with usage of new API otp_SUITE: Warn for calls to erlang:now/0 Replace usage of erlang:now() with usage of new API Multiple timer wheels Erlang based BIF timer implementation for scalability Implement ethread events with timeout ... Conflicts: bootstrap/bin/start.boot bootstrap/bin/start_clean.boot bootstrap/lib/compiler/ebin/beam_asm.beam bootstrap/lib/compiler/ebin/compile.beam bootstrap/lib/kernel/ebin/auth.beam bootstrap/lib/kernel/ebin/dist_util.beam bootstrap/lib/kernel/ebin/global.beam bootstrap/lib/kernel/ebin/hipe_unified_loader.beam bootstrap/lib/kernel/ebin/inet_db.beam bootstrap/lib/kernel/ebin/inet_dns.beam bootstrap/lib/kernel/ebin/inet_res.beam bootstrap/lib/kernel/ebin/os.beam bootstrap/lib/kernel/ebin/pg2.beam bootstrap/lib/stdlib/ebin/dets.beam bootstrap/lib/stdlib/ebin/dets_utils.beam bootstrap/lib/stdlib/ebin/erl_tar.beam bootstrap/lib/stdlib/ebin/escript.beam bootstrap/lib/stdlib/ebin/file_sorter.beam bootstrap/lib/stdlib/ebin/otp_internal.beam bootstrap/lib/stdlib/ebin/qlc.beam bootstrap/lib/stdlib/ebin/random.beam bootstrap/lib/stdlib/ebin/supervisor.beam bootstrap/lib/stdlib/ebin/timer.beam erts/aclocal.m4 erts/emulator/beam/bif.c erts/emulator/beam/erl_bif_info.c erts/emulator/beam/erl_db_hash.c erts/emulator/beam/erl_init.c erts/emulator/beam/erl_process.h erts/emulator/beam/erl_thr_progress.c erts/emulator/beam/utils.c erts/emulator/sys/unix/sys.c erts/preloaded/ebin/erlang.beam erts/preloaded/ebin/erts_internal.beam erts/preloaded/ebin/init.beam erts/preloaded/src/erts_internal.erl lib/common_test/test/ct_hooks_SUITE_data/cth/tests/empty_cth.erl lib/diameter/src/base/diameter_lib.erl lib/kernel/src/os.erl lib/ssh/test/ssh_basic_SUITE.erl system/doc/efficiency_guide/advanced.xml
2015-03-20Multiple timer wheelsRickard Green
2015-03-20Erlang based BIF timer implementation for scalabilityRickard Green
2015-03-20Implement ethread events with timeoutRickard Green
2015-03-20Introduce a new time APIRickard Green
The old time API is based on erlang:now/0. The major issue with erlang:now/0 is that it was intended to be used for so many unrelated things. This tied these unrelated operations together and unnecessarily caused performance, scalability as well as accuracy, and precision issues for operations that do not need to have such issues. The new API spreads different functionality over multiple functions in order to improve on this. The new API consists of a number of new BIFs: - erlang:convert_time_unit/3 - erlang:monotonic_time/0 - erlang:monotonic_time/1 - erlang:system_time/0 - erlang:system_time/1 - erlang:time_offset/0 - erlang:time_offset/1 - erlang:timestamp/0 - erlang:unique_integer/0 - erlang:unique_integer/1 - os:system_time/0 - os:system_time/1 and a number of extensions of existing BIFs: - erlang:monitor(time_offset, clock_service) - erlang:system_flag(time_offset, finalize) - erlang:system_info(os_monotonic_time_source) - erlang:system_info(time_offset) - erlang:system_info(time_warp_mode) - erlang:system_info(time_correction) - erlang:system_info(start_time) See the "Time and Time Correction in Erlang" chapter of the ERTS User's Guide for more information.
2014-12-19Merge branch 'lukas/erts/crashdump_improvements/OTP-12377'Lukas Larsson
* lukas/erts/crashdump_improvements/OTP-12377: erts: Make main thread safe from pipe closed event erts: Improve crash dumps erts: Rename sys_sigset to sys_signal erts: Introduce thread suspend functions erts: Remove usage of QUANTIFY signal erts: Add support for thread names ets: Increase data available in crash dumps and ets:info erts: Start compilation of beam_emu earlier
2014-12-19erts: Improve crash dumpsLukas Larsson
This commit improves crash dumps in several ways: * Suspends schedulers to get a current snapshot * Dumps information about scheduler * Dumps stack trace of current running process (including Garbing processes)
2014-12-19erts: Add support for thread namesLukas Larsson
2014-12-16Merge branch 'sverk/ct-assert'Sverker Eriksson
* sverk/ct-assert: erts: Add compile time assert ERTS_CT_ASSERT
2014-12-10Merge branch 'maint'Rickard Green
* maint: Use the new 64-bit atomic ops API Introduce a 64-bit atomic ops API
2014-12-10Use the new 64-bit atomic ops APIRickard Green
2014-12-02erts: Add compile time assert ERTS_CT_ASSERTSverker Eriksson
and usage
2014-11-25Merge branch 'maint'Rickard Green
* maint: Do not let non-empty run-queue prevent activation of scheduler
2014-11-25Merge branch 'rickard/maint-17/activate-runq/OTP-12287' into maintRickard Green
* rickard/maint-17/activate-runq/OTP-12287: Do not let non-empty run-queue prevent activation of scheduler
2014-11-04Merge branch 'rickard/activate-runq/OTP-12287' into ↵Rickard Green
rickard/maint-17/activate-runq/OTP-12287 * rickard/activate-runq/OTP-12287: Do not let non-empty run-queue prevent activation of scheduler Conflicts: erts/emulator/beam/erl_process.c
2014-11-04Do not let non-empty run-queue prevent activation of schedulerRickard Green
Conflicts: erts/emulator/beam/erl_process.c
2014-09-25Change default to "eager check I/O"Rickard Green
Conflicts: erts/emulator/beam/erl_process.c
2014-09-25No eager check I/O on OSERickard Green
2014-09-25Merge branch 'rickard/eager-check-io/OTP-12117' into ↵Rickard Green
rickard/maint-17/eager-check-io/OTP-12117 * rickard/eager-check-io/OTP-12117: Introduce support for eager check I/O scheduling Conflicts: erts/emulator/beam/erl_bif_info.c erts/emulator/beam/erl_init.c erts/emulator/beam/erl_process.c erts/preloaded/ebin/erlang.beam
2014-09-25Introduce support for eager check I/O schedulingRickard Green
2014-09-05Merge branch 'vinoski/enif-schedule-nif' into maintRickard Green
OTP-12128 * vinoski/enif-schedule-nif: Fix leak of NIF exports Use separate allocation type for NIF export
2014-09-05Fix leak of NIF exportsRickard Green
2014-08-29Merge branch 'rickard/runnable-trace-ooo-bug/OTP-12105' into maintRickard Green
* rickard/runnable-trace-ooo-bug/OTP-12105: Fix busy_port_SUITE:io_to_busy test-case Ensure "runnable port" trace messages are not sent out of order Ensure "runnable proc" trace messages are not sent out of order
2014-08-28add enif_schedule_nif() to NIF APISteve Vinoski
In the #erlang IRC channel Anthony Ramine once mentioned the idea of allowing a NIF to use an emulator trap, similar to a BIF trap, to schedule another NIF for execution. This is exactly how dirty NIFs were implemented for Erlang/OTP 17.0, so this commit refactors and generalizes that dirty NIF code to support a new enif_schedule_nif() API function. The enif_schedule_nif() function allows a long-running NIF to be broken into separate NIF invocations. The NIF first executes part of the long-running task, then calls enif_schedule_nif() to schedule a NIF for later execution to continue the task. Any number of NIFs can be scheduled in this manner, one after another. Since the emulator regains control between invocations, this helps avoid problems caused by native code tying up scheduler threads for too long. The enif_schedule_nif() function also replaces the original experimental dirty NIF API. The function takes a flags parameter that a caller can use to indicate the NIF should be scheduled onto either a dirty CPU scheduler thread, a dirty I/O scheduler thread, or scheduled as a regular NIF on a regular scheduler thread. With this change, the original experimental enif_schedule_dirty_nif(), enif_schedule_dirty_nif_finalizer() and enif_dirty_nif_finalizer() API functions are no longer needed and have been removed. Explicit scheduling of a dirty NIF finalization function is no longer necessary; if an application wants similar functionality, it can have a dirty NIF just invoke enif_schedule_nif() to schedule a non-dirty NIF to complete its task. Lift the restriction that dirty NIFs can't call enif_make_badarg() to raise an exception. This was a problem with the original dirty NIF API because it forced developers to get and check all incoming arguments in a regular NIF, and then schedule the dirty NIF which then had to get all the arguments again. Now, the argument checking can be done in the dirty NIF and it can call enif_make_badarg() itself to flag incorrect arguments. Extend the ErlNifFunc struct with a new flags field that allows NIFs to be declared as dirty. The default value for this field is 0, indicating a regular NIF, so it's backwards compatible with all existing statically initialized ErlNifFunc struct instances, and so such instances require no code changes. Defining the flags field with a value of ERL_NIF_DIRTY_JOB_CPU_BOUND indicates that the NIF should execute on a dirty CPU scheduler thread, or defining it with a value of ERL_NIF_DIRTY_JOB_IO_BOUND indicates that the NIF should execute on a dirty I/O scheduler thread. Any other flags field value causes a NIF library loading error. Extend the ErlNifEntry struct with a new options field that indicates whether a NIF library was built with support for optional features such as dirty NIFs. When a NIF library is loaded, the runtime checks the options field to ensure compatibility. If a NIF library built with dirty NIF support is loaded into a runtime that does not support dirty NIFs, and the library defines one or more ErlNifFunc entries with non-zero flags fields indicating dirty NIFs, a NIF library loading error results. There is no error if a NIF library built with dirty NIF support is loaded into a runtime that does not support dirty NIFs but the library does not have any dirty NIFs. It is also not an error if a library without dirty NIF support is loaded into a runtime built with dirty NIF support. Add documentation and tests for enif_schedule_nif().
2014-08-22Ensure "runnable proc" trace messages are not sent out of orderRickard Green
2014-08-06erts: Print error reason when malloc failsLukas Larsson
2014-05-05Merge branch 'lukas/erts/suspend_assert/OTP-11906' into maintLukas Larsson
* lukas/erts/suspend_assert/OTP-11906: erts: Fix faulty process suspend assert
2014-05-05erts: Fix faulty process suspend assertLukas Larsson
2014-05-05Merge branch 'lukas/erts/etp-lc-macros/OTP-11904' into maintLukas Larsson
* lukas/erts/etp-lc-macros/OTP-11904: erts: Add etp-lc-dump and etp-ppc-stacktrace macro
2014-03-31Merge branch 'sverk/suspend-running-race'Sverker Eriksson
* sverk/suspend-running-race: erts: Fix bug of scheduling a suspended process
2014-03-28erts: Add etp-lc-dump and etp-ppc-stacktrace macroLukas Larsson
2014-03-27erts: Fix bug of scheduling a suspended processSverker Eriksson
This is a race that can cause RUNNING (instead of RUNNING_SYS) set on a SUSPENDED process. The effect of this race happening is probably quite benign. The bug was discovered by process_SUITE:processes_gc_trap on debug VM crashing on last assert in schedule(): /* Never run a suspended process */ ASSERT(!(ERTS_PSFLG_SUSPENDED & erts_smp_atomic32_read_nob(&p->state)));
2014-03-25Merge branch 'paulgray/dtrace_snprintf_fix/OTP-11816'Lukas Larsson
* paulgray/dtrace_snprintf_fix/OTP-11816: Pass full strings to DTrace probes
2014-03-12Pass full strings to DTrace probesMichal Ptaszek
Whenever string is passed as an argument to a DTrace probe, its length should be properly computed. Until now in order to get length of the input buffer size_of(char *) was used - which evalutes to 4 or 8 (depending on the architecture). To get a proper length, size_of(DTRACE_CHARBUF_NAME(buffer_name)) should be used.
2014-02-24Merge branch 'lukas/ose/master/OTP-11334'Lukas Larsson
* lukas/ose/master/OTP-11334: (71 commits) erts: Fix unix efile assert ose: Use -O2 when building ose: Expand OSE docs ose: Add dummy ttsl driver ose: Cleanup cleanup of mutex selection defines ose: Polish mmap configure checks ose: Add ose specific x-compile flags ose: Updating fd_driver and spawn_driver for OSE ose: Updating event and signal API for OSE ose: Cleanup of mutex selection defines win32: Compile erl_log.exe ose: Remove uneccesary define ose: Fix ssl configure test for osx erts: Fix sys_msg_dispatcher assert ose: Fix broken doc links ose: Thread priorities configurable from lmconf ose: Yielding the cpu is done "the OSE" way ose: Start using ppdata for tse key ose: Do not use spinlocks on OSE ose: Fix support for crypto ... Conflicts: lib/crypto/c_src/crypto.c
2014-02-24ose: Thread priorities configurable from lmconfLukas Larsson
The pattern used for getting the priority from the lmconf is based on the name of the process created. The pattern is: ERTS_%%PROCESS_NAME%%_PRIO with the %%PROCESS_NAME%% replaced by the prefix of the process the priority applies to. eg: ERTS_SCHEDULER_PRIO=24 applies to processes with name SCHEDULER_1, SCHEDULER_2 etc.
2014-02-24ose: Cleanup POLL_SCHED_1 codeLukas Larsson
Now schedulers 2..N make sure to wake sched 1 if they find that all io has been consumed and sched 1 is sleeping. Before sched 1 was spinning in sys_schedule waiting for sched 2..N to finish consuming io jobs
2014-02-24ose: Add fair scheduling yieldsLukas Larsson
This is needed on OSs that do not do round robin scheduling of threads.
2014-02-24ose,erts: Specify name for tsd keysLukas Larsson
This simplified debugging on OSE and also limits the number of ppdata keys that are created when beam is restarted.
2014-02-24Added support for ENEA OSELukas Larsson
This port has support for both non-smp and smp. It contains a new way to do io checking in which erts_poll_wait receives the payload of the polled entity. This has implications for all linked-in drivers.
2014-02-24further enhancements for dirty schedulersSteve Vinoski
Add support for setting the number of dirty CPU schedulers online via erlang:system_flag/2. Assuming the emulator is built with dirty schedulers enabled, the number of dirty CPU schedulers online may not be less than 1, nor greater than the number of dirty CPU schedulers available, nor greater than the number of normal schedulers online. Dirty CPU scheduler threads that are taken offline via system_flag/2 are suspended. The number of dirty CPU schedulers online may be adjusted independently of the number of normal schedulers online, but if system_flag/2 is used to set the number of normal schedulers online to a value less than the current number of normal schedulers online, the number of dirty CPU schedulers online is decreased proportionally. Likewise, if the number of normal schedulers online is increased, the number of dirty CPU schedulers online is increased proportionally. For example, if 8 normal schedulers and 4 dirty CPU schedulers are online, and system_flag/2 is called to set the number of normal schedulers online to 4, the number of dirty CPU schedulers online is also decreased by half, to 2. Subsequently setting the number of normal schedulers online back to 8 also sets the number of dirty CPU schedulers online back to 4. Augment the system_flag/2 documentation in the erlang man page to explain this relationship between schedulers_online and dirty_cpu_schedulers_online. Also ensure that all dirty CPU and I/O schedulers are suspended when multi-scheduling is blocked via system_flag/2, and brought back online when multi-scheduling is unblocked. Add Rickard Green's rewritten check_enqueue_in_prio_queue() function that inspects process state more thoroughly to determine if to enqueue it and if so on what queue, including dirty queues when appropriate. Make sure dirty NIF jobs do not trigger erlang:system_monitor long_schedule messages. Add more dirty scheduler testing to the scheduler test suite. Remove the erts_no_dirty_cpu_schedulers_online global variable, since it's no longer needed. Execute dirty NIFs on a normal scheduler thread while multi-scheduling blocking is in effect. Evacuate any dirty jobs residing in the dirty run queues over to a normal run queue when multi-scheduling is blocked. Allow dirty schedulers to execute aux work. Set the dirty run queues halt_in_progress flag when halting the normal schedulers. Change dirty scheduler numbers to a structure including both scheduler number and type, either dirty CPU or dirty I/O. Add some assertions to ensure that dirty CPU schedulers operate only on dirty CPU scheduler process flags, and the same for dirty I/O schedulers.
2014-01-28initial support for dirty schedulers and dirty NIFsSteve Vinoski
Add initial support for dirty schedulers. There are two types of dirty schedulers: CPU schedulers and I/O schedulers. By default, there are as many dirty CPU schedulers as there are normal schedulers and as many dirty CPU schedulers online as normal schedulers online. There are 10 dirty I/O schedulers (similar to the choice of 10 as the default for async threads). By default, dirty schedulers are disabled and conditionally compiled out. To enable them, you must pass --enable-dirty-schedulers to the top-level configure script when building Erlang/OTP. Current dirty scheduler support requires the emulator to be built with SMP support. This restriction will be lifted in the future. You can specify the number of dirty schedulers with the command-line options +SDcpu (for dirty CPU schedulers) and +SDio (for dirty I/O schedulers). The +SDcpu option is similar to the +S option in that it takes two numbers separated by a colon: C1:C2, where C1 specifies the number of dirty schedulers available and C2 specifies the number of dirty schedulers online. The +SDPcpu option allows numbers of dirty CPU schedulers available and dirty CPU schedulers online to be specified as percentages, similar to the existing +SP option for normal schedulers. The number of dirty CPU schedulers created and dirty CPU schedulers online may not exceed the number of normal schedulers created and normal schedulers online, respectively. The +SDio option takes only a single number specifying the number of dirty I/O schedulers available and online. There is no support yet for programmatically changing at run time the number of dirty CPU schedulers online via erlang:system_flag/2. Also, changing the number of normal schedulers online via erlang:system_flag(schedulers_online, NewSchedulersOnline) should ensure that there are no more dirty CPU schedulers than normal schedulers, but this is not yet implemented. You can retrieve the number of dirty schedulers by passing dirty_cpu_schedulers, dirty_cpu_schedulers_online, or dirty_io_schedulers to erlang:system_info/1. Currently only NIFs are able to access dirty scheduler functionality. Neither drivers nor BIFs currently support dirty schedulers. This restriction will be addressed in the future. If dirty scheduler support is present in the runtime, the initial status line Erlang prints before presenting its interactive prompt will include the indicator "[ds:C1:C2:I]" where "ds" indicates "dirty schedulers", "C1" indicates the number of dirty CPU schedulers available, "C2" indicates the number of dirty CPU schedulers online, and "I" indicates the number of dirty I/O schedulers. Document The dirty NIF API in the erl_nif man page. The API closely follows Rickard Green's presentation slides from his talk "Future Extensions to the Native Interface", presented at the 2011 Erlang Factory held in the San Francisco Bay Area. Rickard's slides are available online at http://bit.ly/1m34UHB . Document the new erl command-line options, the additions to erlang:system_info/1, and also add the erlang:system_flag/2 dirty scheduler documentation even though it's not yet implemented. To determine whether the dirty NIF API is available, native code can check to see whether the C preprocessor macro ERL_NIF_DIRTY_SCHEDULER_SUPPORT is defined. To check if dirty schedulers are available at run time, native code can call the boolean enif_have_dirty_schedulers() function, and Erlang code can call erlang:system_info(dirty_cpu_schedulers), which raises badarg if no dirty scheduler support is available. Add a simple dirty NIF test to the emulator NIF suite.
2014-01-27Merge branch 'rickard/load_balance/OTP-11385'Rickard Green
* rickard/load_balance/OTP-11385: Fix usage of non-empty run-queue flag
2014-01-27Fix usage of non-empty run-queue flagRickard Green
2014-01-24Merge branch 'rickard/load_balance/OTP-11385'Rickard Green
* rickard/load_balance/OTP-11385: Add support for scheduler utilization balancing
2014-01-23Add support for scheduler utilization balancingRickard Green
For more information see documentation of the new command line argument +sub
2014-01-10erts: Remove the extra_root feature from the process structureSverker Eriksson
as we don't use it and instead have the feature to disable GC during trapping BIFs.
2013-12-12Merge branch 'sverk/trapping-bin2term' into OTP_R16B03Sverker Eriksson
* sverk/trapping-bin2term: erts: Adjust term_to_binary reduction factors erts: Yield after trapping term_to_binary if gc has been ordered erts: Let term_to_binary disable gc while trapping erts: Improve stress of binary_to_term in binary_SUITE erts: Fix bug in binary_to_term for compressed on halfword erts: Fix crash when binary_to_term throws badarg erts: Trapping memcpy in binary_to_term erts: Cleanup code for trapping binary_to_term erts: Add erlang wrappers to binary_to_term trapping uncompress trapping size calculation trapping binary_to_term/2 trapping STRING_EXT trapping lists and tuples trapping binary_to_term passing binary_SUITE Parallel check_process_code when code_server purge a module Functionality for disabling garbage collection Use asynchronous check_process_code in code_parallel_SUITE Execution of system tasks in context of another process Conflicts: erts/emulator/beam/external.c erts/emulator/beam/sys.h erts/emulator/test/binary_SUITE.erl erts/preloaded/ebin/erlang.beam erts/preloaded/ebin/erts_internal.beam
2013-11-21erts: Yield after trapping term_to_binary if gc has been orderedSverker Eriksson
or if "too much" offheap binaries has been built
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-27 22:49:45 +0000