From 84adefa331c4159d432d22840663c38f155cd4c1 Mon Sep 17 00:00:00 2001 From: Erlang/OTP Date: Fri, 20 Nov 2009 14:54:40 +0000 Subject: The R13B03 release. --- erts/doc/src/erts_alloc.xml | 554 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 554 insertions(+) create mode 100644 erts/doc/src/erts_alloc.xml (limited to 'erts/doc/src/erts_alloc.xml') diff --git a/erts/doc/src/erts_alloc.xml b/erts/doc/src/erts_alloc.xml new file mode 100644 index 0000000000..d51e5b3ea4 --- /dev/null +++ b/erts/doc/src/erts_alloc.xml @@ -0,0 +1,554 @@ + + + + +
+ + 20022009 + Ericsson AB. All Rights Reserved. + + + The contents of this file are subject to the Erlang Public License, + Version 1.1, (the "License"); you may not use this file except in + compliance with the License. You should have received a copy of the + Erlang Public License along with this software. If not, it can be + retrieved online at http://www.erlang.org/. + + Software distributed under the License is distributed on an "AS IS" + basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See + the License for the specific language governing rights and limitations + under the License. + + + + erts_alloc + Rickard Green + 1 + 03-06-11 + 1 + erts_alloc.xml +
+ erts_alloc + An Erlang Run-Time System internal memory allocator library. + +

erts_alloc is an Erlang Run-Time System internal memory + allocator library. erts_alloc provides the Erlang + Run-Time System with a number of memory allocators.

+ + +
+ Allocators + +

Currently the following allocators are present:

+ + temp_alloc + Allocator used for temporary allocations. + eheap_alloc + Allocator used for Erlang heap data, such as Erlang process heaps. + binary_alloc + Allocator used for Erlang binary data. + ets_alloc + Allocator used for ETS data. + driver_alloc + Allocator used for driver data. + sl_alloc + Allocator used for memory blocks that are expected to be + short-lived. + ll_alloc + Allocator used for memory blocks that are expected to be + long-lived, for example Erlang code. + fix_alloc + A very fast allocator used for some fix-sized + data. fix_alloc manages a set of memory pools from + which memory blocks are handed out. fix_alloc + allocates memory pools from ll_alloc. Memory pools + that have been allocated are never deallocated. + std_alloc + Allocator used for most memory blocks not allocated via any of + the other allocators described above. + sys_alloc + This is normally the default malloc implementation + used on the specific OS. + mseg_alloc + A memory segment allocator. mseg_alloc is used by other + allocators for allocating memory segments and is currently only + available on systems that have the mmap system + call. Memory segments that are deallocated are kept for a + while in a segment cache before they are destroyed. When + segments are allocated, cached segments are used if possible + instead of creating new segments. This in order to reduce + the number of system calls made. + +

sys_alloc and fix_alloc are always enabled and + cannot be disabled. mseg_alloc is always enabled if it is + available and an allocator that uses it is enabled. All other + allocators can be enabled or disabled. + By default all allocators are enabled. + When an allocator is disabled, sys_alloc + is used instead of the disabled allocator.

+

The main idea with the erts_alloc library is to separate + memory blocks that are used differently into different memory + areas, and by this achieving less memory fragmentation. By + putting less effort in finding a good fit for memory blocks that + are frequently allocated than for those less frequently + allocated, a performance gain can be achieved.

+
+ +
+ + The alloc_util framework +

Internally a framework called alloc_util is used for + implementing allocators. sys_alloc, fix_alloc, and + mseg_alloc do not use this framework; hence, the + following does not apply to them.

+

An allocator manages multiple areas, called carriers, in which + memory blocks are placed. A carrier is either placed in a + separate memory segment (allocated via mseg_alloc) or in + the heap segment (allocated via sys_alloc). Multiblock + carriers are used for storage of several blocks. Singleblock + carriers are used for storage of one block. Blocks that are + larger than the value of the singleblock carrier threshold + (sbct) parameter are placed + in singleblock carriers. Blocks smaller than the value of the + sbct parameter are placed in multiblock + carriers. Normally an allocator creates a "main multiblock + carrier". Main multiblock carriers are never deallocated. The + size of the main multiblock carrier is determined by the value + of the mmbcs parameter.

+

+ + Sizes of multiblock carriers allocated via mseg_alloc are + decided based on the values of the largest multiblock carrier + size (lmbcs), the smallest + multiblock carrier size (smbcs), + and the multiblock carrier growth stages + (mbcgs) parameters. If + nc is the current number of multiblock carriers (the main + multiblock carrier excluded) managed by an allocator, the size + of the next mseg_alloc multiblock carrier allocated by + this allocator will roughly be + when + , + and lmbcs when mbcgs]]>. If the value of the + sbct parameter should be larger than the value of the + lmbcs parameter, the allocator may have to create + multiblock carriers that are larger than the value of the + lmbcs parameter, though. Singleblock carriers allocated + via mseg_alloc are sized to whole pages.

+

Sizes of carriers allocated via sys_alloc are + decided based on the value of the sys_alloc carrier size + (ycs) parameter. The size of + a carrier is the least number of multiples of the value of the + ycs parameter that satisfies the request.

+

Coalescing of free blocks are always performed immediately. + Boundary tags (headers and footers) in free blocks are used + which makes the time complexity for coalescing constant.

+

+ + The memory allocation strategy used for multiblock carriers by an + allocator is configurable via the as + parameter. Currently the following strategies are available:

+ + Best fit + +

Strategy: Find the smallest block that satisfies the + requested block size.

+

Implementation: A balanced binary search tree is + used. The time complexity is proportional to log N, where + N is the number of sizes of free blocks.

+ + Address order best fit + +

Strategy: Find the smallest block that satisfies the + requested block size. If multiple blocks are found, choose + the one with the lowest address.

+

Implementation: A balanced binary search tree is + used. The time complexity is proportional to log N, where + N is the number of free blocks.

+ + Good fit + +

Strategy: Try to find the best fit, but settle for the best fit + found during a limited search.

+

Implementation: The implementation uses segregated free + lists with a maximum block search depth (in each list) in + order to find a good fit fast. When the maximum block + search depth is small (by default 3) this implementation + has a time complexity that is constant. The maximum block + search depth is configurable via the + mbsd parameter.

+ + A fit + +

Strategy: Do not search for a fit, inspect only one free + block to see if it satisfies the request. This strategy is + only intended to be used for temporary allocations.

+

Implementation: Inspect the first block in a free-list. + If it satisfies the request, it is used; otherwise, a new + carrier is created. The implementation has a time + complexity that is constant.

+

As of erts version 5.6.1 the emulator will refuse to + use this strategy on other allocators than temp_alloc. + This since it will only cause problems for other allocators.

+ + +
+ +
+ + System Flags Effecting erts_alloc + +

Only use these flags if you are absolutely sure what you are + doing. Unsuitable settings may cause serious performance + degradation and even a system crash at any time during + operation.

+ +

Memory allocator system flags have the following syntax: +

]]> + where ]]> is a letter identifying a subsystem, + ]]> is a parameter, and ]]> is the + value to use. The flags can be passed to the Erlang emulator + (erl) as command line + arguments.

+

System flags effecting specific allocators have an upper-case + letter as ]]>. The following letters are used for + the currently present allocators:

+ + B: binary_alloc + D: std_alloc + E: ets_alloc + F: fix_alloc + H: eheap_alloc + L: ll_alloc + M: mseg_alloc + R: driver_alloc + S: sl_alloc + T: temp_alloc + Y: sys_alloc + +

The following flags are available for configuration of + mseg_alloc:

+ + ]]> + + + Absolute max cache bad fit (in kilobytes). A segment in the + memory segment cache is not reused if its size exceeds the + requested size with more than the value of this + parameter. Default value is 4096. + ]]> + + + Relative max cache bad fit (in percent). A segment in the + memory segment cache is not reused if its size exceeds the + requested size with more than relative max cache bad fit + percent of the requested size. Default value is 20. + ]]> + + + Max cached segments. The maximum number of memory segments + stored in the memory segment cache. Valid range is + 0-30. Default value is 5. + ]]> + + + Cache check interval (in milliseconds). The memory segment + cache is checked for segments to destroy at an interval + determined by this parameter. Default value is 1000. + +

The following flags are available for configuration of + fix_alloc:

+ + +MFe true + + + Enable fix_alloc. Note: fix_alloc cannot be disabled. + +

The following flags are available for configuration of + sys_alloc:

+ + +MYe true + + + Enable sys_alloc. Note: sys_alloc cannot be disabled. + +MYm libc + +malloc library to use. Currently only + libc is available. libc enables the standard + libc malloc implementation. By default libc is used. + ]]> + + + Trim threshold size (in kilobytes). This is the maximum amount + of free memory at the top of the heap (allocated by + sbrk) that will be kept by malloc (not + released to the operating system). When the amount of free + memory at the top of the heap exceeds the trim threshold, + malloc will release it (by calling + sbrk). Trim threshold is given in kilobytes. Default + trim threshold is 128. Note: This flag will + only have any effect when the emulator has been linked with + the GNU C library, and uses its malloc implementation. + ]]> + + + Top pad size (in kilobytes). This is the amount of extra + memory that will be allocated by malloc when + sbrk is called to get more memory from the operating + system. Default top pad size is 0. Note: This flag + will only have any effect when the emulator has been linked + with the GNU C library, and uses its malloc + implementation. + +

The following flags are available for configuration of allocators + based on alloc_util. If u is used as subsystem + identifier (i.e., = u]]>) all allocators based on + alloc_util will be effected. If B, D, E, + H, L, R, S, or T is used as + subsystem identifier, only the specific allocator identified will be + effected:

+ + as bf|aobf|gf|af]]> + + + Allocation strategy. Valid strategies are bf (best fit), + aobf (address order best fit), gf (good fit), + and af (a fit). See + the description of allocation strategies in "the alloc_util framework" section. + asbcst ]]> + + + Absolute singleblock carrier shrink threshold (in + kilobytes). When a block located in an + mseg_alloc singleblock carrier is shrunk, the carrier + will be left unchanged if the amount of unused memory is less + than this threshold; otherwise, the carrier will be shrunk. + See also rsbcst. + e true|false]]> + + + Enable allocator ]]>. + lmbcs ]]> + + + Largest (mseg_alloc) multiblock carrier size (in + kilobytes). See the description + on how sizes for mseg_alloc multiblock carriers are decided + in "the alloc_util framework" section. + mbcgs ]]> + + + (mseg_alloc) multiblock carrier growth stages. See + the description on how sizes for + mseg_alloc multiblock carriers are decided + in "the alloc_util framework" section. + mbsd ]]> + + + Max block search depth. This flag has effect only if the + good fit strategy has been selected for allocator + ]]>. When the good fit strategy is used, free + blocks are placed in segregated free-lists. Each free list + contains blocks of sizes in a specific range. The max block + search depth sets a limit on the maximum number of blocks to + inspect in a free list during a search for suitable block + satisfying the request. + mmbcs ]]> + + + Main multiblock carrier size. Sets the size of the main + multiblock carrier for allocator ]]>. The main + multiblock carrier is allocated via and is + never deallocated. + mmmbc ]]> + + + Max mseg_alloc multiblock carriers. Maximum number of + multiblock carriers allocated via mseg_alloc by + allocator ]]>. When this limit has been reached, + new multiblock carriers will be allocated via + sys_alloc. + mmsbc ]]> + + + Max mseg_alloc singleblock carriers. Maximum number of + singleblock carriers allocated via mseg_alloc by + allocator ]]>. When this limit has been reached, + new singleblock carriers will be allocated via + sys_alloc. + ramv ]]> + + + Realloc always moves. When enabled, reallocate operations will + more or less be translated into an allocate, copy, free sequence. + This often reduce memory fragmentation, but costs performance. + + rmbcmt ]]> + + + Relative multiblock carrier move threshold (in percent). When + a block located in a multiblock carrier is shrunk, + the block will be moved if the ratio of the size of the returned + memory compared to the previous size is more than this threshold; + otherwise, the block will be shrunk at current location. + rsbcmt ]]> + + + Relative singleblock carrier move threshold (in percent). When + a block located in a singleblock carrier is shrunk to + a size smaller than the value of the + sbct parameter, + the block will be left unchanged in the singleblock carrier if + the ratio of unused memory is less than this threshold; + otherwise, it will be moved into a multiblock carrier. + rsbcst ]]> + + + Relative singleblock carrier shrink threshold (in + percent). When a block located in an mseg_alloc + singleblock carrier is shrunk, the carrier will be left + unchanged if the ratio of unused memory is less than this + threshold; otherwise, the carrier will be shrunk. + See also asbcst. + sbct ]]> + + + Singleblock carrier threshold. Blocks larger than this + threshold will be placed in singleblock carriers. Blocks + smaller than this threshold will be placed in multiblock + carriers. + smbcs ]]> + + + Smallest (mseg_alloc) multiblock carrier size (in + kilobytes). See the description + on how sizes for mseg_alloc multiblock carriers are decided + in "the alloc_util framework" section. + t true|false|]]> + + + Multiple, thread specific instances of the allocator. + This option will only have any effect on the runtime system + with SMP support. Default behaviour on the runtime system with + SMP support (N equals the number of scheduler threads): + + temp_alloc + N + 1 instances. + ll_alloc + 1 instance. + Other allocators + N instances when N is less than or equal to + 16. 16 instances when N is greater than + 16. + + temp_alloc will always use N + 1 instances when + this option has been enabled regardless of the amount passed. + Other allocators will use the same amount of instances as the + amount passed as long as it isn't greater than N. + + +

Currently the following flags are available for configuration of + alloc_util, i.e. all allocators based on alloc_util + will be effected:

+ + ]]> + +sys_alloc carrier size. Carriers allocated via + sys_alloc will be allocated in sizes which are + multiples of the sys_alloc carrier size. This is not + true for main multiblock carriers and carriers allocated + during a memory shortage, though. + ]]> + + + Max mseg_alloc carriers. Maximum number of carriers + placed in separate memory segments. When this limit has been + reached, new carriers will be placed in memory retrieved from + sys_alloc. + +

Instrumentation flags:

+ + +Mim true|false + + + A map over current allocations is kept by the emulator. The + allocation map can be retrieved via the instrument + module. +Mim true implies +Mis true. + +Mim true is the same as + -instr. + +Mis true|false + + + Status over allocated memory is kept by the emulator. The + allocation status can be retrieved via the instrument + module. + +Mit X + + + Reserved for future use. Do not use this flag. + + +

When instrumentation of the emulator is enabled, the emulator + uses more memory and runs slower.

+ +

Other flags:

+ + +Mea min|max|r9c|r10b|r11b|config + + + min + + Disables all allocators that can be disabled. + + + max + + Enables all allocators (currently default). + + + r9c|r10b|r11b + + Configures all allocators as they were configured in respective + OTP release. These will eventually be removed. + + + config + + Disables features that cannot be enabled while creating an + allocator configuration with + erts_alloc_config(3). + Note, this option should only be used while running + erts_alloc_config, not when using the created + configuration. + + + + +

Only some default values have been presented + here. + erlang:system_info(allocator), + and + erlang:system_info({allocator, Alloc}) + can be used in order to obtain currently used settings and current + status of the allocators.

+ +

Most of these flags are highly implementation dependent, and they + may be changed or removed without prior notice.

+

erts_alloc is not obliged to strictly use the settings that + have been passed to it (it may even ignore them).

+ +

erts_alloc_config(3) + is a tool that can be used to aid creation of an + erts_alloc configuration that is suitable for a limited + number of runtime scenarios.

+
+ +
+ SEE ALSO +

erts_alloc_config(3), + erl(1), + instrument(3), + erlang(3)

+
+ + -- cgit v1.2.3