Fix integer overflow when set a large maximum value for atom table

When setting maximum atom table size using +t option, there will be a
integer overflow for a large size.

	$ erl +t2147482625
	ll_alloc: Cannot allocate 18446744073692774400 bytes of memory
	(of type "atom_tab").

The overflow is caused by the arithmetic operations on int type.

When 2147482625 + 1024 it will become -2147483647 due to the signed
integerger overflow. Then the result will be resized to Uint type, which
is a unsigned long type, the negative int will first be expand to 64
bits long via sign extension, then change to unsigned type, which
becomes 18446744073692774400.

The fix is done by convert `limit` to Uint type before doing any
arithmetic operation. This will expand variable to 64 bits long type via
zero extension, then the following operation are all positive, therefore
no overflow will happen.

Note: here we assume the int `limit` passed in is always positive. If
some future change cause the `limit` passed in maybe negative, then the
current fix will also cause overflow.

1 files changed, 1 insertions, 1 deletions

diff --git a/erts/emulator/beam/index.c b/erts/emulator/beam/index.c
index a1f6f54543..7bf1a032c1 100644
--- a/erts/emulator/beam/index.c
+++ b/erts/emulator/beam/index.c
@@ -58,7 +58,7 @@ IndexTable*
 erts_index_init(ErtsAlcType_t type, IndexTable* t, char* name,
 		int size, int limit, HashFunctions fun)
 {
-    Uint base_size = ((limit+INDEX_PAGE_SIZE-1)/INDEX_PAGE_SIZE)*sizeof(IndexSlot*);
+    Uint base_size = (((Uint)limit+INDEX_PAGE_SIZE-1)/INDEX_PAGE_SIZE)*sizeof(IndexSlot*);
     hash_init(type, &t->htable, name, 3*size/4, fun);
 
     t->size = 0;