View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0000640 | LDMud 3.3 | Portability | public | 2009-05-22 04:55 | 2009-05-28 14:16 |
Reporter | zesstra | Assigned To | zesstra | ||
Priority | normal | Severity | minor | Reproducibility | N/A |
Status | resolved | Resolution | fixed | ||
Platform | x86_64 | OS | MacOS X | OS Version | 10.5.x |
Product Version | 3.3.718 | ||||
Target Version | 3.3.719 | Fixed in Version | 3.3.719 | ||
Summary | 0000640: Support mmap() for allocating memory from the system | ||||
Description | Our allocators usually try to use sbrk()/brk() for getting memory from the system and replace the system malloc. If this does not work, the fall back to allocate memory with malloc(). sbrk/brk() is declared legacy in SUSv2 and removed in POSIX.1-2001. Darwin has the functions, but brk() is basically a noop and sbrk() can only adjust the break value within 4MB. Therefore we should introduce support to get memory by mmapping anonymous pages into our virtual address range. This will support REPLACE_MALLOC on Darwin/MacOS again (and maybe on other systems). When support for brk/sbrk() vanishes in mainline OS, we may remove brk() eventually. | ||||
Tags | No tags attached. | ||||
Attached Files | 0001-Use-mmap-for-getting-memory-from-the-system-slaba.patch (6,249 bytes)
From 80d078af312082fcfc162b5b5f6d379083d10487 Mon Sep 17 00:00:00 2001 From: zesstra <zesstra@zesstra.de> Date: Thu, 21 May 2009 17:22:26 +0200 Subject: [PATCH 1/3] Use mmap() for getting memory from the system (slaballoc). sbrk() and brk() are obsoleted and some systems (like Darwin) don't have a working implementation anymore. If sbrk/brk() are not usable, but mmap() is available, use mmap() for mapping anonymous memory which we then use. If mmap() is not available, use the old fallback (system malloc()). Changes some precompiler checks for SBRK_OK, which should be REPLACE_MALLOC. Signed-off-by: zesstra <zesstra@zesstra.de> --- src/slaballoc.c | 70 ++++++++++++++++++++++++++++++++++++++++++++---------- src/xalloc.c | 4 +- 2 files changed, 59 insertions(+), 15 deletions(-) diff --git a/src/slaballoc.c b/src/slaballoc.c index 96858fd..e32f77e 100644 --- a/src/slaballoc.c +++ b/src/slaballoc.c @@ -25,10 +25,10 @@ * * -- Small Blocks -- * - * Small blocks are allocations of up to (SMALL_BLOCK_NUM+1)*4 Bytes, currently - * 68 Bytes. Such block are allocated from larger memory blocks ('slabs') - * sized rough multiples of 4KByte. The size is determined such that the slab - * holds at least 100 blocks. + * Small blocks are allocations of up to (SMALL_BLOCK_NUM+1)*GRANULARITY Bytes, + * currently 68-136 Bytes. Such block are allocated from larger memory blocks + * ('slabs') sized rough multiples of 4KByte. The size is determined such that + * the slab holds at least 100 blocks. * * A slab consists of a header, followed by the blocks themselves. Free blocks * within a slab are linked together in a free list. When a slab is freshly @@ -200,13 +200,17 @@ #include "driver.h" #include "typedefs.h" -#ifdef HAVE_MADVISE +#if defined(HAVE_MADVISE) || defined(HAVE_MMAP) +# include <sys/types.h> # include <sys/mman.h> # define MADVISE(new,old) madvise(new,old,MADV_RANDOM) #else # define MADVISE(new,old) NOOP #endif +// for sysconf() +#include <unistd.h> + #include "slaballoc.h" #include "mstrings.h" @@ -246,7 +250,11 @@ # else # undef SBRK_OK # endif -#endif +#else if defined (HAVE_MMAP) +# ifdef MALLOC_SBRK +# define REPLACE_MALLOC +# endif +#endif // SBRK_OK #define MEM_MAIN_THREADSAFE @@ -344,7 +352,7 @@ */ -#ifdef SBRK_OK +#if defined(MALLOC_SBRK) && (defined(SBRK_OK) || defined (HAVE_MMAP)) # define CHUNK_SIZE 0x40000 #else /* It seems to be advantagous to be just below a power of two @@ -933,7 +941,7 @@ mem_dump_data (strbuf_t *sbuf) ); dump_stat("\npermanent blocks: %8lu %10lu\n", perm_st); -#ifdef SBRK_OK +#ifdef REPLACE_MALLOC dump_stat("clib allocations: %8lu %10lu\n", clib_st); #else strbuf_addf(sbuf, "clib allocations: n/a n/a\n"); @@ -3442,7 +3450,7 @@ esbrk (word_t size, size_t * pExtra) * #ifdef SBRK_OK * It is system dependent how sbrk() aligns data, so we simpy use brk() - * to insure that we have enough. + * to ensure that we have enough. * At the end of the newly expanded heap we create a fake allocated * block of 0 bytes so that large_malloc() knows where to stop. #else @@ -3487,6 +3495,7 @@ esbrk (word_t size, size_t * pExtra) #else /* not SBRK_OK */ char *block; + size_t req_size = size; // the requested size of memory. word_t *p; /* start of the fake block */ const int overhead = TL_OVERHEAD; size_t overlap = 0; @@ -3494,16 +3503,31 @@ esbrk (word_t size, size_t * pExtra) * the new allocation with the old one. */ - mdb_log_sbrk(size); size += overhead * GRANULARITY; /* for the extra fake "allocated" block */ + // get the new memory block +#ifdef HAVE_MMAP + { + static size_t pagesize = 0; // pagesize - 1 for this system. + if (!pagesize) + pagesize = getpagesize() - 1; + // round to multiples of the real pagesize + if ((size & pagesize) != 0) + { + size += pagesize; + size &= ~pagesize; + } + // get new page(s) + block = mmap(0, size, PROT_READ|PROT_WRITE, MAP_ANON|MAP_PRIVATE, -1, 0); + if (block == MAP_FAILED) + return NULL; + } +#else block = malloc(size); if (!block) - { - *pExtra = 0; return NULL; - } +#endif assert_stack_gap(); @@ -3643,7 +3667,12 @@ esbrk (word_t size, size_t * pExtra) count_up(&sbrk_stat, size); count_up(&large_wasted_stat, overhead * GRANULARITY); + // amount of additional memory that was allocated +#ifdef HAVE_MMAP + *pExtra = overlap + (size - overhead * GRANULARITY - req_size); +#else *pExtra = overlap; +#endif return block + GRANULARITY; #endif /* !SBRK_OK */ } /* esbrk() */ @@ -4371,4 +4400,19 @@ mem_consolidate (Bool force) #endif /* MALLOC_EXT_STATISTICS */ } /* mem_consolidate() */ +/*-------------------------------------------------------------------------*/ +#if !defined(HAVE_GETPAGESIZE) +static INLINE size_t +getpagesize() +/* get the pagesize for this system */ +{ +#ifdef(HAVE_SYSCONF) + return sysconf(_SC_PAGESIZE); +#else +#error Unable to find out the system pagesize. Please report this issue. +#endif +} +#endif /* HAVE_GETPAGESIZE */ + /***************************************************************************/ + diff --git a/src/xalloc.c b/src/xalloc.c index 89e5b58..0ae0784 100644 --- a/src/xalloc.c +++ b/src/xalloc.c @@ -182,7 +182,7 @@ static t_stat xalloc_stat = {0,0}; /* Total number and size of allocations done by the driver (incl overhead). */ -#if defined(SBRK_OK) +#if defined(MALLOC_SBRK) && (defined(SBRK_OK) || defined(HAVE_MMAP)) static t_stat clib_alloc_stat = {0,0}; /* Number and size of allocations done through the clib emulation * functions (incl overhead). @@ -1178,7 +1178,7 @@ dump_malloc_trace (int d /* CLIB ALLOCATION FUNCTIONS */ -#if defined(REPLACE_MALLOC) && defined(SBRK_OK) +#if defined(REPLACE_MALLOC) && (defined(SBRK_OK) || defined(HAVE_MMAP)) /*-------------------------------------------------------------------------*/ static POINTER -- 1.6.1 0002-Use-mmap-for-getting-memory-from-the-system-small.patch (6,988 bytes)
From c488c71a0e14e549699106dad0726e5ffc83faa5 Mon Sep 17 00:00:00 2001 From: zesstra <zesstra@zesstra.de> Date: Thu, 21 May 2009 18:36:39 +0200 Subject: [PATCH 2/3] Use mmap() for getting memory from the system (smalloc.c) sbrk() and brk() are obsoleted and some systems (like Darwin) don't have a working implementation anymore. If sbrk/brk() are not usable, but mmap() is available, use mmap() for mapping anonymous memory which we then use. If mmap() is not available, use the old fallback (system malloc()). Changes some precompiler checks for SBRK_OK, which should be REPLACE_MALLOC. Signed-off-by: zesstra <zesstra@zesstra.de> --- src/slaballoc.c | 9 ++++--- src/smalloc.c | 73 +++++++++++++++++++++++++++++++++++++++++++++---------- 2 files changed, 65 insertions(+), 17 deletions(-) diff --git a/src/slaballoc.c b/src/slaballoc.c index e32f77e..07ad8f7 100644 --- a/src/slaballoc.c +++ b/src/slaballoc.c @@ -364,6 +364,7 @@ /* Bitflags for the size field: * TODO: Assumes a word_t of at least 32 bit. + * TODO: define values for 64-bit word_t to support larger blocks. */ #define PREV_BLOCK 0x80000000 /* Previous block is allocated */ #define THIS_BLOCK 0x40000000 /* This block is allocated */ @@ -3463,9 +3464,10 @@ esbrk (word_t size, size_t * pExtra) */ { -#ifdef SBRK_OK - mdb_log_sbrk(size); + +#ifdef SBRK_OK + *pExtra = 0; if (!heap_end) { @@ -3503,7 +3505,6 @@ esbrk (word_t size, size_t * pExtra) * the new allocation with the old one. */ - mdb_log_sbrk(size); size += overhead * GRANULARITY; /* for the extra fake "allocated" block */ // get the new memory block @@ -3622,7 +3623,7 @@ esbrk (word_t size, size_t * pExtra) next = prev + *prev; } while (next < (word_t *)block); overlap = 0; - + if ((word_t *)block == prev + overhead) { /* Our block directly follows the one we found */ diff --git a/src/smalloc.c b/src/smalloc.c index f3c7c63..d9dfaed 100644 --- a/src/smalloc.c +++ b/src/smalloc.c @@ -213,13 +213,17 @@ #include "driver.h" #include "typedefs.h" -#ifdef HAVE_MADVISE +#if defined(HAVE_MADVISE) || defined(HAVE_MMAP) || defined(HAVE_POSIX_MADVISE) +# include <sys/types.h> # include <sys/mman.h> # define MADVISE(new,old) madvise(new,old,MADV_RANDOM) #else # define MADVISE(new,old) NOOP #endif +// for sysconf() +#include <unistd.h> + #include "smalloc.h" #include "mstrings.h" @@ -258,7 +262,11 @@ # else # undef SBRK_OK # endif -#endif +#else if defined (HAVE_MMAP) +# ifdef MALLOC_SBRK +# define REPLACE_MALLOC +# endif +#endif // SBRK_OK #define MEM_MAIN_THREADSAFE @@ -362,7 +370,7 @@ * CHUNK_SIZE: size of a chunk from which large blocks are allocated. */ -#ifdef SBRK_OK +#if defined(MALLOC_SBRK) && (defined(SBRK_OK) || defined (HAVE_MMAP)) # define SMALL_CHUNK_SIZE 0x04000 /* 16 KByte */ # define CHUNK_SIZE 0x40000 #else @@ -379,6 +387,7 @@ /* Bitflags for the size field: * TODO: Assumes a 32-Bit word_t. + * TODO: define values for 64-bit word_t to support larger blocks. */ #define PREV_BLOCK 0x80000000 /* Previous block is allocated */ #define THIS_BLOCK 0x40000000 /* This block is allocated */ @@ -777,7 +786,7 @@ mem_dump_data (strbuf_t *sbuf) */ { -#ifdef SBRK_OK +#ifdef REPLACE_MALLOC t_stat clib_st; #endif t_stat sbrk_st, perm_st, xalloc_st; @@ -793,7 +802,7 @@ mem_dump_data (strbuf_t *sbuf) #endif /* MALLOC_EXT_STATISTICS */ sbrk_st = sbrk_stat; -#ifdef SBRK_OK +#ifdef REPLACE_MALLOC clib_st = clib_alloc_stat; #endif xalloc_st = xalloc_stat; @@ -828,7 +837,7 @@ mem_dump_data (strbuf_t *sbuf) dump_stat("small wasted: %8lu %10lu (h)\n\n",s_wasted); dump_stat("permanent blocks: %8lu %10lu\n", perm_st); -#ifdef SBRK_OK +#ifdef REPLACE_MALLOC dump_stat("clib allocations: %8lu %10lu\n", clib_st); #else strbuf_addf(sbuf, "clib allocations: n/a n/a\n"); @@ -961,7 +970,7 @@ mem_dinfo_data (svalue_t *svp, int value) ST_NUMBER(DID_MEM_MINC_CALLS, malloc_increment_size_calls); ST_NUMBER(DID_MEM_MINC_SUCCESS, malloc_increment_size_success); ST_NUMBER(DID_MEM_MINC_SIZE, malloc_increment_size_total); -#ifdef SBRK_OK +#ifdef REPLACE_MALLOC ST_NUMBER(DID_MEM_CLIB, clib_alloc_stat.counter); ST_NUMBER(DID_MEM_CLIB_SIZE, clib_alloc_stat.size); #endif @@ -3243,10 +3252,12 @@ esbrk (word_t size, size_t * pExtra) */ { -#ifdef SBRK_OK - mdb_log_sbrk(size); + *pExtra = 0; + +#ifdef SBRK_OK + if (!heap_end) { /* First call: allocate the first fake block */ @@ -3275,6 +3286,7 @@ esbrk (word_t size, size_t * pExtra) #else /* not SBRK_OK */ char *block; + size_t req_size = size; // the requested size of memory. word_t *p; /* start of the fake block */ const int overhead = TL_OVERHEAD; size_t overlap = 0; @@ -3283,15 +3295,31 @@ esbrk (word_t size, size_t * pExtra) */ - mdb_log_sbrk(size); size += overhead * GRANULARITY; /* for the extra fake "allocated" block */ + // get the new memory block +#ifdef HAVE_MMAP + { + static size_t pagesize = 0; // pagesize - 1 for this system. + if (!pagesize) + pagesize = getpagesize() - 1; + // round to multiples of the real pagesize + if ((size & pagesize) != 0) + { + size += pagesize; + size &= ~pagesize; + } + // get new page(s) + block = mmap(0, size, PROT_READ|PROT_WRITE, MAP_ANON|MAP_PRIVATE, -1, 0); + if (block == MAP_FAILED) + return NULL; + } +#else block = malloc(size); if (!block) - { - *pExtra = 0; return NULL; - } +#endif + assert_stack_gap(); /* p points to the start of the fake allocated block used @@ -3430,7 +3458,12 @@ esbrk (word_t size, size_t * pExtra) count_up(&sbrk_stat, size); count_up(&large_wasted_stat, overhead * GRANULARITY); + // amount of additional memory that was allocated +#ifdef HAVE_MMAP + *pExtra = overlap + (size - overhead * GRANULARITY - req_size); +#else *pExtra = overlap; +#endif return block + GRANULARITY; #endif /* !SBRK_OK */ } /* esbrk() */ @@ -4017,4 +4050,18 @@ mem_consolidate (Bool force) } /* for (all chunks) */ } /* mem_consolidate() */ + +/*-------------------------------------------------------------------------*/ +#if !defined(HAVE_GETPAGESIZE) +static INLINE size_t +getpagesize() +/* get the pagesize for this system */ +{ +#ifdef(HAVE_SYSCONF) + return sysconf(_SC_PAGESIZE); +#else +#error Unable to find out the system pagesize. Please report this issue. +#endif +} +#endif /* HAVE_GETPAGESIZE */ /***************************************************************************/ -- 1.6.1 0003-Removed-amalloc-afree-MALLOC_ALIGN-and-MEM_ALIGN.patch (14,690 bytes)
From a7f48838296b7eb3351e77eab5646bcfd20df96b Mon Sep 17 00:00:00 2001 From: zesstra <zesstra@zesstra.de> Date: Thu, 21 May 2009 20:18:52 +0200 Subject: [PATCH 3/3] Removed amalloc/afree(), MALLOC_ALIGN and MEM_ALIGN. If we replace the system malloc, we defined amalloc() to get aligned memory. These function used MEM_ALIGN and MALLOC_ALIGN. Unfortunately it contained an overflow (it reserved MALLOC_ALIGN-MEM_ALIGN, but wrote up to MALLOC_ALIGN chars). Additionally, it never worked. If using slaballoc/smalloc/sysmalloc, MEM_ALIGN and MALLOC_ALIGN were always equal and in case of xptmalloc MALLOC_ALIGN was always <= MEM_ALIGN. Therefore, amalloc never added anything for alignment. Furthermore, if malloc/amalloc() should use the same alignment as the system allocator, our amalloc() was not sufficient. We may re-introduce allocator functions for aligned memory if needed (the driver does not), but then we should do it properly (and ensure to have the same alignment as the system allocator). Signed-off-by: zesstra <zesstra@zesstra.de> --- src/autoconf/configure.in | 10 ---- src/slaballoc.c | 42 +++++++++------ src/smalloc.c | 46 ++++++++++------- src/sysmalloc.c | 1 - src/xalloc.c | 124 ++++---------------------------------------- src/xptmalloc.c | 4 -- 6 files changed, 65 insertions(+), 162 deletions(-) diff --git a/src/autoconf/configure.in b/src/autoconf/configure.in index 0077d9a..cdd5540 100644 --- a/src/autoconf/configure.in +++ b/src/autoconf/configure.in @@ -817,16 +817,6 @@ AC_CHECK_FUNCS(fchmod getrusage bzero memset memcpy memmem strdup strcspn) AC_CHECK_FUNCS(strchr strrchr getcwd memmove sysconf gettimeofday wait3 waitpid) AC_CHECK_FUNCS(fcntl getdomainname poll strtoul trunc) - -AC_CACHE_CHECK(for needed malloc() alignment,lp_cv_sys_malloc_align, -AC_TRY_COMPILE([struct ts {double d; char *p; double e;}; -int i = 96/(sizeof(struct ts)-20); -],, -lp_cv_sys_malloc_align=8, -lp_cv_sys_malloc_align=4)) -AC_DEFINE_UNQUOTED(MALLOC_ALIGN,$lp_cv_sys_malloc_align,[word alignment]) - - if test "x$ac_cv_type_signal" = "xvoid"; then AC_DEFINE(RETSIGTYPE_VOID, 1, [Set in response to the signal handler return type, since not all diff --git a/src/slaballoc.c b/src/slaballoc.c index 07ad8f7..2725aef 100644 --- a/src/slaballoc.c +++ b/src/slaballoc.c @@ -110,12 +110,25 @@ * A new AVL node is created for every free large block. #endif * -#ifdef SBRK_OK && MALLOC_SBRK +#ifdef MALLOC_SBRK && SBRK_OK * Memory is allocated from the system with sbrk() and brk(), which puts * the whole heap under our control. * - * In this mode, several functions like amalloc() are compiled as malloc(), - * implementing all libc memory functions. + * In this mode, we replace the system malloc() and free() by our own + * functions, implementing all libc memory functions (malloc, free, calloc, + * realloc()). +#else if MALLOC_SBRK && HAVE_MMAP + * If brk/sbrk() are non-functional (e.g. on Darwin; remember: they are legacy + * and not POSIX anymore), we check if mmap() is available. If it is, we use + * it to map anonymous memory pages to get memory from the VM system. + * + * The allocated block (modulo joints) is tagged at both ends with fake + * "allocated" blocks of which cover the unallocated areas - large_malloc() + * will perceive this as a fragmented heap. + * + * In this mode, we replace the system malloc() and free() by our own + * functions, implementing all libc memory functions (malloc, free, calloc, + * realloc()). #else * malloc() is used to allocate a new block of memory. If this block borders * previous ones, the blocks are joined. @@ -232,25 +245,19 @@ */ #define GRANULARITY sizeof(word_t) -/* Defines required by the xalloc.c wrapper */ -#define MEM_ALIGN GRANULARITY - /* If possible, request memory using sbrk(). But if using pthreads, do * not replace malloc() with our routines, even if the system allows it, * as slaballoc is not threadsafe. + * If sbrk/brk() are not working, but mmap() is, then use mmap() for getting + * memory. If that is also not available, use malloc(). */ - #if defined(SBRK_OK) # ifdef MALLOC_SBRK -# if MALLOC_ALIGN == 4 -# define REPLACE_MALLOC -# else -# undef SBRK_OK -# endif +# define REPLACE_MALLOC # else # undef SBRK_OK # endif -#else if defined (HAVE_MMAP) +#else if defined(HAVE_MMAP) # ifdef MALLOC_SBRK # define REPLACE_MALLOC # endif @@ -605,12 +612,15 @@ static unsigned long num_avl_nodes = 0; /*-------------------------------------------------------------------------*/ /* Forward declarations */ -static char *esbrk(word_t, size_t * pExtra); +static char *esbrk(word_t size, size_t * pExtra) __attribute__((malloc,warn_unused_result)); -static char *large_malloc(word_t, Bool); +static char *large_malloc(word_t size, Bool force_m) __attribute__((malloc,warn_unused_result)); #define large_malloc_int(size, force_m) large_malloc(size, force_m) static void large_free(char *); +static INLINE size_t mem_overhead (void) __attribute__((const)); + + #ifdef MALLOC_EXT_STATISTICS /*=========================================================================*/ @@ -726,7 +736,7 @@ mem_block_size (POINTER p) if (q[M_SIZE] & M_SMALL) { - return mem_block_total_size(p) - M_OVERHEAD*GRANULARITY; + return mem_block_total_size(p) - mem_overhead(); } return mem_block_total_size(p) - ML_OVERHEAD*GRANULARITY; } /* mem_block_size() */ diff --git a/src/smalloc.c b/src/smalloc.c index d9dfaed..e657e7c 100644 --- a/src/smalloc.c +++ b/src/smalloc.c @@ -112,20 +112,33 @@ * A new AVL node is created for every free large block. #endif * -#ifdef SBRK_OK && MALLOC_SBRK + #ifdef MALLOC_SBRK && SBRK_OK * Memory is allocated from the system with sbrk() and brk(), which puts * the whole heap under our control. * - * In this mode, several functions like amalloc() are compiled as malloc(), - * implementing all libc memory functions. -#else + * In this mode, we replace the system malloc() and free() by our own + * functions, implementing all libc memory functions (malloc, free, calloc, + * realloc()). + #else if MALLOC_SBRK && HAVE_MMAP + * If brk/sbrk() are non-functional (e.g. on Darwin; remember: they are legacy + * and not POSIX anymore), we check if mmap() is available. If it is, we use + * it to map anonymous memory pages to get memory from the VM system. + * + * The allocated block (modulo joints) is tagged at both ends with fake + * "allocated" blocks of which cover the unallocated areas - large_malloc() + * will perceive this as a fragmented heap. + * + * In this mode, we replace the system malloc() and free() by our own + * functions, implementing all libc memory functions (malloc, free, calloc, + * realloc()). + #else * malloc() is used to allocate a new block of memory. If this block borders * previous ones, the blocks are joined. * * The allocated block (modulo joints) is tagged at both ends with fake * "allocated" blocks of which cover the unallocated areas - large_malloc() * will perceive this as a fragmented heap. -#endif + #endif * * -- Privilege Level -- * @@ -245,20 +258,13 @@ */ #define GRANULARITY sizeof(word_t) -/* Defines required by the xalloc.c wrapper */ - -#define MEM_ALIGN GRANULARITY - /* If possible, request memory using sbrk(). + * If sbrk/brk() are not working, but mmap() is, then use mmap() for getting + * memory. If that is also not available, use malloc(). */ - #if defined(SBRK_OK) # ifdef MALLOC_SBRK -# if MALLOC_ALIGN == 4 -# define REPLACE_MALLOC -# else -# undef SBRK_OK -# endif +# define REPLACE_MALLOC # else # undef SBRK_OK # endif @@ -603,11 +609,13 @@ static long num_avl_nodes = 0; /*-------------------------------------------------------------------------*/ /* Forward declarations */ -static char *esbrk(word_t, size_t * pExtra); +static char *esbrk(word_t size, size_t * pExtra) __attribute__((malloc,warn_unused_result)); -static char *large_malloc(word_t, Bool); +static char *large_malloc(word_t size, Bool force_m) __attribute__((malloc,warn_unused_result)); #define large_malloc_int(size, force_m) large_malloc(size, force_m) -static void large_free(char *); +static void large_free(char *ptr); +static INLINE size_t mem_overhead(void) __attribute__((const)); + #ifdef MALLOC_EXT_STATISTICS /*=========================================================================*/ @@ -695,7 +703,7 @@ mem_block_size (POINTER p) word_t size = (q[M_SIZE] & M_MASK); if (size > SMALL_BLOCK_MAX) return mem_block_total_size(p) - ML_OVERHEAD*GRANULARITY; - return mem_block_total_size(p) - M_OVERHEAD*GRANULARITY; + return mem_block_total_size(p) - mem_overhead(); } /* mem_block_size() */ /*-------------------------------------------------------------------------*/ diff --git a/src/sysmalloc.c b/src/sysmalloc.c index 5e0b81a..49c59e1 100644 --- a/src/sysmalloc.c +++ b/src/sysmalloc.c @@ -17,7 +17,6 @@ #include "../mudlib/sys/debug_info.h" /* Defines required by the xalloc.c wrapper */ -#define MEM_ALIGN (MALLOC_ALIGN) /* #undef REPLACE_MALLOC */ #define NO_MEM_BLOCK_SIZE #define MEM_THREADSAFE diff --git a/src/xalloc.c b/src/xalloc.c index 0ae0784..a5c6b2c 100644 --- a/src/xalloc.c +++ b/src/xalloc.c @@ -113,8 +113,6 @@ enum xalloc_header_fields { XM_OVERHEAD_SIZE = XM_OVERHEAD * sizeof(word_t), }; -#define XM_OVERHEAD_SIZE (XM_OVERHEAD * sizeof(word_t)) - /*-------------------------------------------------------------------------*/ /* -- Global Variables/Arguments dealing with memory -- */ @@ -323,8 +321,6 @@ mdb_log_sbrk (p_int size) #endif #endif * - * #define MEM_ALIGN - * the alignment guaranteed by the allocator * #define REPLACE_MALLOC * if the allocator's mem_alloc()/mem_free() can be used to replace the * libc allocation routines (ie. the allocator doesn't use malloc() @@ -1181,122 +1177,26 @@ dump_malloc_trace (int d #if defined(REPLACE_MALLOC) && (defined(SBRK_OK) || defined(HAVE_MMAP)) /*-------------------------------------------------------------------------*/ -static POINTER -amalloc (size_t size) +POINTER +malloc (size_t size) -/* Allocate an aligned block of <size> bytes, if necessary, with - * SYSTEM privilege. The block is not subject to GC. - * Result is the pointer to the allocated block, or NULL. +/* Allocate an empty memory block of size <sizel>. + * The memory is aligned and not subject to GC. */ { - char *temp; - size_t orig_size UNUSED; // unused if no HAVE_MADVISE. - - if (MALLOC_ALIGN > MEM_ALIGN) - { - -#if defined(HAVE_MADVISE) - orig_size = size; -#endif - - size += (MALLOC_ALIGN-MEM_ALIGN); - } - - temp = (char *)pxalloc(size); - if (!temp) + POINTER result = pxalloc(size); + if (!result) { int save_privilege = malloc_privilege; malloc_privilege = MALLOC_SYSTEM; - temp = (char *)pxalloc(size); + result = pxalloc(size); malloc_privilege = save_privilege; } - - if (MALLOC_ALIGN > MEM_ALIGN) - { - if (temp) - { - /* Set the first byte of the alignment area to 0xAF - afree(0 - * is going to look for it - and the rest to 0. - */ - *temp++ = 0xAF; - while ((p_int)temp & (MALLOC_ALIGN-1)) - *temp++ = 0; - MADVISE(temp, orig_size); - } - } - - return (POINTER)temp; -} /* amalloc() */ - -/*-------------------------------------------------------------------------*/ -static void -afree (POINTER p) - -/* Free the aligned memory block <p>. - */ - -{ - char *q = (char *)p; - - if (!q) - return; - - if (MALLOC_ALIGN > MEM_ALIGN) - { - - /* amalloc() filled the alignment area with 0s except for the first byte. - * Search backwards to find that marker and with it the real block begin. - */ - while (!*--q) NOOP; - } - - pfree(q); -} /* afree() */ - -/*-------------------------------------------------------------------------*/ -static INLINE long -get_block_size (POINTER ptr) - -/* Get the allocated block size for the block with user area starting - * at <ptr>. This function is meant only for block allocated with (a)malloc(). - * Result is the size in bytes inclusive overhead. - */ -{ - long size = 0; - - if (ptr) - { - /* Get the allocated size of the block for the statistics */ - - char *q; - - q = (char *)ptr; - - if (MALLOC_ALIGN > MEM_ALIGN) - while ( !(size = *--q) ) NOOP; - - size = xalloced_size(q) + mem_overhead(); - } - - return size; -} /* get_block_size() */ - -/*-------------------------------------------------------------------------*/ -POINTER -malloc (size_t size) - -/* Allocate an empty memory block of size <sizel>. - * The memory is aligned and not subject to GC. - */ - -{ - POINTER result; - - result = amalloc(size); + if (result) { - count_up(&clib_alloc_stat, get_block_size(result)); + count_up(&clib_alloc_stat, xalloced_size(result) + mem_overhead()); } return result; @@ -1312,10 +1212,10 @@ free (POINTER ptr) { if (ptr) { - count_back(&clib_alloc_stat, get_block_size(ptr)); + count_back(&clib_alloc_stat, xalloced_size(ptr) + mem_overhead()); } - afree(ptr); + pfree(ptr); FREE_RETURN } /* free() */ @@ -1354,7 +1254,7 @@ realloc (POINTER p, size_t size) if (!p) return malloc(size); - old_size = get_block_size(p) - xalloc_overhead(); + old_size = xalloced_size(p) - xalloc_overhead(); if (old_size >= size) return p; diff --git a/src/xptmalloc.c b/src/xptmalloc.c index d8fa66f..d467b11 100644 --- a/src/xptmalloc.c +++ b/src/xptmalloc.c @@ -232,10 +232,6 @@ static Bool mem_is_freed (POINTER p, size_t minsize) { #endif /* GC_SUPPORT */ -/* the alignment guaranteed by the allocator */ -//#define MEM_ALIGN (2*(sizeof(size_t))) -#define MEM_ALIGN (2*SIZEOF_INT) - /* If the allocator can replace the libc allocation routines. * See above for the *BSD situation. */ -- 1.6.1 | ||||
|
I attached 3 patches for adding mmap support to slaballoc.c and smalloc.c The third patch removes amalloc() and MALLOC_ALIGN. The reasoning for this is: If we replace the system malloc, we defined amalloc() to get aligned memory. These function used MEM_ALIGN and MALLOC_ALIGN. Unfortunately it contained an overflow (it reserved MALLOC_ALIGN-MEM_ALIGN, but wrote up to MALLOC_ALIGN chars). Additionally, it never worked. If using slaballoc/smalloc/sysmalloc, MEM_ALIGN and MALLOC_ALIGN were always equal and in case of xptmalloc MALLOC_ALIGN was always <= MEM_ALIGN. Therefore, amalloc never added anything for alignment. Furthermore, if malloc/amalloc() should use the same alignment as the system allocator, our amalloc() was not sufficient. We may re-introduce allocator functions for aligned (other than the word_t which the allocators guarantee) memory if needed (the driver does obviously not), but then we should do it properly (and ensure to have the same alignment as the system allocator). |
|
This should be resolved by r2604 - r2606. |
Date Modified | Username | Field | Change |
---|---|---|---|
2009-05-22 04:55 | zesstra | New Issue | |
2009-05-22 04:55 | zesstra | Status | new => assigned |
2009-05-22 04:55 | zesstra | Assigned To | => zesstra |
2009-05-26 15:16 | zesstra | File Added: 0001-Use-mmap-for-getting-memory-from-the-system-slaba.patch | |
2009-05-26 15:16 | zesstra | File Added: 0002-Use-mmap-for-getting-memory-from-the-system-small.patch | |
2009-05-26 15:16 | zesstra | File Added: 0003-Removed-amalloc-afree-MALLOC_ALIGN-and-MEM_ALIGN.patch | |
2009-05-26 15:20 | zesstra | Note Added: 0001165 | |
2009-05-28 14:16 | zesstra | Note Added: 0001173 | |
2009-05-28 14:16 | zesstra | Status | assigned => resolved |
2009-05-28 14:16 | zesstra | Fixed in Version | => 3.3.719 |
2009-05-28 14:16 | zesstra | Resolution | open => fixed |