MPR Memory Allocator

Overview

Appweb provides an application-specific memory allocator to use instead of malloc. This allocator is part of the Multithreaded Portable Runtime (MPR) and is tailored to the needs of embedded applications. It is faster than most general purpose malloc allocators for these workloads. It is deterministic and allocates and frees memory in constant time O(1). It exhibits very low fragmentation and accurate coalescing.

The allocator uses a garbage collector for locating unused memory. The collector is a generational, cooperative and non-compacting collector. The use of a garbage collector is somewhat unusual in a C program. However, garbage collection it is especially well suited for long running applications like a web server, as it eliminates most memory leaks. Unlike traditional memory allocation where free must be called to release memory, Appweb uses the opposite approach. Memory that must be retained, must be actively managed to prevent garbage collection. This means that a managed reference must be held for all active memory.

In practice there are thus two kinds of memory:

• Managed memory — This is memory allocated via the MPR which is actively managed by the garbage collector. The user is responsible for marking the memory as being active via the mprMark API. The collector will automatically reclaim the memory when it is no longer marked as being actively used.
• Malloc memory — This is memory allocated directly or indirectly by malloc, and is not managed by the garbage collector. The user is responsible for explicitly calling free() when the memory is no longer required. You must not call mprMark() on such memory.

Allocator

The allocator is optimized for frequent allocations of small blocks (< 4K). It uses a scheme of free queues for fast allocation. Memory allocations are aligned on 16 byte boundaries on 64-bit systems and otherwise on 8 byte boundaries. It will return chunks of unused memory back to the O/S.

Appweb 3 Allocator

Appweb 3 memory API required a memory context argument for all memory allocations. This required many APIs to have a memory context as the first parameter. Appweb 4 does not require this and these APIs have removed the memory context parameter.

Error Handling

It is difficult for programmers to consistently check the result of every API call that can fail due to memory allocation errors. Calls such as strdup and asprintf are often assumed to succeed, but they can, and do fail when memory is depleted.

A better approach is to proactively detect and handle memory allocation errors in one place. The MPR allocator handles memory allocation errors globally. It has has a configurable memory redline limit and a memory depletion policy handler. Appweb configures this memory limit so that memory depletion can be proactively detected and handled before memory allocations actually fail. When memory usage exceeds a pre-configured redline value, the depletion handler is invoked. The application can then determine what action to take. Typically, Appweb will restart in such circumstances.

Garbage Collection

The MPR garbage collector will run periodically to reclaim unused memory and potentially return that memory to the Operating System. The collector runs in its own thread, but is cooperative, in that each other thread must yield to the collector before memory can be reclaimed. Worker threads yield to the collector by calling the mprYield API. This strategy permits worker threads to allocate temporary memory without fear. The memory will not be prematurely collected until the worker explicitly acknowledges and yields to the collector by calling mprYield. Appweb will ensure threads call mprYield when waiting for I/O or when a request is complete. Users only need to explicitly call mprYield when they are doing a long-blocking operation.

To prevent collection of a memory block and retain the block over a yield point, the application must hold a managed reference for the block. A managed reference, is a reference to an allocated block that will be marked as active by mprMark() during a collection cycle by the parent block's manager function. Manager functions are defined when allocating blocks that will hold managed references. See below for more details.

Collection Phases

The collector reclaims memory in three phases: Wait, Mark and Sweep. The Wait phase waits for all threads to yield. This quiesces the system for reliable collection. NOTE: this does not mean that all request activity must cease. Rather, pre-determined rendezvous yield points are inserted in key locations in the Appweb HTTP processing engine.

The Mark phase traverses memory and marks all blocks that are currently being used. The Sweep phase finally reclaims all blocks that are not marked.

Marking Blocks

The Mark phase beings with a set of known root memory blocks. The ultimate root is the Mpr object returned from the mprCreate API. However, other roots can be added at any time via mprAddRoot. For each root, the collector invokes the manager function defined when the block was allocated. It is the responsibility of that manager function to call mprMark on every managed reference owned by the block. The mprMark function will then invoke the manager for these managed references and so on. In this manner, managed memory forms a tree from the roots to the leaves and the mark phase visits every managed block currently in use.

Allocating Memory

Managers are defined when allocating a block of memory. For example, this code will allocate a block that will contain a reference to a managed string and a reference to an un-managed malloc block.

typedef struct MyBlock
    char    *managedString;
    void    *privateMalloc;
} MyBlock;
MyBlock *blk = mprAllocObj(MyBlock, manageBlock);
blk->managedString = sclone("Hello World");
blk->privateMalloc = malloc(1024);

This will allocate a new structure and define manageBlock as the manager function for the structure. If you need to keep the allocated structure, you must ensure the blk reference will be marked during the garbage collector mark phase. To do this, you must ensure the reference is marked via mprMark during some other object's manager function. Alternatively, you can call mprAddRoot to specify that this reference is a top level root of a new memory tree. You should do this sparingly. It is more effective to mark the reference from another manager routine.

If you only require the allocated structure temporarily, you do not need to retain a reference or call mprAddRoot. In this manner, the memory will be automatically collected during the next garbage collection sweep because there will not be a managed (marked) reference to the block.

the mprAllocMem may be used to allocate a block of memory and reserve room for a manager function. Then use mprSetManager to define a manager function.

Managers

A manager function is invoked by the collector during each collector Mark phase of the collection cycle. The manager is passed a reference to the block and a flags word. The flags are set to either MPR_MANAGE_MARK during the Mark phase and to MPR_MANAGE_FREE if the collector determines the block is to be freed.

void manageBlock(MyBlock *blk, int flags) 
{
    if (flags & MPR_MANAGE_MARK) {
        mprMark(blk->managedString);
        /* Don't mark privateMalloc */
    } else if (flags & MPR_MANAGE_FREE) {
        /* Custom code when the block is freed */
    }
}

Note that it is safe to call mprMark with NULL reference. This is a convenient patter so you do not need to test if the element is null or not.

Convenient References

Appweb defines two empty fields that can be used by request handlers to hold managed references. The HttpConn.data field is marked by the HttpConn manager. A handler can store a managed-memory reference in this field. The HttpConn manager will then call mprMark(conn->data) to mark the reference as active and required.

Similarly, HttpQueue.queueData field is marked by the HttpQueue manager. A queue stage (filter or handler) can store a managed-memory reference in this field. The HttpQueue manager will then call mprMark(q->queueData) to mark the reference as active and required.

Appweb defines two fields that can be used to store unmanaged memory references: HttpConn.staticData and HttpQueue.staticData. Use these to store references to memory allocated by malloc.

Another common technique it to define a top level application structure which will be the root memory block for the entire application. Store top level managed references in this block and call mprAddRoot to define it as a root block.

Simple Rules

Here are some simple rules for allocating memory with Appweb and using the Garbage Collector.

Must Mark to Retain Managed Memory

Any memory allocated from the MPR must be marked to retain past the next garbage collection cycle.

Don't Mix Memory

You must not mix MPR allocated memory and non-managed memory. This means don't mark un-managed memory that has been allocated via malloc(). And you must not call free() on managed memory allocated from the MPR.

Define a Manager

If you allocate a managed structure that has references to managed memory, you should define a manager function that invokes mprMark on the structure elements that are managed references.