Memory Management

Most non-trivial programs needs to allocate and free memory. Memory management techniques become more and more important as programs increase in complexity, size, and performance.

Built-in types that allocate GC memory

D has built-in types that may be difficult to use without the GC: exceptions, strings, dynamic arrays, associative arrays, and delegate closures. See D Builtin Rationale.

Options for managing memory

The three primary methods for allocating memory in D are:

• Static data, allocated in the default data segment.
• Stack data, allocated on the CPU program stack.
• Garbage collected data, allocated dynamically on the garbage collection heap.

The techniques for using them, as well as some advanced alternatives are:

Strings (and Array) Copy-on-Write

Consider the case of passing an array to a function, possibly modifying the contents of the array, and returning the modified array. As the contents of an array are accessed through a reference, a crucial issue is who owns the contents of the array? For example, a function to convert an array of characters to upper case:

char[] toupper(char[] s)
{
    int i;

    for (i = 0; i < s.length; i++)
    {
        char c = s[i];
        if ('a' <= c && c <= 'z')
            s[i] = cast(char)(c - ('a' - 'A'));
    }
    return s;
}

Note that the caller's version of s[] is also modified. This may be not at all what was intended, or worse, s[] may be a slice into a read-only section of memory.

If a copy of s[] was always made by toupper(), then that will unnecessarily consume time and memory for strings that are already all upper case.

The solution is to implement copy-on-write, which means that a copy is made only if the string needs to be modified. Some string processing languages do do this as the default behavior, but there is a huge cost to it. The string "abcdeF" will wind up being copied 5 times by the function. To get the maximum efficiency using the protocol, it'll have to be done explicitly in the code. Here's toupper() rewritten to implement copy-on-write in an efficient manner:

char[] toupper(char[] s)
{
    int changed;
    int i;

    changed = 0;
    for (i = 0; i < s.length; i++)
    {
        char c = s[i];
        if ('a' <= c && c <= 'z')
        {
            if (!changed)
            {
                char[] r = new char[s.length];
                r[] = s;
                s = r;
                changed = 1;
            }
            s[i] = cast(char)(c - ('a' - 'A'));
        }
    }
    return s;
}

Copy-on-write is the protocol implemented by array processing functions in the D Phobos runtime library.

Real Time

Real-time programming means that a program must be able to guarantee a maximum latency, or time to complete an operation. With most memory allocation schemes, including malloc/free and garbage collection, the latency is theoretically not bound. The most reliable way to guarantee latency is to preallocate all data that will be needed by the time critical portion. If no calls to allocate memory are done, the GC will not run and so will not cause the maximum latency to be exceeded.

It is possible to create a real-time thread by detaching it from the runtime, marking the thread function @nogc, and ensuring the real-time thread does not hold any GC roots. GC objects can still be used in the real-time thread, but they must be referenced from other threads to prevent them from being collected.

Smooth Operation

Related to real time programming is the need for a program to operate smoothly, without arbitrary pauses while the garbage collector stops everything to run a collection. An example of such a program would be an interactive shooter type game. Having the game play pause erratically, while not fatal to the program, can be annoying to the user.

There are several techniques to eliminate or mitigate the effect:

• Preallocate all data needed before the part of the code that needs to be smooth is run.
• Manually run a GC collection cycle at points in program execution where it is already paused. An example of such a place would be where the program has just displayed a prompt for user input and the user has not responded yet. This reduces the odds that a collection cycle will be needed during the smooth code.
• Call core.memory.GC.disable() before the smooth code is run, and core.memory.GC.enable() afterwards. This will cause the GC to favor allocating more memory instead of running a collection pass.

Free Lists

Free lists are a great way to accelerate access to a frequently allocated and discarded type. The idea is simple - instead of deallocating an object when done with it, put it on a free list. When allocating, pull one off the free list first.

class Foo
{
    static Foo freelist;		// start of free list

    static Foo allocate()
    {   Foo f;

	if (freelist)
	{   f = freelist;
	    freelist = f.next;
	}
	else
	    f = new Foo();
	return f;
    }

    static void deallocate(Foo f)
    {
	f.next = freelist;
	freelist = f;
    }

    Foo next;		// for use by FooFreeList
    ...
}

void test()
{
    Foo f = Foo.allocate();
    ...
    Foo.deallocate(f);
}

Such free list approaches can be very high performance.

• If used by multiple threads, the allocate() and deallocate() functions need to be synchronized.
• The Foo constructor is not re-run by allocate() when allocating from the free list, so the allocator may need to reinitialize some of the members.
• It is not necessary to practice RAII with this, since if any objects are not passed to deallocate() when done, because of a thrown exception, they'll eventually get picked up by the GC anyway.

Reference Counting

The idea behind reference counting is to include a count field in the object. Increment it for each additional reference to it, and decrement it whenever a reference to it ceases. When the count hits 0, the object can be deleted.

D does not provide automated support for reference counting.

RefCounted provides limited support for reference counting; not for class types. Unlike C++ shared_ptr, it does not support multi-threading, weak pointers, or destructors. (See this forum thread.)

Win32 COM programming uses the members AddRef() and Release() to maintain the reference counts.

Explicit Class Instance Allocation

Note: Class allocators and deallocators are deprecated in D2, but the following example provides an alternative.

D provides a means of creating custom allocators and deallocators for class instances. Normally, these would be allocated on the garbage collected heap, and deallocated when the collector decides to run. For specialized purposes, this can be handled by creating New Declarations (heapAllocate in the example below) and Delete Declarations(heapDeallocate in the example below). For example, to allocate using the C runtime library's malloc and free:

import std.stdio;
 
class TestClass 
{
    int x;
    
    this(int x) 
    {
        writeln("TestClass's constructor called");
        this.x = x;
    }
    
    ~this() 
    {
        writeln("TestClass's destructor called");
    }
}      
 
T heapAllocate(T, Args...) (Args args) 
{
    import std.conv : emplace;
    import core.stdc.stdlib : malloc;
    import core.memory : GC;
    
    // get class size of class instance in bytes
    auto size = __traits(classInstanceSize, T);
    
    // allocate memory for the object
    auto memory = malloc(size)[0..size];
    if(!memory)
    {
        import core.exception : onOutOfMemoryError;
        onOutOfMemoryError();
    }                    
    
    writeln("Memory allocated");

    // notify garbage collector that it should scan this memory
    GC.addRange(memory.ptr, size);
    
    // call T's constructor and emplace instance on
    // newly allocated memory
    return emplace!(T, Args)(memory, args);                                    
}
 
void heapDeallocate(T)(T obj) 
{
    import core.stdc.stdlib : free;
    import core.memory : GC;
    
    // calls obj's destructor
    destroy(obj); 

    // garbage collector should no longer scan this memory
    GC.removeRange(cast(void*)obj);
    
    // free memory occupied by object
    free(cast(void*)obj);
    
    writeln("Memory deallocated");
}
       
void main() 
{
    // allocate new instance of TestClass on the heap
    auto test = heapAllocate!TestClass(42);
    scope(exit)
    {
        heapDeallocate(test);    
    }
    
    writefln("test.x = %s", test.x);
}

Output:

Memory allocated 
TestClass's constructor called 
test.x = 42 
TestClass's destructor called 
Memory deallocated

The critical features of heapAllocate are:

• A null is not returned if storage cannot be allocated. Instead, an exception is thrown. Which exception gets thrown is up to the programmer, in this case, onOutOfMemoryError() is called to throw an OutOfMemoryError exception.
• When scanning memory for root pointers into the garbage collected heap, the static data segment and the stack are scanned automatically. The C heap is not. Therefore, if Foo or any class derived from Foo using the allocator contains any references to data allocated by the garbage collector, the GC needs to be notified. This is done with the std.gc.addRange() method.

The critical features of heapDeallocate are:

• The destructor (if any) is executed with a call to destroy, so the data obj points to should be assumed to be garbage.
• If the GC was notified with std.gc.addRange(), a corresponding call to std.gc.removeRange() must happen in the deallocator.
• If there is a heapDeallocate, there should be a corresponding heapAllocate.

If memory is allocated using this method, careful coding practices must be followed to avoid memory leaks and dangling references. In the presence of exceptions, it is particularly important to practice RAII to prevent memory leaks.

Custom allocators and deallocators can be done for structs and unions, too.

Mark/Release

Mark/Release is equivalent to a stack method of allocating and freeing memory. A ‘stack’ is created in memory. Objects are allocated by simply moving a pointer down the stack. Various points are ‘marked’, and then whole sections of memory are released simply by resetting the stack pointer back to a marked point.

import std.stdio;

void[] buffer;
size_t currentIndex;
const size_t bufferSize = 100;

// Static constructor will be called before main to allocate the buffer memory
static this()
{
    import core.stdc.stdlib : malloc;
    import core.memory : GC;
    
    writeln("Allocating buffer memory");
    
    auto p = malloc(bufferSize);
    
    if (p is null)
    {
        import core.exception : onOutOfMemoryError;
        onOutOfMemoryError();
    }
    
    // notify garbage collector that it should scan this memory
    GC.addRange(p, bufferSize);

    buffer = p[0 .. bufferSize];
}

// Static destructor will be called after leaving main to deallocate buffer memory
static ~this()
{
    if (buffer.length)
    {
        import core.stdc.stdlib : free;
        import core.memory : GC;

        // garbage collector should no longer scan this memory
        GC.removeRange(buffer.ptr);

        free(buffer.ptr);
        buffer = null;
    }
    
    writeln("Deallocated buffer memory");
}

// remember where to return to when memory is released
size_t mark()
{
    writefln("Marked at %s", currentIndex);
    return currentIndex;
}

// release the memory by returning to where it was marked
void release(size_t markedIndex)
{
    currentIndex = markedIndex;
    
    writefln("Released memory back to %s", currentIndex);
}

// Reserve memory for the object, and instantiate it
T create(T, Args...) (Args args) 
{
    import std.conv : emplace;
    
    writefln("Reserving memory for %s", T.stringof);
    
    // get class size of class instance in bytes
    auto size = __traits(classInstanceSize, T);
    
    // check if there's enough room in the buffer
    auto newIndex = currentIndex + size;
    if (newIndex >= buffer.length)
    {
        import core.exception : onOutOfMemoryError;
        onOutOfMemoryError();
    }
    
    // get location where new instance will be emplaced
    auto memory = buffer[currentIndex..newIndex];
    
    // reserve the memory by advancing the index
    currentIndex = newIndex;
    
    // call T's constructor and emplace instance on
    // newly reserved memory
    return emplace!(T, Args)(memory, args);
}

class TestClass 
{
    int x;
 
    this(int x) 
    {
        writeln("TestClass's constructor called");
        this.x = x;
    }
 
    ~this() 
    {
        writeln("TestClass's destructor called");
    }
} 

void main()
{
    writeln("Entered main");
    
    // Remember where to return to when memory is released
    size_t markedIndex = mark();
    scope(exit)
    {
        release(markedIndex);   
    }
    
    // Instantiate new class on the buffer
    auto test = create!TestClass(42);
    scope (exit)
    {
        // be sure to call its destructor
        destroy(test);
    }
    
    writefln("test.x = %s", test.x);
    
    writeln("Leaving main");
}

Output:

Allocating buffer memory 
Entered main 
Marked at 0 
Reserving memory for TestClass 
TestClass's constructor called 
test.x = 42 
Leaving main 
TestClass's destructor called 
Released memory back to 0 
Deallocated buffer memory

The allocation of buffer[] itself is added as a region to the GC, so there is no need for a separate call when instantiating TestClass.

RAII (Resource Acquisition Is Initialization)

RAII techniques can be useful in avoiding memory leaks when using explicit allocators and deallocators. Adding the scope attribute to class instances will allocate them on the stack, potentially avoiding such issues.

Allocating Class Instances On The Stack

Class instances are normally allocated on the garbage collected heap. However, if they:

• are allocated as local symbols in a function and
• are allocated using new and
• use new with no arguments (constructor arguments are allowed) and
• have the scope storage class

then they are allocated on the stack. This is more efficient than doing an allocate/free cycle on the instance. But be careful that any reference to the object does not survive the return of the function.

class C { ... }

scope c = new C();	// c is allocated on the stack
scope c2 = new C(5);    // allocated on stack
scope c3 = new(5) C();  // allocated by a custom allocator

If the class has a destructor, then that destructor is guaranteed to be run when the class object goes out of scope, even if the scope is exited via an exception.

Allocating Uninitialized Arrays On The Stack

Arrays are always initialized in D. So, the following declaration:

void foo()
{   byte[1024] buffer;

    fillBuffer(buffer);
    ...
}

will not be as fast as it might be since the buffer[] contents are always initialized. If careful profiling of the program shows that this initialization is a speed problem, it can be eliminated using a VoidInitializer:

void foo()
{   byte[1024] buffer = void;

    fillBuffer(buffer);
    ...
}

Uninitialized data on the stack comes with some caveats that need to be carefully evaluated before using:

• The uninitialized data that is on the stack will get scanned by the garbage collector looking for any references to allocated memory. Since the uninitialized data consists of old D stack frames, it is highly likely that some of that garbage will look like references into the GC heap, and the GC memory will not get freed. This problem really does happen, and can be pretty frustrating to track down.
• It's possible for a function to pass out of it a reference to data on that function's stack frame. By then allocating a new stack frame over the old data, and not initializing, the reference to the old data may still appear to be valid. The program will then behave erratically. Initializing all data on the stack frame will greatly increase the probability of forcing that bug into the open in a repeatable manner.
• Uninitialized data can be a source of bugs and trouble, even when used correctly. One design goal of D is to improve reliability and portability by eliminating sources of undefined behavior, and uninitialized data is one huge source of undefined, unportable, erratic and unpredictable behavior. Hence this idiom should only be used after other opportunities for speed optimization are exhausted and if benchmarking shows that it really does speed up the overall execution.

Interrupt Service Routines

When the garbage collector does a collection pass, it must pause all running threads in order to scan their stacks and register contents for references to GC allocated objects. If an ISR (Interrupt Service Routine) thread is paused, this can break the program.

Therefore, the ISR thread should not be paused. Threads created with the core.thread functions will be paused but threads created with C's _beginthread() or equivalent won't be. The GC won't know they exist.

For this to work successfully:

• The ISR thread cannot allocate any memory using the GC. This means that the global new cannot be used. Nor can dynamic arrays be resized, nor can any elements be added to associative arrays. Any use of the D runtime library should be examined for any possibility of allocating GC memory - or better yet, the ISR should not call any D runtime library functions at all.
• The ISR cannot hold the sole reference to any GC allocated memory, otherwise the GC may free the memory while the ISR is still using it. The solution is to have one of the paused threads hold a reference to it too, or store a reference to it in global data.

Writing GC free code

As of D release version 2.066, it is possible to disallows GC-heap allocation in code sections, by marking them with the @nogc attribute.

@nogc void foo(char[] a)
{
    auto p = new int;  // error, operator new allocates
    a ~= 'c';          // error, appending to arrays allocates
    bar();             // error, bar() may allocate
}

void bar() { }

This allows the compiler to enforce that the GC heap is not going to be used.

Array concatenation, dynamic closures, calls to new operator and calls to functions not marked as @nogc are not allowed inside functions marked as @nogc.

Work is in progress to make the D standard library, Phobos, usable without any use of the garbage collector. Eventually Phobos will be fully @nogc compliant.

Locating GC Allocations

As of release 2.066, the compiler switch -vgc provides a mechanism to locate GC heap allocations. When enabled, the compiler will list locations in the source code where GC heap allocations are being performed. The following example illustrates this.

// main.d
module main;

class TestClass
{ }

void main(string[] args)
{
    auto c = new TestClass();
}

Compile with:

dmd -vgc main.d

Output:

test.d(9): vgc: 'new' causes GC allocation

Note, however, that this is not a compile-time error; the source code still compiles to an executable.

Composable Memory Allocators

There is work in progress to add composable memory allocators as part of Phobos.

A peek at the current state of implementation is possible at:

https://github.com/D-Programming-Language/phobos/tree/master/std/experimental (Source code)

http://dlang.org/phobos/std_experimental_allocator.html (Documentation)

External links

• Some other GC related hacks
• Don’t Fear the Reaper - A Basic Introduction to the GC
• Life in the Fast Lane - Introduction to Avoiding the GC and GC Profiling
• Go Your Own Way (Part One: The Stack)
• Go Your Own Way (Part Two: The Heap)
• Memory Safety