Last update July 28, 2009

Suggested Features /
Array Reserve



NOTE:
This feature has been available for some time now.

This page is under construction. It's preparing a feature suggestion in NewsDmD perhaps Sep03 or later. -- HelmutLeitner

Table of contents of this page
The suggestion in short   
Costs and Gains   
Reasoning   
Effects on library design   

The suggestion in short    

Add a .reserve property to the basic D array type to ease runtime-efficient programming. reserve allows to handle arrays, that have allocated more space for elements than they hold according to their length. This would remove the need for the outbuffer module and similar attemps to gain performance.

The cost of 4 byte per array may look prohibitive but it shouldn't compared to existing overheads of 12-16 byte (in typical C malloc systems).

The overhead of an D array (estimated 16-20 bytes) should be measured empirically before making this suggestion.

Costs and Gains    

runtime space:

  • cost: 4 bytes for each array used
runtime:
  • cost: checks when changing length whether this needs a reallocation .
  • gain: if the check is negative (reserve > length + delta) no operation is necessary. This is a big advantage.
library space (this may add to runtime space):
  • cost: about 20-50 lines for the additional logic
  • gain: drop of the outbuffer module in Phobos (300 lines)
  • gain: simplifies any programming for efficient array size change (growth)

Reasoning    

It is common knowledge, that for writing fast code you have to avoiding all unnecessary memory allocations, reallocations and deallocations. This conflicts with OO practice because the creation and destruction of objects (und their object components) is made so easy and often happens unknow by the user. So programmers wanting to optimize they OO programs have to be aware of what is going on hidden by the language.

The inevitable need for performance often surfaces in special language features for efficient building of string from many small part:

  • JAVA: StrBuffer? object
  • D: OutBuffer? object
At the base they do what this C struct (rewritten from my library)

typedef struct genericbuffer {
  void *array_pointer;               
  unsigned int length;   /* current # of elements */
  unsigned int reserve;  /* reserved space, unit: elements */
  unsigned int elsize;   /* elementsize (byte) */
} GENERICBUFFER;

also does. It allows to manage allocated array size (.reserve) and number of elements (.length) separately.

tbc

Effects on library design    

A number of library functions could profit from the increased efficiency.

Example 1:

The reading of a file in a single junk is currently the fastest way to read files:

char [] data = (char []) read (filename);

Using existing and reuseable memory memory buffer this could be done faster:

char [] buffer;
buffer.reserve=100000;
bufferread(buffer,filename);

Using reserve the function bufferread has important advantages:

  • typically it needn't do any momory allocations.
  • it can give the correct file length back in buffer.length without affecting (treducing) the buffer size.
  • it can increase the buffer size if there should be a need.
  • it can use standard syntax ("~") for array operations internally in case that reading is not done in one operation.

FrontPage | News | TestPage | MessageBoard | Search | Contributors | Folders | Index | Help | Preferences | Edit

Edit text of this page (date of last change: July 28, 2009 0:06 (diff))