[mkgmap-dev] [PATCHv4]splitter memory usage

Mon Nov 14 17:56:03 GMT 2011

Hello Klaus,

> I'm a little bit confused about the changes of the changes.
> 
> >The default is now to assume a highest node id of 2^31 and resize when
> needed.
> 
> Does this mean that the 'old' limit is back ?

no, the deault is to allocate two arrays with 2^31 / 64 elements, that means 256Mbyte. Splitter version r181  typically allocates 8 array with ~ 1.5*2^30,  so that was ~800 MByte as a fixed part.
With the new parameter   --optimize-mem   the program doesn't even allocate these arrays, but uses hash maps. This parameter may help when you have only 1GB or less (e.g. with a netbook) and small input files.
BUT this is only the storage needed to address the so called "chunks". 
There is also a variable part that depends on the size of the input file and on the number of areas that are processed in one part.
A version that is also reducing the number of bytes needed for that variable is work in progress.

ciao,
Gerd

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20111114/9a50a69d/attachment.html 