[mkgmap-dev] splitter memory usage

Sun Oct 30 10:20:19 GMT 2011

Hello list,

I think I found a way to enhance splitter memory usage.

I have now a version on my pc that is
- a bit faster (at least for my largest test data "germany.osm.pbf" it saves
~29% runtime: 725 sec. instead of 1009 sec (AMD Dualcode 5050e+4GB+Win-XP) 
- able to handle node ids up to 137.438.953.471L  (= 2^37-1) (from both xml
and osm.pbf as far as I can say)
- still less memory consuming. esp. for data with very high node ids

I achived that by using a long for the id values and by replacing the two
ArrayLists in SparseInt2ShortMapInline by an "Int2ObjectOpenHashMap" 
(fastutil) so that the HashMap stores objects like this:
	class Storer {
		private long chunkmask;
		private short [] chunk;
	} 
The meaning of these two fields is the same as in the existing code in
SparseInt2ShortMapInline. 

I did not test the change with a whole planet because my machine is too slow 
for that and I have problems with the download of the large file. 

Since the new code needs a different fastutil.jar, I can't simply add a
patch. If anybody is interested in testing with whole planet or reviewing my
code, what is the proprosed way to do this?

Ciao,
Gerd

--
View this message in context: http://gis.638310.n2.nabble.com/splitter-memory-usage-tp6935688p6945302.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.