[mkgmap-dev] Memory limits for mkgmap and splitter

Wed Aug 5 23:45:02 BST 2009

Aha, thank you, this was very helpful! I've been doing most of my testing 
with compressed osm files but I just did a few runs with an uncompressed 
file instead and I see what you mean.

The problem was due to some buffering I'd added when reading the osm file. 
It didn't make much difference for compressed files, but made the uncompressed 
reading a lot worse. Here are the times I get when I run the splitter using 
the same parameters and osm file as you:

old splitter: 117 secs
new splitter: 127 secs
new splitter (fixed buffering): 103 secs

I'm going to start getting this checked in to the codestore, but in the meantime 
you can grab a new copy from http://redyeti.net/splitter.jar

Chris

> I have run my tests on another machine (my first tests were made with
> a
> notebook with virus scanner, harddisk encryption ...):
> old splitter ( 51922 bytes): 196s
> new splitter (181951 bytes): 213s
> This is nearly the same performance (+8%).
> 
> Windows XP SP2 32Bit, Java 1.6.0_15
> Athlon 64 X2 4600+, 2 GB RAM
> No virus scanner, no harddisk encryption
> Data:
> http://download.geofabrik.de/osm/europe/germany/bayern.osm.bz2
> I've unzipped the file and called the splitter with these parameters:
> 
> java -Xmx1400M -ea -jar splitter.jar --mapid=1 --max-nodes=800000
> bayern.osm
> 
> I also made some tests with long ways in germany. With max-nodes
> 800000 I have 3 ways in more than 4 tiles, but they are not important
> (Oberpfalz polygons). With max-nodes 400000 I get 12 ways: 2 rivers
> (Rhein, Mosel), 5 power lines and some unimportant ways.
> 
> So if I do not use low max-nodes values I have no urgent problem with
> long ways in germany. Thanks for solving the 4 tiles restriction of
> relations, this was more important!
> 
> Rudi
> 