[mkgmap-dev] splitter r254

Thu Dec 13 14:39:14 GMT 2012

I  used max-nodes=2 300 000
so it ends up with 143 tiles.

255 didn't give significant improvements (maybe 20 seconds).

I will probably run splitter now with 1024 max-areas. Hope 8.4GB heap 
will be enough for it..
It would probably be good if the splitter assumes +50% or whatever you 
seem reasonable for max memory on areas for the first pass..
I will give 2048 max-areas a try too, just to see if I can split Europe 
in 1 pass. If it works, I'll stick with it. Still need to wait for the 
2GB additional Ram for my server though...
On 13.12.2012 10:12, Gerd Petermann wrote:
> Hi Felix,
>
> thanks for the tests. I am quite happy with your results.
> In the meantime I have fixed some memory leaks.
> Could you please tell whether you use the default of max-areas=255 or 
> a higher value?
>
> The reason why I ask:
> When spliting france with max-nodes=1000000 you'll get > 255 areas,
> and 255 is the default for max-areas.
> Splitter uses the max-areas value for the problem-list-generator AND 
> for the final distribution pass.
> Both use the same data structures to store informations, but the 
> distribution pass requires
> additional heap for the buffers allocated for each open output file, 
> and with
> a large number of areas and pbf or o5m output these additional buffers 
> can
> require > 1 GB.
> I try to find a good algo to decide how many passes are needed in the
> problem-list generator. I am pretty sure that most users have enough
> heap to run this phase with a single read pass.
>
> Gerd
>
> > Date: Tue, 11 Dec 2012 14:23:57 +0100
> > From: extremecarver at gmail.com
> > To: mkgmap-dev at lists.mkgmap.org.uk
> > Subject: Re: [mkgmap-dev] splitter r254
> >
> > Well, I just ran a couple of tests with it. On france.osm.pbf splitting
> > takes 20minutes with --keep-complete, vs 12 minutes with 
> --overlap=4000.
> > On the other hand, mkgmap needs about 4 minutes less (albeit consuming
> > more memory). Currently on France and a few other countries, the added
> > memory consumption means, I have to use 3 processes for mkgmap instead
> > of 4 (the comparison times were with 3) - I'll spec up to 10GB RAM,
> > which should then be enough for 4 processes parallel. Overall using
> > keep-complete instead of overlap=4000, will addup a bit of time, but 
> not
> > much - as mkgmap runs a bit quicker (of course compared to 
> overlap=2000,
> > mkgmap runs not much quicker, as the bigger the overlap, the bigger the
> > time increase for mkgmap)
> >
> > For other countries results were similar.
> >
> > The new algo to avoid very small tiles is working great. France.osm.pbf
> > from Geofabrik should lock up / BSOD any Garmin GPS devices any more.
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

-- 
keep on biking and discovering new trails

Felix
openmtbmap.org & www.velomap.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20121213/a1cf7ddb/attachment.html 