[mkgmap-dev] splitter r325: improved split algo and new option

Wed May 7 11:18:39 BST 2014

You can also save your country.o5m files from one use to the next and 
then use osmupdate to update them. This way you don't have to download 
from Geofabrik and convert from pbf to o5m.

El 07/05/14 11:59, Gerd Petermann escribió:
> Hi Felix,
>
> try o5m for both input and output, it is much faster.
> The command
> osmcovert --drop-version europe-latest.osm.pbf -o=europe.o5m
> runs quite fast (~70 seconds on my machine),
> the o5m file is ~2.430 MB, the pbf file has 2.104 MB.
>
> Splitter is much faster reading from o5m when
> the keep-complete option is in use.
> (210 secs for the o5m, 441 for pbf)
>
> With --output=pbf both are slower, and mkgmap is also much slower.
>
> All times with I/O on one normal hard disk. Even better results if you 
> have
> two different disks for reading and writing.
>
> Gerd
>
> ------------------------------------------------------------------------
> Date: Wed, 7 May 2014 11:37:58 +0200
> From: extremecarver at gmail.com
> To: mkgmap-dev at lists.mkgmap.org.uk
> Subject: Re: [mkgmap-dev] splitter r325: improved split algo and new 
> option
>
> Well I still use pbf and not o5m.
> First pbf is smaller..
> Second - Geofabrik only offers pbf - that's why I stayed with it.
>
> I don't think I can cut a lot of time by first converting to 05m, then 
> hand it over to splitter...
> Actually I also let splitter output pbf... Maybe I could change that 
> in future to 05m..
> On 07.05.2014 11:36, Gerd Petermann wrote:
>
>     Hi Felix,
>
>     well, nowadays splitter performance mostly depends on I/O if you
>     use o5m format
>     for input and output and give enough heap.
>
>     Reg. mkgmap performance improvements: yes, that's what I expected.
>     In short, the branch improved the evaluation of tags and the
>     creation of the NOD file.
>
>     Gerd
>
>
>     ------------------------------------------------------------------------
>     Date: Wed, 7 May 2014 11:29:10 +0200
>     From: extremecarver at gmail.com <mailto:extremecarver at gmail.com>
>     To: mkgmap-dev at lists.mkgmap.org.uk
>     <mailto:mkgmap-dev at lists.mkgmap.org.uk>
>     Subject: Re: [mkgmap-dev] splitter r325: improved split algo and
>     new option
>
>     Well - I'll update all my maps on Thursday again, to recheck.
>     Maybe it has to do with increasing-maxnodes? Though I thought the
>     higher the max-nodes, the faster...
>     And I only meant splitter. I upgraded mkgmap at the same time (now
>     integrating performance branch changes) - so mkgmap by itself got
>     faster (though it depends on the country - seems like well mapped
>     countries profit a lot more (e.g. Austria like 30% time off), than
>     countries where few continue commands will be in action cause
>     their mapping is basic like Asia).
>
>     I'm not using any pre-split files or cached files of any sort
>     either...
>     On 07.05.2014 06:49, Gerd Petermann wrote:
>
>         Hi Felix,
>
>         reg. speed: I can't reproduce that. I compared a split of
>         Germany,
>         both versions (r321 and r325) are more or less running the
>         same time.
>         (I've executed both programs two times to make sure that disk
>         caches
>         are not causing big differences)
>
>         Or did you mean the combination of splitter + mkgmap to
>         process e.g. Asia?
>
>         Gerd
>
>         ------------------------------------------------------------------------
>         Date: Tue, 6 May 2014 18:22:00 +0200
>         From: extremecarver at gmail.com <mailto:extremecarver at gmail.com>
>         To: mkgmap-dev at lists.mkgmap.org.uk
>         <mailto:mkgmap-dev at lists.mkgmap.org.uk>
>         Subject: Re: [mkgmap-dev] splitter r325: improved split algo
>         and new option
>
>         Seems to be much better now. I don't think I can increase the
>         max-nodes value though, but for most maps the new algo creates
>         less tiles for the same max-nodes value (e.g. Austria from 43
>         down to 35 for me, with the smallest tile now around 5MB
>         instead of 2.8, and the biggest 12MB instead of 11MB, for Asia
>         I simultaneously increased max-nodes from 800k to 900k- so I'm
>         down from 624 tiles to 493.... and size from 970KB-16MB to now
>         ). So it still seems to depend on the country, but it's
>         already a lot better...
>         It's a bit slower (about 10% more time)
>
>         On 06.05.2014 13:56, Gerd Petermann wrote:
>
>             Hi all,
>
>             I've applied num-tiles-v1.patch  and improved the split
>             algo, see
>             http://gis.19327.n5.nabble.com/mkgmap-ToDo-list-tp5803388p5805165.html
>
>
>             It is now less likely that splitter creates tiles with a
>             low number of
>             nodes, it is more likely that all tiles have nearly  the
>             same number of nodes,
>             and typically you will see fewer tiles.
>             Maybe this also means that you can increase the max-nodes
>             value.
>
>             I hope this also reduces the need for complex interactions
>             between
>             spltter and mkgmap.
>
>
>
>             Gerd
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20140507/497e2cba/attachment.html>