[mkgmap-dev] Splitter r483 in refactoring2 branch

Tue Dec 6 14:13:33 GMT 2016

Hi Arndt,

I've corrected one error in r488 which might cause trouble in your situation.

The keep-complete option is enabled by default, it makes sure that the data written to each file is complete.

This requires more memory and more read passes but also avoids several problems like incomplete multipolygons

or route=ferry ways.

With keep-complete=false splitter simply calculates an enlarged bbox for each tile (by default 2000 map units)

and writes all data that falls into this enlarged area.

The 2nd effect of keep-complete=false is that splitter doesn't care about the order of the data in the input files.

With keep-complete=true the order is important, at least the node ids must appear in ascending order .

So, if you compile a map for Europe + SRTM data you probably need at least -Xmx7G and
the SRTM file must have ids which are higher than the ones in the OSM file
In your scripts I see
--start-node-id=10000000000 --start-way-id=10000000000
for phyghtmap, so that 2nd point should be okay.

See also the general tuning hints for splitter:
http://wiki.openstreetmap.org/wiki/Mkgmap/help/splitter#Tuning

Before you try again with the latest version in the branch please post a link to the (zipped) log file prodcued by splitter r483
(the one that crashed), I might have more advices.

Gerd

________________________________
Von: Arndt Röhrig <arndt at speichenkarte.de>
Gesendet: Dienstag, 6. Dezember 2016 14:42
An: Development list for mkgmap; Gerd Petermann
Betreff: Re: [mkgmap-dev] Splitter r483 in refactoring2 branch

Hi @ all,

r483 don´t start, if the option "keep-complete=false" is set.

Without that option, splitter crashes after a long time. (Europa: splitter run 4 hours before it stops)

I use OSM and SRTM Data. Maybe the SRTM Data is the reason? I dont´t really know, what "keep-complete=false" do.

With little Maps like "Bayern" r483 works well.

Greetings

Arndt

Gerd Petermann <GPetermann_muenchen at hotmail.com> hat am 2. Dezember 2016 um 17:21 geschrieben:

Hi all,

I think it is ready for a first public test .

Improvements:

1) Faster (esp. with large files), mostly because of an additional thread for OSM reade.

If you are interested, please play with the max-threads value. On my machine with a core i5 I see better results with max-threads=6

than with the default ("4 auto").

2) Splitting planet with a rather small max-nodes value like 800000 now stores ~ 41000 of possible 65535 entries,

with r442 I saw > 58.000. If you reach a value of > 60000  in one of the "number of area dictionary entries: xxxxx of 65535"

please contact the list. This limit may be increased to ~ 2^31 but with a larger memory footprint.

3) Some optimizations in the memory footprint of the custom maps may allow higher max-areas values.

I've changed the default for max-areas from 512 to 2048, this should work with typical PCs.

See http://www.mkgmap.org.uk/download/splitter.html

Gerd

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20161206/209337bc/attachment.html>