[mkgmap-dev] splitter: option for maximum tile area?

Fri Nov 2 18:18:34 GMT 2018

Hi all,
still struggling with that strange issue. But I guess I found some hint 
to the cause: inconsistent file sizes.
- extracted OSM data: 400 MB pbf
- elevation contour lines: 600 MB pbf
- merged file: 1 GB pbf
So far, sizes are consistent.

When I run splitter on the OSM data only file, it produces many tiles 
summing up to some 400 MB. Consistent, too. When I run mkgmap on this 
output, I get a map within about half an hour, and it looks OK (tested 
with QLandkarte).

When I run splitter on the merged file, the tiles sum up to some 9 GB. 
That is 9 times the size I'd expect. mkgmap can render several of those 
tiles (but very slowly); and then crashes on one of them with an 
OutOfMemory exception.

So I think that the problem is somewhere in those contour lines, either 
when merged or alone.
I'll try to create a contour lines only map as the next step to test 
this hypothesis.

Kind regards,
Bernhard

Am 01.11.2018 um 09:32 schrieb Gerd Petermann:
> Hi Bernhard,
>
> I tried to reproduce the memory problems with tile 47120005. I don't think that the sea itself is a problem here, at least not when you use the --precomp-sea option. It took only 15 secs to process that tile with asia data from August with the default style and typical options for routing etc.
> My input file didn't contain SRTM data so I assume this is the reason. Maybe you have many contour lines with ele=in your data?
>
> Gerd
>
> ________________________________________
> Von: Gerd Petermann <gpetermann_muenchen at hotmail.com>
> Gesendet: Mittwoch, 31. Oktober 2018 22:49
> An: Gerd Petermann; Development list for mkgmap
> Betreff: AW: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi Bernhard,
>
> looked again at the splitter command in your last post. You also use a rather high max-nodes value.
> Such a high value means that you get rather large tiles and that mkgmap needs more memory for each tile
> compared to the default 1400000. Many users use a value near 1200000.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen at hotmail.com>
> Gesendet: Mittwoch, 31. Oktober 2018 22:20
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi Bernhard,
> there is no option for this. Do you use option --precomp-sea in splitter? Maybe you use --no-trim ?
> If that doesn't help you can try to change the values for
> private static final int MAX_LAT_DEGREES =5;
> private static final int MAX_LON_DEGREES =0;
> in SplittableDensityArea.java and compile your own version of splitter.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Bernhard Hiller <bhil at gmx.de>
> Gesendet: Mittwoch, 31. Oktober 2018 22:03
> An: mkgmap-dev at lists.mkgmap.org.uk
> Betreff: [mkgmap-dev] splitter: option for maximum tile area?
>
> Hi all,
> currently a Java OutOfMemory exception prevents me from creating a map.
> I already use option --max-jobs=(the machine has 4 physical cores) and
> -Xmx5G (of 8 GB installed). Beyond OSM data, the map contains DEM and
> elevation contour lines.
>   From the tiles finished and those with a new timestamp but about 0
> bytes length, I can see that mkgmap was rendering tiles 47120005,
> 47120006, 47120007 at the time of crash.
> Tile 47120005 is extremely large by physical area - some 6° x 5.5° (see
> attached file), covering a lot of the south chinese sea, i.e. there are
> not many actual data in that area.
> I guess that the problem arises with that tile. I remember some case in
> the past where a single tile covering such a large area of mainly sea
> caused mkgmap to take an enormous amount of time for rendering - also
> here, mkgmap already spent about 1 hour before crashing.
> So I'd like to ask: is there some possibility for limiting the area of a
> tile among the splitter options?
> Kind regards,
> Bernhard
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>