[mkgmap-dev] tile takes very long time to generate

Mon Mar 8 18:17:42 GMT 2021

Hi.
I have uploaded the splitter output 
http://files.mkgmap.org.uk/download/502/77700003.osm.pbf

This is not the largest splitted pbf, but it generates the largest img-file.
Around 12 MB compared to most other tiles which end up being 4-8 MB.

I tested running mkgmap (--max-jobs=1) with this file only as input (with 
default style as well), it took 23 minutes before it was done, including any 
startup over head.

I then took one random other pbf file and run mkgmap with the same parameters, 
it took 48 seconds including startup and overhead.

I have a portion of data covering nordic and baltic countries, poland, chech, 
slovakia, austria, switserland, parts of germany and aslo greenland.

Splitter options:
java -Xmx2200m -jar ${SPLITTER} \
--output-dir=./splitted/ \
--mapid=77700001 \
--max-nodes=1275000 \
--no-trim \
kartdata/nord_ost.o5m 

mkgmap options:
java -Xmx2800m -jar ${MKGMAP} \
--max-jobs \
--family-name="some name" \
--family-id=7770 \
--mapname=77710001 \
--draw-priority=20 \
--latin1 \
--net --route --index --split-name-index \
--housenumbers \
--add-boundary-nodes-at-admin-boundaries=2 \
--output-dir=./tiles_7771/ \
--bounds=bounds/ \
--precomp-sea=sea/ \
--style-file=styles_7771/ \
--description="some name" \
--add-pois-to-areas \
--lower-case \
./splitted/777*.pbf

is there some other tool or some options i can give to java runtime instead of 
using virualvm to see the amount of time/cpu spent in various sections of the 
program?

Regards
Karl

On måndag 8 mars 2021 kl. 07:31:44 CET Gerd Petermann wrote:
> Hi Karl,
> 
> I don't think that the order of the tiles should make a big difference. It
> is quite normal that some tiles take longer than others, but 30 minutes is
> far too long. One well known reason for slow processing are very complex
> multipolygons like Lake Huron. Another might be huge areas of sea. So, to
> find out if the order or the content is causing the delay you can process
> the tiles in a different order. I often use VisualVM to monitor a running
> mkgmap, it helps to find out which routines are called most often. If you
> identify a single tile that takes very long (also with the default style)
> you can upload that tile to http://files.mkgmap.org.uk/ Best is to add a
> small file with the options that were used, but you can also post them
> here. Hope that helps.
> 
> Gerd
> 
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von 7770
> <7770 at foskan.eu> Gesendet: Montag, 8. März 2021 06:53
> An: mkgmap-dev at lists.mkgmap.org.uk
> Betreff: [mkgmap-dev] tile takes very long time to generate
> 
> Hi.
> 
> I observe a situation that one of the very first tiles (often the first or
> the third) that mkgmap generates takes 20 - 30 minutes to generate, wheres
> the others take about 30 seconds each.
> 
> The map data produced by splitter is a total of around 700 files with
> splitter option --max-nodes=1275000.
> 
> sea and bouds are used for mkgmap.
> 
> At first i thought i am running low on memory but changing to max-jobs=1
> (instead of the possible max of 2) did not make any change.
> 
> Is mkgmap doing something specific in the beginning that can explain this
> long time generating one of the very first tiles?
> 
> I can provide more details, let me know how i can best collect those details
> in case i need to set some parameters to have mkgmap to say more about what
> it is doing.
> 
> Regards
> Karl
> 
> 
> 
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev