[mkgmap-dev] tile takes very long time to generate

Tue Mar 9 08:20:16 GMT 2021

Hi Karl,

There are several ways to measure performance  of java programs, but the options depend on the used JRE. You can search the web for "java performance command line".

I think in your file the multipolygon relation (MP) 9488835 ("name:en"="Labrador Sea") is the problem,. I can reproduce slow processing without any options. The MP has ~25.000 members. The "physical" tag is place=sea and it contains thousands of islands. This requires a lot of computational power.

One question here is why this happens as the default style doesn't render place=sea.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von 7770 <7770 at foskan.eu>
Gesendet: Montag, 8. März 2021 19:17
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] tile takes very long time to generate

Hi.
I have uploaded the splitter output
http://files.mkgmap.org.uk/download/502/77700003.osm.pbf

This is not the largest splitted pbf, but it generates the largest img-file.
Around 12 MB compared to most other tiles which end up being 4-8 MB.

I tested running mkgmap (--max-jobs=1) with this file only as input (with
default style as well), it took 23 minutes before it was done, including any
startup over head.

I then took one random other pbf file and run mkgmap with the same parameters,
it took 48 seconds including startup and overhead.

I have a portion of data covering nordic and baltic countries, poland, chech,
slovakia, austria, switserland, parts of germany and aslo greenland.

Splitter options:
java -Xmx2200m -jar ${SPLITTER} \
--output-dir=./splitted/ \
--mapid=77700001 \
--max-nodes=1275000 \
--no-trim \
kartdata/nord_ost.o5m

mkgmap options:
java -Xmx2800m -jar ${MKGMAP} \
--max-jobs \
--family-name="some name" \
--family-id=7770 \
--mapname=77710001 \
--draw-priority=20 \
--latin1 \
--net --route --index --split-name-index \
--housenumbers \
--add-boundary-nodes-at-admin-boundaries=2 \
--output-dir=./tiles_7771/ \
--bounds=bounds/ \
--precomp-sea=sea/ \
--style-file=styles_7771/ \
--description="some name" \
--add-pois-to-areas \
--lower-case \
./splitted/777*.pbf

is there some other tool or some options i can give to java runtime instead of
using virualvm to see the amount of time/cpu spent in various sections of the
program?

Regards
Karl

On måndag 8 mars 2021 kl. 07:31:44 CET Gerd Petermann wrote:
> Hi Karl,
>
> I don't think that the order of the tiles should make a big difference. It
> is quite normal that some tiles take longer than others, but 30 minutes is
> far too long. One well known reason for slow processing are very complex
> multipolygons like Lake Huron. Another might be huge areas of sea. So, to
> find out if the order or the content is causing the delay you can process
> the tiles in a different order. I often use VisualVM to monitor a running
> mkgmap, it helps to find out which routines are called most often. If you
> identify a single tile that takes very long (also with the default style)
> you can upload that tile to http://files.mkgmap.org.uk/ Best is to add a
> small file with the options that were used, but you can also post them
> here. Hope that helps.
>
> Gerd
>
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von 7770
> <7770 at foskan.eu> Gesendet: Montag, 8. März 2021 06:53
> An: mkgmap-dev at lists.mkgmap.org.uk
> Betreff: [mkgmap-dev] tile takes very long time to generate
>
> Hi.
>
> I observe a situation that one of the very first tiles (often the first or
> the third) that mkgmap generates takes 20 - 30 minutes to generate, wheres
> the others take about 30 seconds each.
>
> The map data produced by splitter is a total of around 700 files with
> splitter option --max-nodes=1275000.
>
> sea and bouds are used for mkgmap.
>
> At first i thought i am running low on memory but changing to max-jobs=1
> (instead of the possible max of 2) did not make any change.
>
> Is mkgmap doing something specific in the beginning that can explain this
> long time generating one of the very first tiles?
>
> I can provide more details, let me know how i can best collect those details
> in case i need to set some parameters to have mkgmap to say more about what
> it is doing.
>
> Regards
> Karl
>
>
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev