[mkgmap-dev] mkgmap splitter - Liechtenstein - horrible split if --polygon-file option used.

Wed Jun 23 15:26:53 BST 2021

Hi Felix,
splitter reports when it's not happy with the split result. Sometimes it gives hints how to improve it.

If you find "Solution is not nice. " in the log you should try to find different options.
Sometimes it helps to increase or decrease --max-nodes a little bit, sometimes it helps to use a polygon, sometimes it helps to use a large search-limit value = 10000000

In case of Brazil the higher search-limit helps. The effect of the polygon-file is that some nodes in the OSM file are ignored because they are outside of the polygon.
So, just an edge case and the reported numbers are a bit misleading, as splitter really "solves" a split with fewer nodes.
Will try to improve that...

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Felix Hartmann <extremecarver at gmail.com>
Gesendet: Mittwoch, 23. Juni 2021 15:38
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] mkgmap splitter - Liechtenstein - horrible split if --polygon-file option used.

sorry - I thought after your previous reply that this behaviour is to be expected - but as the differences are so big I wanted to post it here so others are warned. I did not log those other files anywhere and usually use >NUL 2>NUL because I want to keep the terminal clean - if splitter crashes on a split there are usually 0byte files produced - so I just check from them. Same for mkgmap. All those error messages I only look at if there is actually something broken. Else they are simply too much when compiling 180 extracts in 4 versions every week.

On Wed, 23 Jun 2021 at 16:33, Gerd Petermann <gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>> wrote:
Hi Felix,

why don't you keep the extra files produced by splitter somewhere for debugging? They are rather small and make it much easier to understand what happened (at least for me).

Anyhow, no need to run something again for now.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com>>
Gesendet: Mittwoch, 23. Juni 2021 15:25
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] mkgmap splitter - Liechtenstein - horrible split if --polygon-file option used.

No I did not log them - I only used a for loop to get the number of tiles. Brazil, Australia-Ocieania, Asia those differences are really so big that it is hard to believe mkgmap cannot do better. Options otherwise identical. Maxnodes sometimes a bit less than 140... If this is important I can write down for which countries they differed - but I think this happens at any maxnode value.

If you would like me to run all of them again and do a splitter.log I can. Will take 12 hours or so however. I am pretty sure all those splits which are vastly different in size have an error message. But I did not check that. Africa and Italy are quite stunning too. Why is it soo much less tiles not using a boundary file (I downloaded the extracts at the same time as the bound files - so they should be identical if geofabrik has no error).

On Wed, 23 Jun 2021 at 15:58, Gerd Petermann <gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>> wrote:
Hi Felix,

I can reproduce the surprising results for Brazil with --max-nodes=1400000. This really looks like an error in splitter because splitter should simply do the same after reporting
"Warning: shape is too complex, using rectangle (-35.5078125,-74.091796875) to (5.537109375,-27.6416015625) with 213.392.248 nodes instead"

But the search for a good split shows slightly different numbers and the log without polygon file says "Solution is not nice. Can't find a better solution ..."
I suggest to look for this message.

I'll try to find out why exactly splitter behaves like that.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>>> im Auftrag von Gerd Petermann <gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>>
Gesendet: Mittwoch, 23. Juni 2021 13:49
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] mkgmap splitter - Liechtenstein - horrible split if --polygon-file option used.

Hi Felix,

quite interesting. Do you have the log files from splitter and /or the densities-out.txt files? If yes, please share them.
Did you always use the same splitter options?

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>>> im Auftrag von Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>
Gesendet: Mittwoch, 23. Juni 2021 13:12
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] mkgmap splitter - Liechtenstein - horrible split if --polygon-file option used.

Okay - well I ran splitter against all geofrabrik extracts that I am having on my server (so nearly all - notably excluding DACH and Greater Britain extract) - and wrote a script comparing the splits. Overall because of Asia continent map it is better to use polygon-file. Seeing those huge differences on Asia, australia-oceania, Italy, Africa, Japan and Brazil
I do feel splitter should be doing a bit better. Especially Asia continent is hard to understand the huge huge difference.
In general for tiny countries - do not use it. Do not use it for German Bundesländer. Do not use polygon-file for Italy. But do use it on Australia-Oceania, Africa, Japan, Brazil, and Asia continent map.

Now I wonder - will this change in the future? So meaning I have to run this assessment once a year? Usually the additional tiles are mostly tiny - there is no reason for having them. If the split results in the same amount of tiles no matter using --polygon-file or not - it is not listed (overall 176 extracts). I put those countries with huge difference in bold.
26 splits are better not using --polygon-file. 30 splits are better using --polygon-file. 906 tiles less for Asia continent shows that splitting Asia without --polygon-file is a right mess. On the other hand for some tiny countries like Lichtenstein or German Bundesländer, and Africa using a --polygon-file from geofabrik created a way too many useless tiles.
cnt1 = using --polygon-file, cnt0=not using polygon-file for splitting.

"for montenegro do not use polygon-file - cnt1 = 2 cnt0 = 7"
"for france use polygon-file - cnt1 = 336 cnt0 = 332"
"for netherlands do not use polygon-file - cnt1 = 91 cnt0 = 92"
"for germany use polygon-file - cnt1 = 259 cnt0 = 258"
"for liechtenstein do not use polygon-file - cnt1 = 1 cnt0 = 6"
"for monaco do not use polygon-file - cnt1 = 1 cnt0 = 2"
"for slovenia do not use polygon-file - cnt1 = 27 cnt0 = 28"
"for ukraine use polygon-file - cnt1 = 62 cnt0 = 61"
"for norway use polygon-file - cnt1 = 128 cnt0 = 115"
"for switzerland do not use polygon-file - cnt1 = 29 cnt0 = 30"
"for poland use polygon-file - cnt1 = 133 cnt0 = 127"
"for sweden use polygon-file - cnt1 = 60 cnt0 = 54"
"for finland do not use polygon-file - cnt1 = 66 cnt0 = 85"
"for spain use polygon-file - cnt1 = 91 cnt0 = 89"
"for italy do not use polygon-file - cnt1 = 155 cnt0 = 201"
"for denmark use polygon-file - cnt1 = 33 cnt0 = 32"
"for andorra do not use polygon-file - cnt1 = 1 cnt0 = 4"
"for estonia use polygon-file - cnt1 = 9 cnt0 = 8"
"for saarland do not use polygon-file - cnt1 = 4 cnt0 = 16"
"for hamburg do not use polygon-file - cnt1 = 3 cnt0 = 12"
"for hessen do not use polygon-file - cnt1 = 17 cnt0 = 18"
"for niedersachsen do not use polygon-file - cnt1 = 26 cnt0 = 27"
"for baden-wuerttemberg use polygon-file - cnt1 = 38 cnt0 = 36"
"for nordrhein-westfalen do not use polygon-file - cnt1 = 56 cnt0 = 57"
"for bayern do not use polygon-file - cnt1 = 47 cnt0 = 48"
"for berlin do not use polygon-file - cnt1 = 5 cnt0 = 13"
"for australia-oceania use polygon-file - cnt1 = 210 cnt0 = 110"
"for south-america do not use polygon-file - cnt1 = 337 cnt0 = 339"
"for africa do not use polygon-file - cnt1 = 673 cnt0 = 742"
"for russia use polygon-file - cnt1 = 431 cnt0 = 409"
"for central-america use polygon-file - cnt1 = 61 cnt0 = 58"
"for morocco do not use polygon-file - cnt1 = 19 cnt0 = 32"
"for azerbaijan do not use polygon-file - cnt1 = 3 cnt0 = 4"
"for malaysia-singapore-brunei do not use polygon-file - cnt1 = 16 cnt0 = 17"
"for china use polygon-file - cnt1 = 106 cnt0 = 100"
"for india do not use polygon-file - cnt1 = 109 cnt0 = 110"
"for indonesia do not use polygon-file - cnt1 = 193 cnt0 = 198"
"for japan use polygon-file - cnt1 = 196 cnt0 = 154"
"for kazakhstan use polygon-file - cnt1 = 14 cnt0 = 13"
"for philippines use polygon-file - cnt1 = 53 cnt0 = 52"
"for afghanistan do not use polygon-file - cnt1 = 12 cnt0 = 21"
"for myanmar use polygon-file - cnt1 = 18 cnt0 = 17"
"for australia use polygon-file - cnt1 = 68 cnt0 = 67"
"for new-zealand do not use polygon-file - cnt1 = 27 cnt0 = 28"
"for argentina use polygon-file - cnt1 = 33 cnt0 = 27"
"for brazil use polygon-file - cnt1 = 265 cnt0 = 177"
"for peru use polygon-file - cnt1 = 22 cnt0 = 21"
"for canada use polygon-file - cnt1 = 299 cnt0 = 275"
"for us-midwest use polygon-file - cnt1 = 155 cnt0 = 154"
"for us-pacific use polygon-file - cnt1 = 21 cnt0 = 18"
"for us-south use polygon-file - cnt1 = 265 cnt0 = 262"
"for us-west use polygon-file - cnt1 = 240 cnt0 = 226"
"for greenland use polygon-file - cnt1 = 4 cnt0 = 2"
"for mexico use polygon-file - cnt1 = 44 cnt0 = 43"
"for reunion do not use polygon-file - cnt1 = 3 cnt0 = 5"
"for asia use polygon-file - cnt1 = 2489 cnt0 = 1505"

On Sun, 20 Jun 2021 at 19:12, Gerd Petermann <gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>>> wrote:
Hi Felix,

it tries to reduce the area that is outside the polygon. I think for most countries there is no good reason to use a polygon.
Candidates are those which cross 180° and maybe those which have lots of unrelateed coastline inside the bounding box.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>>>> im Auftrag von Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>>
Gesendet: Sonntag, 20. Juni 2021 15:12
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] mkgmap splitter - Liechtenstein - horrible split if --polygon-file option used.

But why is it splitting up into so many parts?
So that means you kinda have to check for every country individually if using poly or nor gives the split with less tiles? I would just prefer the split using less tiles so best I write a script to test that for every single country?

On Sun, 20 Jun 2021, 15:27 Gerd Petermann <gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>>>> wrote:
Hi Felix,

the polygon-file tells splitter that you don't want the rectangular bounds given in the *.osm file.
This means you don't want the areas outside of the polygon. If you are happy with the result without polygon file just don't use it

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>>><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>>>>> im Auftrag von Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>>>
Gesendet: Sonntag, 20. Juni 2021 10:17
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] mkgmap splitter - Liechtenstein - horrible split if --polygon-file option used.

I think this applies to all "countries" / areas that are so small that they do not need a split or would split into very few areas. Another example is Monaco (2 tiles instead of 1 tile). For "countries" that are 10 tiles or more anyhow - I did not find this problem.

On Sun, 20 Jun 2021 at 11:08, Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>>>> wrote:
After last week looking at australia-oceania which was not splitted well without polygon-file - I now noticed that several small countries are split into way too many small tiles IF using .poly boundary file.

I guess there is some bug? This tile should not be split at all - as the node count fits into 1 tile!

Attached the splitter.log both with and without .poly file.

--
Felix Hartman - Openmtbmap.org & VeloMap.org

--
Felix Hartman - Openmtbmap.org & VeloMap.org

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>>>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
Felix Hartman - Openmtbmap.org & VeloMap.org

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
Felix Hartman - Openmtbmap.org & VeloMap.org

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
Felix Hartman - Openmtbmap.org & VeloMap.org