[mkgmap-dev] mkgmap splitter - Liechtenstein - horrible split if --polygon-file option used.

Wed Jun 23 14:38:30 BST 2021

sorry - I thought after your previous reply that this behaviour is to be
expected - but as the differences are so big I wanted to post it here so
others are warned. I did not log those other files anywhere and usually use
>NUL 2>NUL because I want to keep the terminal clean - if splitter crashes
on a split there are usually 0byte files produced - so I just check from
them. Same for mkgmap. All those error messages I only look at if there is
actually something broken. Else they are simply too much when compiling 180
extracts in 4 versions every week.

On Wed, 23 Jun 2021 at 16:33, Gerd Petermann <
gpetermann_muenchen at hotmail.com> wrote:

> Hi Felix,
>
> why don't you keep the extra files produced by splitter somewhere for
> debugging? They are rather small and make it much easier to understand what
> happened (at least for me).
>
> Anyhow, no need to run something again for now.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
> Felix Hartmann <extremecarver at gmail.com>
> Gesendet: Mittwoch, 23. Juni 2021 15:25
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] mkgmap splitter - Liechtenstein - horrible split
> if --polygon-file option used.
>
> No I did not log them - I only used a for loop to get the number of tiles.
> Brazil, Australia-Ocieania, Asia those differences are really so big that
> it is hard to believe mkgmap cannot do better. Options otherwise identical.
> Maxnodes sometimes a bit less than 140... If this is important I can write
> down for which countries they differed - but I think this happens at any
> maxnode value.
>
> If you would like me to run all of them again and do a splitter.log I can.
> Will take 12 hours or so however. I am pretty sure all those splits which
> are vastly different in size have an error message. But I did not check
> that. Africa and Italy are quite stunning too. Why is it soo much less
> tiles not using a boundary file (I downloaded the extracts at the same time
> as the bound files - so they should be identical if geofabrik has no error).
>
> On Wed, 23 Jun 2021 at 15:58, Gerd Petermann <
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>
> wrote:
> Hi Felix,
>
> I can reproduce the surprising results for Brazil with
> --max-nodes=1400000. This really looks like an error in splitter because
> splitter should simply do the same after reporting
> "Warning: shape is too complex, using rectangle
> (-35.5078125,-74.091796875) to (5.537109375,-27.6416015625) with
> 213.392.248 nodes instead"
>
> But the search for a good split shows slightly different numbers and the
> log without polygon file says "Solution is not nice. Can't find a better
> solution ..."
> I suggest to look for this message.
>
> I'll try to find out why exactly splitter behaves like that.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von Gerd Petermann <
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>
> Gesendet: Mittwoch, 23. Juni 2021 13:49
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] mkgmap splitter - Liechtenstein - horrible split
> if --polygon-file option used.
>
> Hi Felix,
>
> quite interesting. Do you have the log files from splitter and /or the
> densities-out.txt files? If yes, please share them.
> Did you always use the same splitter options?
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>
> Gesendet: Mittwoch, 23. Juni 2021 13:12
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] mkgmap splitter - Liechtenstein - horrible split
> if --polygon-file option used.
>
> Okay - well I ran splitter against all geofrabrik extracts that I am
> having on my server (so nearly all - notably excluding DACH and Greater
> Britain extract) - and wrote a script comparing the splits. Overall because
> of Asia continent map it is better to use polygon-file. Seeing those huge
> differences on Asia, australia-oceania, Italy, Africa, Japan and Brazil
> I do feel splitter should be doing a bit better. Especially Asia continent
> is hard to understand the huge huge difference.
> In general for tiny countries - do not use it. Do not use it for German
> Bundesländer. Do not use polygon-file for Italy. But do use it on
> Australia-Oceania, Africa, Japan, Brazil, and Asia continent map.
>
> Now I wonder - will this change in the future? So meaning I have to run
> this assessment once a year? Usually the additional tiles are mostly tiny -
> there is no reason for having them. If the split results in the same amount
> of tiles no matter using --polygon-file or not - it is not listed (overall
> 176 extracts). I put those countries with huge difference in bold.
> 26 splits are better not using --polygon-file. 30 splits are better using
> --polygon-file. 906 tiles less for Asia continent shows that splitting Asia
> without --polygon-file is a right mess. On the other hand for some tiny
> countries like Lichtenstein or German Bundesländer, and Africa using a
> --polygon-file from geofabrik created a way too many useless tiles.
> cnt1 = using --polygon-file, cnt0=not using polygon-file for splitting.
>
> "for montenegro do not use polygon-file - cnt1 = 2 cnt0 = 7"
> "for france use polygon-file - cnt1 = 336 cnt0 = 332"
> "for netherlands do not use polygon-file - cnt1 = 91 cnt0 = 92"
> "for germany use polygon-file - cnt1 = 259 cnt0 = 258"
> "for liechtenstein do not use polygon-file - cnt1 = 1 cnt0 = 6"
> "for monaco do not use polygon-file - cnt1 = 1 cnt0 = 2"
> "for slovenia do not use polygon-file - cnt1 = 27 cnt0 = 28"
> "for ukraine use polygon-file - cnt1 = 62 cnt0 = 61"
> "for norway use polygon-file - cnt1 = 128 cnt0 = 115"
> "for switzerland do not use polygon-file - cnt1 = 29 cnt0 = 30"
> "for poland use polygon-file - cnt1 = 133 cnt0 = 127"
> "for sweden use polygon-file - cnt1 = 60 cnt0 = 54"
> "for finland do not use polygon-file - cnt1 = 66 cnt0 = 85"
> "for spain use polygon-file - cnt1 = 91 cnt0 = 89"
> "for italy do not use polygon-file - cnt1 = 155 cnt0 = 201"
> "for denmark use polygon-file - cnt1 = 33 cnt0 = 32"
> "for andorra do not use polygon-file - cnt1 = 1 cnt0 = 4"
> "for estonia use polygon-file - cnt1 = 9 cnt0 = 8"
> "for saarland do not use polygon-file - cnt1 = 4 cnt0 = 16"
> "for hamburg do not use polygon-file - cnt1 = 3 cnt0 = 12"
> "for hessen do not use polygon-file - cnt1 = 17 cnt0 = 18"
> "for niedersachsen do not use polygon-file - cnt1 = 26 cnt0 = 27"
> "for baden-wuerttemberg use polygon-file - cnt1 = 38 cnt0 = 36"
> "for nordrhein-westfalen do not use polygon-file - cnt1 = 56 cnt0 = 57"
> "for bayern do not use polygon-file - cnt1 = 47 cnt0 = 48"
> "for berlin do not use polygon-file - cnt1 = 5 cnt0 = 13"
> "for australia-oceania use polygon-file - cnt1 = 210 cnt0 = 110"
> "for south-america do not use polygon-file - cnt1 = 337 cnt0 = 339"
> "for africa do not use polygon-file - cnt1 = 673 cnt0 = 742"
> "for russia use polygon-file - cnt1 = 431 cnt0 = 409"
> "for central-america use polygon-file - cnt1 = 61 cnt0 = 58"
> "for morocco do not use polygon-file - cnt1 = 19 cnt0 = 32"
> "for azerbaijan do not use polygon-file - cnt1 = 3 cnt0 = 4"
> "for malaysia-singapore-brunei do not use polygon-file - cnt1 = 16 cnt0 =
> 17"
> "for china use polygon-file - cnt1 = 106 cnt0 = 100"
> "for india do not use polygon-file - cnt1 = 109 cnt0 = 110"
> "for indonesia do not use polygon-file - cnt1 = 193 cnt0 = 198"
> "for japan use polygon-file - cnt1 = 196 cnt0 = 154"
> "for kazakhstan use polygon-file - cnt1 = 14 cnt0 = 13"
> "for philippines use polygon-file - cnt1 = 53 cnt0 = 52"
> "for afghanistan do not use polygon-file - cnt1 = 12 cnt0 = 21"
> "for myanmar use polygon-file - cnt1 = 18 cnt0 = 17"
> "for australia use polygon-file - cnt1 = 68 cnt0 = 67"
> "for new-zealand do not use polygon-file - cnt1 = 27 cnt0 = 28"
> "for argentina use polygon-file - cnt1 = 33 cnt0 = 27"
> "for brazil use polygon-file - cnt1 = 265 cnt0 = 177"
> "for peru use polygon-file - cnt1 = 22 cnt0 = 21"
> "for canada use polygon-file - cnt1 = 299 cnt0 = 275"
> "for us-midwest use polygon-file - cnt1 = 155 cnt0 = 154"
> "for us-pacific use polygon-file - cnt1 = 21 cnt0 = 18"
> "for us-south use polygon-file - cnt1 = 265 cnt0 = 262"
> "for us-west use polygon-file - cnt1 = 240 cnt0 = 226"
> "for greenland use polygon-file - cnt1 = 4 cnt0 = 2"
> "for mexico use polygon-file - cnt1 = 44 cnt0 = 43"
> "for reunion do not use polygon-file - cnt1 = 3 cnt0 = 5"
> "for asia use polygon-file - cnt1 = 2489 cnt0 = 1505"
>
> On Sun, 20 Jun 2021 at 19:12, Gerd Petermann <
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com
> ><mailto:gpetermann_muenchen at hotmail.com<mailto:
> gpetermann_muenchen at hotmail.com>>> wrote:
> Hi Felix,
>
> it tries to reduce the area that is outside the polygon. I think for most
> countries there is no good reason to use a polygon.
> Candidates are those which cross 180° and maybe those which have lots of
> unrelateed coastline inside the bounding box.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>>> im Auftrag von Felix Hartmann <
> extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>
> Gesendet: Sonntag, 20. Juni 2021 15:12
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] mkgmap splitter - Liechtenstein - horrible split
> if --polygon-file option used.
>
> But why is it splitting up into so many parts?
> So that means you kinda have to check for every country individually if
> using poly or nor gives the split with less tiles? I would just prefer the
> split using less tiles so best I write a script to test that for every
> single country?
>
> On Sun, 20 Jun 2021, 15:27 Gerd Petermann <gpetermann_muenchen at hotmail.com
> <mailto:gpetermann_muenchen at hotmail.com><mailto:
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com
> >><mailto:gpetermann_muenchen at hotmail.com<mailto:
> gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com
> <mailto:gpetermann_muenchen at hotmail.com>>>> wrote:
> Hi Felix,
>
> the polygon-file tells splitter that you don't want the rectangular bounds
> given in the *.osm file.
> This means you don't want the areas outside of the polygon. If you are
> happy with the result without polygon file just don't use it
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>>>> im Auftrag von Felix Hartmann <
> extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>>
> Gesendet: Sonntag, 20. Juni 2021 10:17
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] mkgmap splitter - Liechtenstein - horrible split
> if --polygon-file option used.
>
> I think this applies to all "countries" / areas that are so small that
> they do not need a split or would split into very few areas. Another
> example is Monaco (2 tiles instead of 1 tile). For "countries" that are 10
> tiles or more anyhow - I did not find this problem.
>
> On Sun, 20 Jun 2021 at 11:08, Felix Hartmann <extremecarver at gmail.com
> <mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>>>>> wrote:
> After last week looking at australia-oceania which was not splitted well
> without polygon-file - I now noticed that several small countries are split
> into way too many small tiles IF using .poly boundary file.
>
> I guess there is some bug? This tile should not be split at all - as the
> node count fits into 1 tile!
>
>
> Attached the splitter.log both with and without .poly file.
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

-- 
Felix Hartman - Openmtbmap.org & VeloMap.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20210623/e2cc0762/attachment-0001.html>