[mkgmap-dev] splitter r609 released

Thu Jul 1 08:58:18 BST 2021

Well here are the results for 615:
If you would like the log files for any of those runs - tell me.
Significantly better with polygon file are Norway, Russia,
Australia-Oceania and Asia. This version has much less problem countries
like 614. It would be too big to attach all of them here. If you like all
of them then I can upload them to my server.

South America is really bad using polygon-file.

"for netherlands use polygon-file - cnt1 = 89 cnt0 = 88"
"for great-britain use polygon-file - cnt1 = 111 cnt0 = 110"
"for germany use polygon-file - cnt1 = 258 cnt0 = 257"
"for liechtenstein do not use polygon-file - cnt1 = 1 cnt0 = 6"
"for monaco do not use polygon-file - cnt1 = 1 cnt0 = 2"
"for slovenia do not use polygon-file - cnt1 = 27 cnt0 = 28"
"for ukraine use polygon-file - cnt1 = 62 cnt0 = 61"
*"for norway use polygon-file - cnt1 = 129 cnt0 = 119" *
"for switzerland do not use polygon-file - cnt1 = 29 cnt0 = 30"
"for poland use polygon-file - cnt1 = 132 cnt0 = 128"
"for sweden use polygon-file - cnt1 = 60 cnt0 = 54"
"for finland do not use polygon-file - cnt1 = 65 cnt0 = 84"
"for denmark use polygon-file - cnt1 = 33 cnt0 = 32"
"for andorra do not use polygon-file - cnt1 = 1 cnt0 = 4"
"for estonia use polygon-file - cnt1 = 9 cnt0 = 8"
"for saarland do not use polygon-file - cnt1 = 4 cnt0 = 17"
"for hamburg do not use polygon-file - cnt1 = 3 cnt0 = 12"
"for hessen do not use polygon-file - cnt1 = 17 cnt0 = 18"
"for bayern do not use polygon-file - cnt1 = 47 cnt0 = 48"
"for berlin do not use polygon-file - cnt1 = 5 cnt0 = 13"
*"for australia-oceania use polygon-file - cnt1 = 131 cnt0 = 109" *
"for south-america do not use polygon-file - cnt1 = 334 cnt0 = 463"
"for africa use polygon-file - cnt1 = 665 cnt0 = 663"
*"for asia use polygon-file - cnt1 = 1592 cnt0 = 1549" *
*"for russia use polygon-file - cnt1 = 426 cnt0 = 408" *
"for central-america use polygon-file - cnt1 = 61 cnt0 = 58"
"for morocco do not use polygon-file - cnt1 = 19 cnt0 = 27"
"for tanzania use polygon-file - cnt1 = 75 cnt0 = 74"
"for mozambique use polygon-file - cnt1 = 24 cnt0 = 23"
"for azerbaijan do not use polygon-file - cnt1 = 3 cnt0 = 4"
"for iran use polygon-file - cnt1 = 16 cnt0 = 15"
"for malaysia-singapore-brunei do not use polygon-file - cnt1 = 16 cnt0 =
17"
"for china use polygon-file - cnt1 = 106 cnt0 = 101"
"for india do not use polygon-file - cnt1 = 109 cnt0 = 112"
"for indonesia do not use polygon-file - cnt1 = 191 cnt0 = 200"
"for philippines use polygon-file - cnt1 = 52 cnt0 = 51"
"for afghanistan do not use polygon-file - cnt1 = 11 cnt0 = 20"
"for australia use polygon-file - cnt1 = 67 cnt0 = 66"
"for argentina use polygon-file - cnt1 = 32 cnt0 = 27"
"for brazil use polygon-file - cnt1 = 177 cnt0 = 172"
"for chile do not use polygon-file - cnt1 = 31 cnt0 = 33"
"for paraguay use polygon-file - cnt1 = 17 cnt0 = 16"
"for canada use polygon-file - cnt1 = 274 cnt0 = 272"
"for us-northeast do not use polygon-file - cnt1 = 109 cnt0 = 110"
"for us-pacific use polygon-file - cnt1 = 21 cnt0 = 18"
"for us-south use polygon-file - cnt1 = 262 cnt0 = 261"
"for us-west use polygon-file - cnt1 = 229 cnt0 = 227"
"for greenland use polygon-file - cnt1 = 4 cnt0 = 2"
"for mexico use polygon-file - cnt1 = 44 cnt0 = 42"
"for reunion do not use polygon-file - cnt1 = 3 cnt0 = 5"

On Wed, 30 Jun 2021 at 12:30, Gerd Petermann <
gpetermann_muenchen at hotmail.com> wrote:

> Hi Felix,
>
> I've released r615 to fix the problems with Saarland (now 17 tiles with
> polygon instead of crash, 4 tiles without) and Norway (121 with polygon,
> 128 without)
> Numbers for max-nodes=1400000
>
> Gerd
>
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
> Felix Hartmann <extremecarver at gmail.com>
> Gesendet: Mittwoch, 30. Juni 2021 10:52
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter r609 released
>
> well yeah by now my list is down to very few countries (different then
> with older splitter.jar) that profit from --polygon-file.
> I feel like I will only use polygon-file for Asia continent map and for
> Russia - they fit your description of spanning over 180° and needing
> polygon-file to be split into less tiles. They pretty consistently split
> better with polygon-file on several splitter versions.
>
> Then maybe retest a year later.
>
> On Wed, 30 Jun 2021 at 11:25, Gerd Petermann <
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>
> wrote:
> Hi Felix,
>
> sure, it's not good if splitter fails to split correct data. Still, an
> automated process should check the return code of the program. If splitter
> returns a values !=  0 you can be sure that something is wrong (same with
> mkgmap)
>
> It's okay to split small files like Liechtenstein, no problem with that.
> Just don't expect to always get fewer tiles when you use a polygon-file ;)
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>
> Gesendet: Mittwoch, 30. Juni 2021 10:10
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter r609 released
>
> yes my script only counts the produced tiles. And if it's 0 that's very
> bad for any automated process. 0 should only happen if the input data is
> broken (yes then it should happen).
> (and why do I split countries like Liechtenstein - well first I do not
> know if the extract would fit into one tile without splitting, second if
> the download is broken for some reason - then splitter rightfully cannot
> split it - and the old maps on the server will not be overwritten by maps
> based on a broken extract from geofabrik).
>
> On Wed, 30 Jun 2021 at 11:03, Gerd Petermann <
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com
> ><mailto:gpetermann_muenchen at hotmail.com<mailto:
> gpetermann_muenchen at hotmail.com>>> wrote:
> Hi Felix,
>
> I can now reproduce the poor result with norway when I use both
> --polygon-file and --precomp-sea.
> Problem seems to be that splitter decides to try the slower algo first.
> After a while it gives up and tries the alternative faster algo and that
> would find a good solution soon but splitter gives up too early, so this is
> really not an improvement.
> Without the polygon the slower algo finds a good solution.
>
> I am working on a new solution which executes both algos in parallel and
> selects the better result. This works much better for this test case but
> sometimes worse when option --num-tiles is used.
>
> reg. Saarland with polygon: r614 stops with an error message (and returns
> 1 to signal a bad split). Your scripts seems to ignore this and just counts
> the number of written tiles?
> I'll try to find a fix so that splitter doesn't try to fit the tiles into
> the polygon when this happens...
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>>> im Auftrag von Felix Hartmann <
> extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>
> Gesendet: Mittwoch, 30. Juni 2021 08:55
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter r609 released
>
> okay yeah I think I misunderstood the polygon-file a bit. But why is the
> split for Norway so much worse now? Before it was possible to get it down
> to 115 tiles (using the polygon-file option) now its 168 instead. Not using
> polygon-file no change. (guess the 1 tile more is maybe added data
> somewhere).
>
> On Wed, 30 Jun 2021 at 07:59, Gerd Petermann <
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com
> ><mailto:gpetermann_muenchen at hotmail.com<mailto:
> gpetermann_muenchen at hotmail.com>><mailto:gpetermann_muenchen at hotmail.com
> <mailto:gpetermann_muenchen at hotmail.com><mailto:
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>>>
> wrote:
> Hi Felix,
>
> reg. Saarland: I'll try to reproduce. Looks like an error.
> reg. Liechtenstein:
> I think we still have different ideas what the polygon-file option is
> about.
> The documentation is a bit outdated since r433 (see
> https://www.mkgmap.org.uk/websvn/revision.php?repname=splitter&rev=433 )
> but the basic idea was that splitter should try to produce tiles which
> don't overlap the given polygon too much. This sometimes means that it
> produces more and smaller tiles.
> Splitter should be able to find good splits for geofabrik downloads
> without a polygon-file, but there are use cases: If the user only wants a
> part of a download in the map they can use the corresponding polygon.
>
> You seem to suggest that splitter should just use the polygon to be able
> to ignore data that is in the input file but not in the polygon and never
> try to fit the tiles into the polygon?
>
> Gerd
>
>
>
>
>
>
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>>>> im Auftrag von Felix Hartmann <
> extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>>
> Gesendet: Dienstag, 29. Juni 2021 23:04
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] splitter r609 released
>
> I found (server) time to run my test again - using splitter v614 - search
> limit 1000000
> Problem cases in bold. I also added the number of tiles that splitter v602
> needed (well on one week older geofabrik extract - but that should not make
> such a difference for Norway or South America). I feel 602 with
> search-limit 1000000 was producing more consistent results.
>
> "for netherlands use polygon-file - cnt1 = 93 cnt0 = 88"
> "for alps do not use polygon-file - cnt1 = 225 cnt0 = 226"
> "for liechtenstein do not use polygon-file - cnt1 = 1 cnt0 = 6"
> I thought that the new version fixes the splitting if it is not actually
> needed.
>
> "for monaco do not use polygon-file - cnt1 = 1 cnt0 = 2"
> "for slovenia do not use polygon-file - cnt1 = 27 cnt0 = 28"
> "for ukraine use polygon-file - cnt1 = 62 cnt0 = 61"
> "for norway do not use polygon-file - cnt1 = 129 cnt0 = 168"
> Big degradation here - with the older splitter it was: "for norway use
> polygon-file - cnt1 = 128 cnt0 = 115" (also using 1000000 search limit)
>
> "for switzerland do not use polygon-file - cnt1 = 29 cnt0 = 30"
> "for poland do not use polygon-file - cnt1 = 127 cnt0 = 128"
> "for sweden use polygon-file - cnt1 = 60 cnt0 = 54"
> "for finland do not use polygon-file - cnt1 = 65 cnt0 = 84"
> "for czech-republic use polygon-file - cnt1 = 63 cnt0 = 62"
> "for denmark use polygon-file - cnt1 = 33 cnt0 = 32"
> "for austria use polygon-file - cnt1 = 56 cnt0 = 54"
> "for andorra do not use polygon-file - cnt1 = 1 cnt0 = 4"
> "for estonia use polygon-file - cnt1 = 9 cnt0 = 8"
> "for saarland use polygon-file - cnt1 = 4 cnt0 = 0"
> Seems using a polygon-file no spit at all - however it should still write
> the data again out into the newly defined filename.
>
> "for hamburg do not use polygon-file - cnt1 = 3 cnt0 = 12"
> "for hessen do not use polygon-file - cnt1 = 17 cnt0 = 18"
> "for bayern do not use polygon-file - cnt1 = 47 cnt0 = 48"
> "for berlin do not use polygon-file - cnt1 = 5 cnt0 = 13"
> "for australia-oceania use polygon-file - cnt1 = 112 cnt0 = 109"
> "for south-america do not use polygon-file - cnt1 = 335 cnt0 = 463"
> old splitter "for south-america do not use polygon-file - cnt1 = 337 cnt0
> = 339"  (using default search limit)
>
> "for africa do not use polygon-file - cnt1 = 653 cnt0 = 663"
> "for asia use polygon-file - cnt1 = 1697 cnt0 = 1549"
> "for russia use polygon-file - cnt1 = 429 cnt0 = 408"
> 0ld identical
>
> "for central-america use polygon-file - cnt1 = 60 cnt0 = 58"
> "for antarctica use polygon-file - cnt1 = 7 cnt0 = 6"
> "for morocco do not use polygon-file - cnt1 = 19 cnt0 = 27"
> old identical
>
> "for congo-democratic-republic do not use polygon-file - cnt1 = 37 cnt0 =
> 38"
> "for mozambique use polygon-file - cnt1 = 24 cnt0 = 23"
> "for azerbaijan do not use polygon-file - cnt1 = 3 cnt0 = 4"
> "for malaysia-singapore-brunei do not use polygon-file - cnt1 = 16 cnt0 =
> 17"
> "for china use polygon-file - cnt1 = 104 cnt0 = 101"
> "for india do not use polygon-file - cnt1 = 109 cnt0 = 112"
> "for indonesia do not use polygon-file - cnt1 = 175 cnt0 = 200"
> "for japan use polygon-file - cnt1 = 163 cnt0 = 160"
> "for philippines do not use polygon-file - cnt1 = 50 cnt0 = 51"
> "for afghanistan do not use polygon-file - cnt1 = 11 cnt0 = 20"
> old identical
>
> "for australia use polygon-file - cnt1 = 67 cnt0 = 66"
> "for argentina use polygon-file - cnt1 = 31 cnt0 = 27"
> "for brazil use polygon-file - cnt1 = 176 cnt0 = 172"
> "for chile do not use polygon-file - cnt1 = 31 cnt0 = 33"
> "for paraguay use polygon-file - cnt1 = 17 cnt0 = 16"
> "for us-midwest do not use polygon-file - cnt1 = 152 cnt0 = 154"
> "for us-northeast do not use polygon-file - cnt1 = 109 cnt0 = 110"
> "for us-pacific use polygon-file - cnt1 = 20 cnt0 = 18"
> "for us-south do not use polygon-file - cnt1 = 259 cnt0 = 261"
> "for us-west do not use polygon-file - cnt1 = 224 cnt0 = 227"
> "for greenland use polygon-file - cnt1 = 4 cnt0 = 2"
> "for mexico use polygon-file - cnt1 = 43 cnt0 = 42"
> "for reunion do not use polygon-file - cnt1 = 3 cnt0 = 5"
>
> On Fri, 25 Jun 2021 at 16:51, Gerd Petermann <
> GPetermann_muenchen at hotmail.com<mailto:GPetermann_muenchen at hotmail.com
> ><mailto:GPetermann_muenchen at hotmail.com<mailto:
> GPetermann_muenchen at hotmail.com>><mailto:GPetermann_muenchen at hotmail.com
> <mailto:GPetermann_muenchen at hotmail.com><mailto:
> GPetermann_muenchen at hotmail.com<mailto:GPetermann_muenchen at hotmail.com
> >>><mailto:GPetermann_muenchen at hotmail.com<mailto:
> GPetermann_muenchen at hotmail.com><mailto:GPetermann_muenchen at hotmail.com
> <mailto:GPetermann_muenchen at hotmail.com>><mailto:
> GPetermann_muenchen at hotmail.com<mailto:GPetermann_muenchen at hotmail.com
> ><mailto:GPetermann_muenchen at hotmail.com<mailto:
> GPetermann_muenchen at hotmail.com>>>>> wrote:
> Hi all,
>
> I think with r609 there should be no need to use a polygon file. See
> https://www.mkgmap.org.uk/websvn/revision.php?repname=splitter&rev=609
>
> Please let me know when this version produces much worse results compared
> to older releases (using the same options and input) and provide the files
> densities-out.txt and the log.
>
> If you are interested in good splits you should check the splitter log for
> "Solution is not nice. Can't find a better solution"
>
> When this is printed splitter did not find a good split. Expect almost
> empty tiles in this case.
> It is likely that this happens when rather small files are split with a
> polygon and the default resolution. Normal users don't do this, but some
> map providers try to use the same options for very different downloads ...
>
> Gerd
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> >>><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk>><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>>>>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>

-- 
Felix Hartman - Openmtbmap.org & VeloMap.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20210701/724ad630/attachment-0001.html>