[mkgmap-dev] Please test branch NET-no-NOD

Wed Oct 23 21:10:02 BST 2019

Hi Ticker

sorry, just noticed that I had a typo in the mail address. First part is gpetermann_muenchen

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen at hotmail.com>
Gesendet: Mittwoch, 23. Oktober 2019 18:57
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Ticker,

wow, that is even older than mine ;)
Might be worth to try an update to the latest 1.8 release

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
Gesendet: Mittwoch, 23. Oktober 2019 18:43
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD

Hi Gerd

$ java -version
openjdk version "1.8.0_91"
OpenJDK Runtime Environment (build 1.8.0_91-b14)
OpenJDK Server VM (build 25.91-b14, mixed mode)

Ticker

On Wed, 2019-10-23 at 16:21 +0000, Gerd Petermann wrote:
> Hi Ticker,
>
> okay, please let me also know the output of command
> java -version
>
> Mine is:
> e:\ld>java -version
> java version "1.8.0_221"
> Java(TM) SE Runtime Environment (build 1.8.0_221-b11)
> Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> Gesendet: Mittwoch, 23. Oktober 2019 13:47
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
>
> Hi Gerd
>
> The last test was with my style. The routing island check was faster:
>                 r4317   r4309 ms
>   74210002:      593     5252
>   74310001:     17696   45755
>
> The style I'm testing with does add a lot of footways (around car
> parks) and has the highway/barrier point > highway splitting.
>
> I've got to be out for the rest of the day now, but I can send you
> the
> data later tonight or tomorrow morning.
>
> Ticker
>
> On Wed, 2019-10-23 at 11:08 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > if I got that right the new version is slower for you. I see  two
> > possible reasons:
> > - bottleneck in heap memory when processing multiple files at the
> > same time (max-jobs)
> > - your style adds a lot more roads than the default style. For my
> > tests I've already used a modified default style with the
> > highway=traffic_signals | highway=crossing {set mkgmap:road
> > -speed=1}
> > rule added to points.
> >
> > Maybe you can analyse this on your side? Or you can send me your
> > complete test data (the two input files created by splitter and
> > your
> > style files)
> > to gptermann_muenchen <at> hotmail.com?
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> > von Gerd Petermann <gpetermann_muenchen at hotmail.com>
> > Gesendet: Mittwoch, 23. Oktober 2019 12:49
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
> >
> > Hi Ticker,
> >
> > so you get nearly no improvement. Strange. What do you see with
> > default style?
> >
> > Regarding the messages from StyledConverter: That's what I am
> > working
> > on. Question is wheter we should add those obvious routing islands
> > to the road network and let island removal take care of it or if
> > StyledConverter should also calculate the island length given by -
> > -x
> > -check-routing-island-len before removing it.
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> > von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> > Gesendet: Mittwoch, 23. Oktober 2019 12:38
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
> >
> > Hi Gerd
> >
> > with mkgmap-NET-no-NOD-r4317:
> >
> > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork  74210002.osm.pbf:
> > search for routing islands found 293 islands in 65 ms
> > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork  74210002.osm.pbf:
> > routing island removal took 1953 ms
> > EVE: uk.me.parabola.imgfmt.app.net.RoadNetwork  74210001.osm.pbf:
> > search for routing islands found 2820 islands in 62 ms
> > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork  74210001.osm.pbf:
> > routing island removal took 17696 ms
> > Total time taken: 1 minute 20 seconds
> >
> > 13719552 Oct 23 11:26 gmapsupp.img (size is same as from rmkgmap
> > -NET
> > -no
> > -NOD-r4304)
> >
> > It still give lots (691) of these messages:
> > SEVE: uk.me.parabola.mkgmap.osmstyle.StyledConverter
> >  74210002.osm.pbf:
> > check: road without connection is not written to NOD (OSM id
> > 40586008)
> > 51.280882,-1.096959
> >
> > Ticker
> >
> > On Wed, 2019-10-23 at 10:00 +0000, Gerd Petermann wrote:
> > > Hi Ticker,
> > >
> > > fixed, see
> > > http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4
> > > 31
> > > 7
> > >
> > > Working on some more code improvements now, so look for the
> > > latest
> > > binary for the branch at
> > > http://www.mkgmap.org.uk/download/mkgmap.html
> > >
> > > Gerd
> > >
> > > ________________________________________
> > > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im
> > > Auftrag
> > > von Gerd Petermann <gpetermann_muenchen at hotmail.com>
> > > Gesendet: Dienstag, 22. Oktober 2019 20:58
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
> > >
> > > Hi Ticker,
> > >
> > > okay, I can reproduce the poor performance in this area. My
> > > timings:
> > > SCHW: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > >  74210001.osm.pbf:
> > > routing island check took 5281 ms
> > > Total time taken: 35 seconds
> > >
> > > Most of the additional time is used to remove the nodes from NOD.
> > > This can be improved...
> > >
> > > Gerd
> > >
> > > ________________________________________
> > > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im
> > > Auftrag
> > > von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> > > Gesendet: Dienstag, 22. Oktober 2019 19:24
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
> > >
> > > Hi Gerd
> > >
> > > split.list contains:
> > > 74210001: 2362592,-73088 to 2376288,-50304
> > >
> > > Note feed in a lot of points (postcodes) into the splitter, but
> > > there
> > > is nothing in the default style that will process them.
> > >
> > > splitter command line is:
> > > java -Xmx1540M -ea -jar ../splitter/splitter.jar --geonames
> > > -file=../cities15000.zip --mapid=74210001 --split-file=split.list
> > > ../mapGB/british-isles-latest.osm.pbf postcode.so.osm.xml
> > > > splitter.log
> > > 2>&1
> > >
> > >
> > > # significant lines in ../mkgmap.opt
> > > max-jobs
> > > gmapsupp
> > > code-page=1252
> > > index
> > > bounds=../bounds.zip
> > > location-autofill=is_in,nearest
> > > no-housenumbers
> > > name-tag-list=name:en,int_name,name,place_name,loc_name
> > > family-name=osm
> > > series-name=OpenStreetMap
> > > area-name=berkin
> > > x-mapset-name=OSM ticker
> > > route
> > > drive-on=detect,left
> > > preserve-element-order
> > > add-pois-to-areas
> > > generate-sea=multipolygon,extend-sea-sectors,close-gaps=350
> > > link-pois-to-ways
> > > process-destination
> > > process-exits
> > > remove-ovm-work-files
> > > poi-address
> > > verbose
> > > order-by-decreasing-area
> > > add-boundary-nodes-at-admin-boundaries=0
> > >
> > > Command line is:
> > >
> > > java -Xmx1540M -ea -Dlog.config=../mkgmap_log.props -jar
> > > ../mkgmap.noNod/mkgmap.jar -c ../mkgmap.opts --no-style-file --x
> > > -check
> > > -routing-island-len=700 --family-id=7421 --family-name=Hants -c
> > > template.args ../mytyp.txt
> > >
> > > I'm going to be out for the next few hours, but can do the log
> > > stuff
> > > later if you need it.
> > >
> > > Ticker
> > >
> > > On Tue, 2019-10-22 at 16:40 +0000, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > >
> > > > please let me know the mkgmap options and the area for
> > > > 74210001.
> > > > Maybe try with commented the log statements (except the one for
> > > > the
> > > > timing )
> > > >
> > > > Gerd
> > > >
> > > > ________________________________________
> > > > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im
> > > > Auftrag
> > > > von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> > > > Gesendet: Dienstag, 22. Oktober 2019 18:33
> > > > An: Development list for mkgmap
> > > > Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
> > > >
> > > > Hi Gerd
> > > >
> > > > These are with default style.
> > > >
> > > > mkgmap-NET-no-NOD-r4304 --x-check-routing-island-len=700
> > > >
> > > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > >  74210002.osm.pbf:
> > > > check for routing islands found 38 islands
> > > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > >  74210002.osm.pbf:
> > > > routing island check took 689 ms
> > > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > >  74210001.osm.pbf:
> > > > check for routing islands found 554 islands
> > > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > >  74210001.osm.pbf:
> > > > routing island check took 8279 ms
> > > >
> > > > Total time taken: 1 minute 12 seconds
> > > >
> > > > gmapsupp.img 15126528 bytes
> > > > Block size: 2048
> > > >         .       22016
> > > > MAKEGMAP.MPS    138
> > > > 00007421.MDR    728896
> > > > 000MYTYP.TYP    1580
> > > > 74210001.RGN    5870951
> > > > 74210001.TRE    45927
> > > > 74210001.LBL    624175
> > > > 74210001.NET    1347117
> > > > 74210001.NOD    2384384
> > > > 74210002.RGN    2120704
> > > > 74210002.TRE    17330
> > > > 74210002.LBL    229542
> > > > 74210002.NET    592062
> > > > 74210002.NOD    1124864
> > > > 00007421.SRT    912
> > > >
> > > > With r4295:
> > > >
> > > > Total time taken: 1 minute 3 seconds
> > > >
> > > > gmapsupp.img 15943680 bytes
> > > > Block size: 2048
> > > >         .       23040
> > > > MAKEGMAP.MPS    138
> > > > 00007421.MDR    728896
> > > > 000MYTYP.TYP    1580
> > > > 74210001.RGN    5870951
> > > > 74210001.TRE    45927
> > > > 74210001.LBL    624175
> > > > 74210001.NET    1354678
> > > > 74210001.NOD    2975539
> > > > 74210002.RGN    2120704
> > > > 74210002.TRE    17330
> > > > 74210002.LBL    229542
> > > > 74210002.NET    593443
> > > > 74210002.NOD    1343157
> > > > 00007421.SRT    912
> > > >
> > > > Ticker
> > > >
> > > > On Tue, 2019-10-22 at 16:02 +0000, Gerd Petermann wrote:
> > > > > Hi Ticker,
> > > > >
> > > > > strange, in my tests the test for routing islands performed
> > > > > in
> > > > > <
> > > > > 50
> > > > > ms. So, there seems to be a special case.
> > > > > Also, I did not see that big effect on img siz.
> > > > > Please can you test with default style for comparison?
> > > > >
> > > > > Gerd
> > > > >
> > > > > ________________________________________
> > > > > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im
> > > > > Auftrag
> > > > > von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> > > > > Gesendet: Dienstag, 22. Oktober 2019 17:51
> > > > > An: Development list for mkgmap
> > > > > Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
> > > > >
> > > > > Hi Gerd
> > > > >
> > > > > Testing mkgmap-NET-no-NOD-r4304 with 2 tile local area with
> > > > > option
> > > > > --x-check-routing-island-len=700
> > > > >
> > > > > Some build statistics:
> > > > >
> > > > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > > >  74210002.osm.pbf:
> > > > > check for routing islands found 293 islands
> > > > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > > >  74210002.osm.pbf:
> > > > > routing island check took 5252 ms
> > > > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > > >  74210001.osm.pbf:
> > > > > check for routing islands found 2820 islands
> > > > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > > >  74210001.osm.pbf:
> > > > > routing island check took 45755 ms
> > > > >
> > > > > Total time taken: 1 minute 53 seconds
> > > > > gmapsupp.img 13719552 bytes
> > > > > block size: 2048
> > > > >         .       20992
> > > > > MAKEGMAP.MPS    138
> > > > > 00007421.MDR    1121108
> > > > > 000MYTYP.TYP    1580
> > > > > 74210001.RGN    3802657
> > > > > 74210001.TRE    17874
> > > > > 74210001.LBL    758810
> > > > > 74210001.NET    1536856
> > > > > 74210001.NOD    2556416
> > > > > 74210002.RGN    1645275
> > > > > 74210002.TRE    8820
> > > > > 74210002.LBL    268782
> > > > > 74210002.NET    718899
> > > > > 74210002.NOD    1241600
> > > > > 00007421.SRT    912
> > > > >
> > > > > For comparison, testing with r4295 (I get similar figures for
> > > > > trunk+):
> > > > >
> > > > > Total time taken: 1 minute 8 seconds
> > > > > gmapsupp.img 14710784 bytes
> > > > > Block size: 2048
> > > > >         .       22016
> > > > > MAKEGMAP.MPS    138
> > > > > 00007421.MDR    1121108
> > > > > 000MYTYP.TYP    1580
> > > > > 74210001.RGN    3802683
> > > > > 74210001.TRE    17874
> > > > > 74210001.LBL    758810
> > > > > 74210001.NET    1562592
> > > > > 74210001.NOD    3285327
> > > > > 74210002.RGN    1645275
> > > > > 74210002.TRE    8820
> > > > > 74210002.LBL    268782
> > > > > 74210002.NET    722291
> > > > > 74210002.NOD    1477491
> > > > > 00007421.SRT    912
> > > > >
> > > > > So, for my test case, with my style, which probably creates
> > > > > quite
> > > > > a
> > > > > few
> > > > > more isolated networks than the default style:
> > > > >
> > > > > It adds quite a lot of processing time (increased by 66%)
> > > > >
> > > > > Reduces the NOD size significantly (22%) and full
> > > > > gmapsupp.img
> > > > > by
> > > > > 6%.
> > > > >
> > > > > It fixes a lot of "Route calculation errors" where the
> > > > > nearest
> > > > > road
> > > > > to
> > > > > the start/end point is an isolated network. However the
> > > > > behaviour
> > > > > of
> > > > > this changes in a way not apparent to the map user if what
> > > > > should
> > > > > have
> > > > > been an isolated network overlaps tiles.
> > > > >
> > > > > I vote for keeping something like this provided the routing
> > > > > -island
> > > > > detection only happens of the length option is specified
> > > > > (needs
> > > > > a
> > > > > better name than --x-check-routing-island-len)
> > > > >
> > > > > The unconnected road handling should be moved out of
> > > > > StyledConverter
> > > > > and handled as part of the length controlled general logic.
> > > > >
> > > > > Ticker
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > mkgmap-dev at lists.mkgmap.org.uk
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > _______________________________________________
> > > > > mkgmap-dev mailing list
> > > > > mkgmap-dev at lists.mkgmap.org.uk
> > > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > mkgmap-dev at lists.mkgmap.org.uk
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > mkgmap-dev at lists.mkgmap.org.uk
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev at lists.mkgmap.org.uk
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev at lists.mkgmap.org.uk
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev at lists.mkgmap.org.uk
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev