[mkgmap-dev] Please test branch NET-no-NOD
From Ticker Berkin rwb-mkgmap at jagit.co.uk on Wed Oct 23 12:47:08 BST 2019
Hi Gerd
The last test was with my style. The routing island check was faster:
r4317 r4309 ms
74210002: 593 5252
74310001: 17696 45755
The style I'm testing with does add a lot of footways (around car
parks) and has the highway/barrier point > highway splitting.
I've got to be out for the rest of the day now, but I can send you the
data later tonight or tomorrow morning.
Ticker
On Wed, 2019-10-23 at 11:08 +0000, Gerd Petermann wrote:
> Hi Ticker,
>
> if I got that right the new version is slower for you. I see two
> possible reasons:
> - bottleneck in heap memory when processing multiple files at the
> same time (max-jobs)
> - your style adds a lot more roads than the default style. For my
> tests I've already used a modified default style with the
> highway=traffic_signals | highway=crossing {set mkgmap:road-speed=1}
> rule added to points.
>
> Maybe you can analyse this on your side? Or you can send me your
> complete test data (the two input files created by splitter and your
> style files)
> to gptermann_muenchen <at> hotmail.com?
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> von Gerd Petermann <gpetermann_muenchen at hotmail.com>
> Gesendet: Mittwoch, 23. Oktober 2019 12:49
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
>
> Hi Ticker,
>
> so you get nearly no improvement. Strange. What do you see with
> default style?
>
> Regarding the messages from StyledConverter: That's what I am working
> on. Question is wheter we should add those obvious routing islands
> to the road network and let island removal take care of it or if
> StyledConverter should also calculate the island length given by --x
> -check-routing-island-len before removing it.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> Gesendet: Mittwoch, 23. Oktober 2019 12:38
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
>
> Hi Gerd
>
> with mkgmap-NET-no-NOD-r4317:
>
> SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf:
> search for routing islands found 293 islands in 65 ms
> SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210002.osm.pbf:
> routing island removal took 1953 ms
> EVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf:
> search for routing islands found 2820 islands in 62 ms
> SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf:
> routing island removal took 17696 ms
> Total time taken: 1 minute 20 seconds
>
> 13719552 Oct 23 11:26 gmapsupp.img (size is same as from rmkgmap-NET
> -no
> -NOD-r4304)
>
> It still give lots (691) of these messages:
> SEVE: uk.me.parabola.mkgmap.osmstyle.StyledConverter
> 74210002.osm.pbf:
> check: road without connection is not written to NOD (OSM id
> 40586008)
> 51.280882,-1.096959
>
> Ticker
>
> On Wed, 2019-10-23 at 10:00 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > fixed, see
> > http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=431
> > 7
> >
> > Working on some more code improvements now, so look for the latest
> > binary for the branch at
> > http://www.mkgmap.org.uk/download/mkgmap.html
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> > von Gerd Petermann <gpetermann_muenchen at hotmail.com>
> > Gesendet: Dienstag, 22. Oktober 2019 20:58
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
> >
> > Hi Ticker,
> >
> > okay, I can reproduce the poor performance in this area. My
> > timings:
> > SCHW: uk.me.parabola.imgfmt.app.net.RoadNetwork 74210001.osm.pbf:
> > routing island check took 5281 ms
> > Total time taken: 35 seconds
> >
> > Most of the additional time is used to remove the nodes from NOD.
> > This can be improved...
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> > von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> > Gesendet: Dienstag, 22. Oktober 2019 19:24
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
> >
> > Hi Gerd
> >
> > split.list contains:
> > 74210001: 2362592,-73088 to 2376288,-50304
> >
> > Note feed in a lot of points (postcodes) into the splitter, but
> > there
> > is nothing in the default style that will process them.
> >
> > splitter command line is:
> > java -Xmx1540M -ea -jar ../splitter/splitter.jar --geonames
> > -file=../cities15000.zip --mapid=74210001 --split-file=split.list
> > ../mapGB/british-isles-latest.osm.pbf postcode.so.osm.xml
> > > splitter.log
> > 2>&1
> >
> >
> > # significant lines in ../mkgmap.opt
> > max-jobs
> > gmapsupp
> > code-page=1252
> > index
> > bounds=../bounds.zip
> > location-autofill=is_in,nearest
> > no-housenumbers
> > name-tag-list=name:en,int_name,name,place_name,loc_name
> > family-name=osm
> > series-name=OpenStreetMap
> > area-name=berkin
> > x-mapset-name=OSM ticker
> > route
> > drive-on=detect,left
> > preserve-element-order
> > add-pois-to-areas
> > generate-sea=multipolygon,extend-sea-sectors,close-gaps=350
> > link-pois-to-ways
> > process-destination
> > process-exits
> > remove-ovm-work-files
> > poi-address
> > verbose
> > order-by-decreasing-area
> > add-boundary-nodes-at-admin-boundaries=0
> >
> > Command line is:
> >
> > java -Xmx1540M -ea -Dlog.config=../mkgmap_log.props -jar
> > ../mkgmap.noNod/mkgmap.jar -c ../mkgmap.opts --no-style-file --x
> > -check
> > -routing-island-len=700 --family-id=7421 --family-name=Hants -c
> > template.args ../mytyp.txt
> >
> > I'm going to be out for the next few hours, but can do the log
> > stuff
> > later if you need it.
> >
> > Ticker
> >
> > On Tue, 2019-10-22 at 16:40 +0000, Gerd Petermann wrote:
> > > Hi Ticker,
> > >
> > > please let me know the mkgmap options and the area for 74210001.
> > > Maybe try with commented the log statements (except the one for
> > > the
> > > timing )
> > >
> > > Gerd
> > >
> > > ________________________________________
> > > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im
> > > Auftrag
> > > von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> > > Gesendet: Dienstag, 22. Oktober 2019 18:33
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
> > >
> > > Hi Gerd
> > >
> > > These are with default style.
> > >
> > > mkgmap-NET-no-NOD-r4304 --x-check-routing-island-len=700
> > >
> > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > 74210002.osm.pbf:
> > > check for routing islands found 38 islands
> > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > 74210002.osm.pbf:
> > > routing island check took 689 ms
> > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > 74210001.osm.pbf:
> > > check for routing islands found 554 islands
> > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > 74210001.osm.pbf:
> > > routing island check took 8279 ms
> > >
> > > Total time taken: 1 minute 12 seconds
> > >
> > > gmapsupp.img 15126528 bytes
> > > Block size: 2048
> > > . 22016
> > > MAKEGMAP.MPS 138
> > > 00007421.MDR 728896
> > > 000MYTYP.TYP 1580
> > > 74210001.RGN 5870951
> > > 74210001.TRE 45927
> > > 74210001.LBL 624175
> > > 74210001.NET 1347117
> > > 74210001.NOD 2384384
> > > 74210002.RGN 2120704
> > > 74210002.TRE 17330
> > > 74210002.LBL 229542
> > > 74210002.NET 592062
> > > 74210002.NOD 1124864
> > > 00007421.SRT 912
> > >
> > > With r4295:
> > >
> > > Total time taken: 1 minute 3 seconds
> > >
> > > gmapsupp.img 15943680 bytes
> > > Block size: 2048
> > > . 23040
> > > MAKEGMAP.MPS 138
> > > 00007421.MDR 728896
> > > 000MYTYP.TYP 1580
> > > 74210001.RGN 5870951
> > > 74210001.TRE 45927
> > > 74210001.LBL 624175
> > > 74210001.NET 1354678
> > > 74210001.NOD 2975539
> > > 74210002.RGN 2120704
> > > 74210002.TRE 17330
> > > 74210002.LBL 229542
> > > 74210002.NET 593443
> > > 74210002.NOD 1343157
> > > 00007421.SRT 912
> > >
> > > Ticker
> > >
> > > On Tue, 2019-10-22 at 16:02 +0000, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > >
> > > > strange, in my tests the test for routing islands performed in
> > > > <
> > > > 50
> > > > ms. So, there seems to be a special case.
> > > > Also, I did not see that big effect on img siz.
> > > > Please can you test with default style for comparison?
> > > >
> > > > Gerd
> > > >
> > > > ________________________________________
> > > > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im
> > > > Auftrag
> > > > von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> > > > Gesendet: Dienstag, 22. Oktober 2019 17:51
> > > > An: Development list for mkgmap
> > > > Betreff: Re: [mkgmap-dev] Please test branch NET-no-NOD
> > > >
> > > > Hi Gerd
> > > >
> > > > Testing mkgmap-NET-no-NOD-r4304 with 2 tile local area with
> > > > option
> > > > --x-check-routing-island-len=700
> > > >
> > > > Some build statistics:
> > > >
> > > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > > 74210002.osm.pbf:
> > > > check for routing islands found 293 islands
> > > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > > 74210002.osm.pbf:
> > > > routing island check took 5252 ms
> > > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > > 74210001.osm.pbf:
> > > > check for routing islands found 2820 islands
> > > > SEVE: uk.me.parabola.imgfmt.app.net.RoadNetwork
> > > > 74210001.osm.pbf:
> > > > routing island check took 45755 ms
> > > >
> > > > Total time taken: 1 minute 53 seconds
> > > > gmapsupp.img 13719552 bytes
> > > > block size: 2048
> > > > . 20992
> > > > MAKEGMAP.MPS 138
> > > > 00007421.MDR 1121108
> > > > 000MYTYP.TYP 1580
> > > > 74210001.RGN 3802657
> > > > 74210001.TRE 17874
> > > > 74210001.LBL 758810
> > > > 74210001.NET 1536856
> > > > 74210001.NOD 2556416
> > > > 74210002.RGN 1645275
> > > > 74210002.TRE 8820
> > > > 74210002.LBL 268782
> > > > 74210002.NET 718899
> > > > 74210002.NOD 1241600
> > > > 00007421.SRT 912
> > > >
> > > > For comparison, testing with r4295 (I get similar figures for
> > > > trunk+):
> > > >
> > > > Total time taken: 1 minute 8 seconds
> > > > gmapsupp.img 14710784 bytes
> > > > Block size: 2048
> > > > . 22016
> > > > MAKEGMAP.MPS 138
> > > > 00007421.MDR 1121108
> > > > 000MYTYP.TYP 1580
> > > > 74210001.RGN 3802683
> > > > 74210001.TRE 17874
> > > > 74210001.LBL 758810
> > > > 74210001.NET 1562592
> > > > 74210001.NOD 3285327
> > > > 74210002.RGN 1645275
> > > > 74210002.TRE 8820
> > > > 74210002.LBL 268782
> > > > 74210002.NET 722291
> > > > 74210002.NOD 1477491
> > > > 00007421.SRT 912
> > > >
> > > > So, for my test case, with my style, which probably creates
> > > > quite
> > > > a
> > > > few
> > > > more isolated networks than the default style:
> > > >
> > > > It adds quite a lot of processing time (increased by 66%)
> > > >
> > > > Reduces the NOD size significantly (22%) and full gmapsupp.img
> > > > by
> > > > 6%.
> > > >
> > > > It fixes a lot of "Route calculation errors" where the nearest
> > > > road
> > > > to
> > > > the start/end point is an isolated network. However the
> > > > behaviour
> > > > of
> > > > this changes in a way not apparent to the map user if what
> > > > should
> > > > have
> > > > been an isolated network overlaps tiles.
> > > >
> > > > I vote for keeping something like this provided the routing
> > > > -island
> > > > detection only happens of the length option is specified (needs
> > > > a
> > > > better name than --x-check-routing-island-len)
> > > >
> > > > The unconnected road handling should be moved out of
> > > > StyledConverter
> > > > and handled as part of the length controlled general logic.
> > > >
> > > > Ticker
> > > >
> > > >
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > mkgmap-dev at lists.mkgmap.org.uk
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > mkgmap-dev at lists.mkgmap.org.uk
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev at lists.mkgmap.org.uk
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev at lists.mkgmap.org.uk
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
- Previous message: [mkgmap-dev] Please test branch NET-no-NOD
- Next message: [mkgmap-dev] Please test branch NET-no-NOD
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the mkgmap-dev mailing list