[mkgmap-dev] Performance with --bounds option

Thu Mar 19 15:18:12 GMT 2020

Hi Ticker,

OK, those numbers look plausible. I see 24 secs without --bounds and 1 minute 49 seconds with --bounds.
The latter value goes down to 1 minute 19 seconds when I use -Xmx2G
I've tried my idea reg. splitting of the complex boundary relations and found no effect on runtime, only a small increase in the bounds.zip.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
Gesendet: Donnerstag, 19. März 2020 15:47
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Performance with --bounds option

Hi Gerd

These are the timings I get for the single tile 74220001:

trunk with --bounds=bounds.zip
 Total time taken: 3 minutes 44 seconds

r4295 with --bounds=bounds.zip
 Total time taken: 3 minutes 51 seconds

trunk with --no-bounds
 Total time taken: 1 minute 11 seconds

r4295 with --no-bounds
 Total time taken: 1 minute 6 seconds

so the latest version is about the same speed as versions before the
is_in merge.

What I meant by 'acceptable' was that the extra time for bounds
processing, although bad for this tile, is not a concern.
Building britain-and-ireland with bounds takes 36 minutes, without it
takes 29 minutes.

Ticker

On Thu, 2020-03-19 at 12:46 +0000, Gerd Petermann wrote:
> Hi Ticker,
>
> when you say acceptable does that mean that latest mkgmap is still
> much slower than e.g. r4290?
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> Gesendet: Donnerstag, 19. März 2020 11:04
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Performance with --bounds option
>
> Hi Gerd
>
> With the reversion of the code in r4469, the performance is
> acceptable
> again. It is only a few tiles that have this problem and, for GB,
> they
> don't add significantly to the overall map generation time
>
> Ticker
>
>
> On Thu, 2020-03-19 at 06:52 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> >
> > some more details:
> > I didn't think of boundary relations like r1959008 which contains >
> > 700 outer rings. For each node close to that boundary mkgmap called
> > Java2DConverter.areaToSingularAreas() which calculated all the
> > areas
> > again and again.
> > In Germany, boundary relations typically have only one ring ;)
> > I think I should look into the code which compiles bounds.zip,
> > somehow I expected that those complex areas are split to singular
> > areas, might improve performance.
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> > von Gerd Petermann <gpetermann_muenchen at hotmail.com>
> > Gesendet: Mittwoch, 18. März 2020 19:46
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Performance with --bounds option
> >
> > Hi Ticker,
> >
> > I forgot to remove some experimental code for is-in branch in
> > BoundaryQuadTree.
> > Fixed with r4469.
> > I'll probably also remove the methods pointInsideArea() and
> > pointInsideSingularArea() in BoundaryUtil.
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> > von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> > Gesendet: Mittwoch, 18. März 2020 18:16
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Performance with --bounds option
> >
> > Have done:
> >
> > http://files.mkgmap.org.uk/download/462/74220001.osm.pbf
> >
> > Ticker
> >
> > On Wed, 2020-03-18 at 16:26 +0000, Gerd Petermann wrote:
> > > Hi Ticker,
> > >
> > > will look at it tomorrow. Maybe you can upload the file to
> > > http://files.mkgmap.org.uk/
> > >
> > > Gerd
> > >
> > > ________________________________________
> > > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im
> > > Auftrag
> > > von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> > > Gesendet: Mittwoch, 18. März 2020 11:56
> > > An: Development list for mkgmap
> > > Betreff: Re: [mkgmap-dev] Performance with --bounds option
> > >
> > > Hi Gerd
> > >
> > > It's
> > >
> > > 74220001: 2674688,-700416 to 2811904,-233472
> > > #       : 57.392578,-15.029297 to 60.336914,-5.009766
> > >
> > > Ticker
> > >
> > > On Wed, 2020-03-18 at 10:46 +0000, Gerd Petermann wrote:
> > > > Hi Ticker,
> > > >
> > > > please tell me the tile boundaries
> > > >
> > > > Gerd
> > > >
> > > >
> > > > ________________________________________
> > > > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im
> > > > Auftrag
> > > > von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> > > > Gesendet: Mittwoch, 18. März 2020 11:43
> > > > An: mkgmap development
> > > > Betreff: [mkgmap-dev] Performance with --bounds option
> > > >
> > > > Hi Gerd
> > > >
> > > > With the current version I have a tile that takes 3.5 hours to
> > > > build.
> > > > With an old version (r4295) it took about 3 minutes. Without -
> > > > -bounds
> > > > it takes 1 minute 15 secs.
> > > >
> > > > Relevant options are:
> > > >  --bounds=bounds.zip --location-autofill=is_in,nearest
> > > >
> > > >
> > > > From the mkgmap.log files:
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: Checking bounds dir took 66 ms
> > > > ...
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: Starting with location hook
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> > > >  ../mapGB/74220001.osm.pbf: loading boundary file:
> > > > bounds_2650000_
> > > > -750000.bnd
> > > > ... about 40 similar messages ...
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryUtil
> > > >  ../mapGB/74220001.osm.pbf: loading boundary file:
> > > > bounds_2800000_
> > > > -250000.bnd
> > > > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> > > >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > > > available for raster tile 2750000_-400000
> > > > ... 16 similar messages ...
> > > > WARN: uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryGrid
> > > >  ../mapGB/74220001.osm.pbf: no precompiled boundary information
> > > > available for raster tile 2800000_-250000
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 97834
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: Location hook finished in 12747318
> > > > ms
> > > >
> > > > The old version had similar lines, but the stats at the end
> > > > are:
> > > >
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: ======= LocationHook Stats =====
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: QuadTree searches    : 95639
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: unsuccesfull         : 20
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: unsuccesfull for ways: 9
> > > > INFO: uk.me.parabola.mkgmap.reader.osm.LocationHook
> > > >  ../mapGB/74220001.osm.pbf: Location hook finished in 113704 ms
> > > >
> > > > Ticker
> > > >
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > mkgmap-dev at lists.mkgmap.org.uk
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > mkgmap-dev at lists.mkgmap.org.uk
> > > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev at lists.mkgmap.org.uk
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev at lists.mkgmap.org.uk
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev