[mkgmap-dev] RE Commit r3801: merge split-shape branch

Wed Feb 15 17:12:57 GMT 2017

Hi Ticker, that stops the unbound recursion, however the map it produces is faulty. The only areas that show up as land are complete rectangles within the mainland (see attachment). Even small islets do not have any land area. Of course this may be a different problem.

Regards,
Mike

-----Original Message-----
From: Ticker Berkin [mailto:rwb-mkgmap at jagit.co.uk] 
Sent: 15 February 2017 08:53
To: mkgmap development <mkgmap-dev at lists.mkgmap.org.uk>
Subject: Re: [mkgmap-dev] RE Commit r3801: merge split-shape branch

Hi Mike

OK - I think I understand the problem.

Can you try with --order-by-decreasing-area and let me know if that
stops the unbound recursion, and, in the meantime, I'll work on it.

Thanks
Ticker

On Wed, 2017-02-15 at 00:48 +0000, Mike Baggaley wrote:
> HI Ticker, having run this under the debugger now, it appears that it
> is not the postcode data after all - I was getting confused between
> the too small to split messages and too many POIs messages. I was
> previously getting too small to split messages in splitter, but not
> in mkgmap.
> 
> I temporarily changed the code to:
> 
> 				} else if (mustSplit) { // can't reduce
> size, so force more subdivisions
> 					if (depth < 1000) {
> 						log.debug("splitting
> area by contents", area);
> 						MapArea[] sublist =
> area.split(1, 1, bounds, true);
> 						addAreasToList(sublist,
> alist, depth + 1);
> 						continue;
> 					} else {
> 						log.error("Area too
> small to split at " + area.getBounds().getCenter().toOSMURL() + "
> (reduce the density of points, length of lines, etc.)");		
> 				
> 					}
> 				}
> 
> This ran successfully and gave a single error:
>  Area too small to split at 
> http://www.openstreetmap.org/?mlat=51.797190&mlon=0.918646&zoom=17 (r
> educe the density of points, length of lines, etc.
> 
> Stopping in the debugger, I can see that sizes[MapArea.SHAPE_KIND] is
> 68076 and from the coordinate it is processing the coastline. The
> call stack has the same value for the sizes[MapArea.SHAPE_KIND] in
> every instance of addAreasToList, so it looks like the call to
> area.split is not actually splitting. Looking at the resulting file
> in BaseCamp, I can see that the coastline is corrupt.
> 
> Hope that helps,
> Mike
> 
> -----Original Message-----
> From: Ticker Berkin [mailto:rwb-mkgmap at jagit.co.uk] 
> Sent: 14 February 2017 23:21
> To: Mike Baggaley <mike at tvage.co.uk>
> Subject: Re: [mkgmap-dev] RE Commit r3801: merge split-shape branch
> 
> Hi Mike
> 
> I have the raw postcode data (codePointOpen.zip) which I processes
> just
> to take the first few at the same point but I can change it do all of
> them individually. Do you feed it with a map through the splitter or
> just make a postcode only overlay? What is the bit of style you use
> to
> represent each postcode? I can probably knock up something similar.
> 
> The code should cope - I can't see anything obviously wrong so need
> to
> try and reproduce the problem
> 
> Ticker 
> 
> On Tue, 2017-02-14 at 18:17 +0000, Mike Baggaley wrote:
> > Hi Ticker the data is postcode centres (points). As well as
> > geographic postcodes, the Royal Mail postcode data includes a
> > number
> > of postcodes that are centred on sorting offices (presumably for
> > box
> > numbers). There are quite a few at large sorting offices, but if
> > each
> > split should be able to accept 255 points, and it is exceeding 2000
> > splits before crashing, it looks like the splitting is not working 
> > -
> > there won't be more than 2000 * 255 postcodes at the same place. My
> > OSM file is rather large. I can see whether I can extract a map
> > centred on a sorting office.
> > 
> > Regards,
> > Mike
> > 
> > -----Original Message-----
> > From: Ticker Berkin [mailto:rwb-mkgmap at jagit.co.uk] 
> > Sent: 14 February 2017 17:52
> > To: mkgmap-dev at lists.mkgmap.org.uk
> > Subject: Re: [mkgmap-dev] RE Commit r3801: merge split-shape branch
> > 
> > Hi Mike
> > 
> > This is an area of code I've changed and it should be able to cope
> > with
> > many items at the same location without recursing.
> > 
> > Do you have some sample data? what are the items (point/line/shape
> > extended?)
> > 
> > Ticker
> > 
> > On Tue, 2017-02-14 at 17:40 +0000, Mike Baggaley wrote:
> > > Hi Gerd, since this change I am getting a
> > > java.lang.StackOverflowError crash
> > > caused by the code recursively attempting to split something
> > > which
> > > is
> > > unsplittable (assuming the split is based on location), as I have
> > > a
> > > large
> > > number of points at exactly the same location (from external data
> > > I
> > > add to
> > > the OSM data).
> > > 
> > > The offending line is at
> > > uk.me.parabola.mkgmap.build.MapSplitter.addAreasToList(MapSplitte
> > > r.
> > > ja
> > > va:187)
> > > . It is failing on my system with a depth of 2342. I suggest
> > > there
> > > needs to
> > > be a maximum depth after which it should give up trying to split.
> > > 
> > > 				} else if (mustSplit) { // can't reduce
> > > size, so force more subdivisions
> > > 					log.debug("splitting area by
> > > contents", area);
> > > 					MapArea[] sublist =
> > > area.split(1, 1,
> > > bounds, true);
> > > 					addAreasToList(sublist, alist,
> > > depth
> > > + 1);
> > > 					continue;
> > > 				}
> > > 
> > > I am unsure whether you would want a fixed depth limit, would
> > > want
> > > it
> > > to be
> > > set by parameter, would prefer to catch the error or think it
> > > would
> > > be
> > > possible to see whether the proposed area split has made any
> > > improvement,
> > > and give up if it hasn't.
> > > 
> > > Regards,
> > > Mike
> > > 
> > > 
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev at lists.mkgmap.org.uk
> > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > 
> > 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: map.jpg
Type: image/jpeg
Size: 199361 bytes
Desc: not available
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20170215/d30469ca/attachment-0001.jpg>