[mkgmap-dev] [PATCH v1] LocationHook speedup

Tue Jan 3 21:13:24 GMT 2012

Gerd,

don't worry. The link to the description was broken on the new help 
page. I've just corrected it before I replied to you....

The last time (beginning of December) I've created the boundaries for 
europe with -Xmx3g and that was *very* close to the OOME.

But there is a workaround which I also use to create the world 
boundaries: One can workout smaller overlapping parts (containing 
complete countries!) and merge the results of the smaller parts:
java uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryMerger 
<boundsdir1> <boundsdir2> <resultdir>

WanMil

> Hello WanMil,
>
> Shame on me. I finally found that the description is on the new help
> page, the place I should have visited first ...
>
> By the way: On windows, r2159 is not able to process the european
> boundaries, but the patch for Coord.java seems to allow the action on
> windows. So I think this is an argument for it.
> Working with these large files is still no fun with mkgmap, so I hope I
> can find something which really improves performance ;-)
>
> Gerd
>
>
> G:\mkgmap_all\trunk\dist>java -Xmx1600m -jar mkgmap.jar
> --createboundsfile=f:\osm\europe-boundaries.osm
> Time started: Tue Jan 03 20:05:54 CET 2012
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at sun.awt.geom.Order1.getReversedCurve(Order1.java:198)
> at sun.awt.geom.Curve.getWithDirection(Curve.java:706)
> at sun.awt.geom.CurveLink.getSubCurve(CurveLink.java:56)
> at sun.awt.geom.AreaOp.pruneEdges(AreaOp.java:389)
> at sun.awt.geom.AreaOp.calculate(AreaOp.java:141)
> at java.awt.geom.Area.pathToCurves(Area.java:177)
> at java.awt.geom.Area.<init>(Area.java:108)
> at uk.me.parabola.util.Java2DConverter.createArea(Java2DConverter.java:60)
> at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryRelation.processElements(BoundaryRelation.java:276)
> at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryElementSaver.addRelation(BoundaryElementSaver.java:132)
> at
> uk.me.parabola.mkgmap.reader.osm.xml.Osm5XmlHandler$SaxHandler.endElement(Osm5XmlHandler.java:182)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:601)
> at
> com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.endElement(XIncludeHandler.java:1016)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScan
> nerImpl.java:1782)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocument
> FragmentScannerImpl.java:2939)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
> at
> com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScanne
> rImpl.java:511)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
> at
> com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
> at
> com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
> at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
> at javax.xml.parsers.SAXParser.parse(SAXParser.java:198)
> at
> uk.me.parabola.mkgmap.reader.osm.xml.Osm5MapDataSource.load(Osm5MapDataSource.java:72)
> at
> uk.me.parabola.mkgmap.reader.osm.boundary.BoundaryPreparer.run(BoundaryPreparer.java:105)
> at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:341)
> at
> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126)
> at uk.me.parabola.mkgmap.main.Main.main(Main.java:114)
>
> Gerd
>
>  > Date: Tue, 3 Jan 2012 20:38:42 +0100
>  > From: wmgcnfg at web.de
>  > To: mkgmap-dev at lists.mkgmap.org.uk
>  > Subject: Re: [mkgmap-dev] [PATCH v1] LocationHook speedup
>  >
>  > Hi Gerd,
>  >
>  > don't use osmosis: it's *very* slow for such a job.
>  >
>  > Better use the osmconvert/osmfilter toolchain. That's magnitudes
>  > faster... You can find a how to at:
>  >
> http://wiki.openstreetmap.org/wiki/Mkgmap/help/options#Using_precompiled_bounds_for_the_address_index
>  >
>  > WanMil
>  >
>  > > Hello WanMil,
>  > >
>  > > thanks for the util. I am still trying to produce the input for
> mkgmap with
>  > > osmosis. The program produces very large temp files, and after hours it
>  > > crashed because the disk was full :-(
>  > >
>  > > Maybe I am not using the best parms? Or doesn't it work on windows?
>  > > I use this:
>  > > osmosis -v --rb f:\osm\europe.osm.pbf --tf accept-relations
>  > > boundary=administrative --used-node idTrackerType=BitSet --used-way
>  > > idTrackerType=BitSet --wb boundary_relations.osm.pbf
>  > >
>  > > Maybe I can download the result somewhere?
>  > >
>  > > Ciao,
>  > > Gerd
>  > >
>  > > --
>  > > View this message in context:
> http://gis.638310.n2.nabble.com/PATCH-v1-LocationHook-speedup-tp7135704p7147396.html
>  > > Sent from the Mkgmap Development mailing list archive at Nabble.com.
>  > > _______________________________________________
>  > > mkgmap-dev mailing list
>  > > mkgmap-dev at lists.mkgmap.org.uk
>  > > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>  >
>  > _______________________________________________
>  > mkgmap-dev mailing list
>  > mkgmap-dev at lists.mkgmap.org.uk
>  > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev