[mkgmap-dev] splitter r261: performance improvements

Sun Dec 16 17:44:17 GMT 2012

Hi Carlos,

thanks for reporting this. I try to reproduce the problem. Do you use the 
latest file from geofabrik or is it an older one?

Gerd

Carlos Dávila-2 wrote
> I'm getting the error below using r261 on south-america.osm.pbf. r257 
> works fine for the same input file and parameters
> Starting multi-tile analyses pass 1
> Processing south-america.osm.pbf
> Bounding box -107.03999990000001 -61.829999900000004 -23.229999900000003 
> 13.25
> Exception in thread "main" java.lang.NegativeArraySizeException
>          at
> crosby.binary.file.FileBlockHead.readHead(FileBlockHead.java:53)
>          at crosby.binary.file.FileBlock.process(FileBlock.java:130)
>          at 
> crosby.binary.file.BlockInputStream.process(BlockInputStream.java:34)
>          at uk.me.parabola.splitter.Main.processMap(Main.java:679)
>          at uk.me.parabola.splitter.Main.writeAreas(Main.java:599)
>          at uk.me.parabola.splitter.Main.split(Main.java:248)
>          at uk.me.parabola.splitter.Main.start(Main.java:154)
>          at uk.me.parabola.splitter.Main.main(Main.java:143)
> El 16/12/12 16:51, GerdP escribió:
>> Hi,
>>
>> I've committed r261: Changes
>> -improve throughput:
>>   + use BufferedInputStream with 4MB buffer for pbf and o5m input
>>   + reduced (oversized) o5m writer hashtable (saves around 635KB for each
>> open output file),
>>   + improved hash function to avoid collisions
>>   + avoid GC because of obsolete new String()
>>
>> On my machine, the effect is great:
>> When splitting germany.o5m with r260, the hard disk is rattling and
>> rattling, and it takes
>> between 75 and 110 secs to split the file (yes, very random results)
>> With r261, the disk sound is normal again and it takes more less
>> constantly 60 secs to split the same file with the same parms.
>>
>> I did some other tests regarding performance and found out that
>> splitter works best when input file is on a different disk (even a slow
>> USB 2.0 drive) and heap (-Xmx) is big enough to avoid massive GC.
>> It has no positive effect if you specify -Xmx7000m when -Xmx1600m is
>> already good enough.
>>
>> Ciao,
>> Gerd
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://gis.19327.n5.nabble.com/splitter-r261-performance-improvements-tp5740592.html
>> Sent from the Mkgmap Development mailing list archive at Nabble.com.
>> _______________________________________________
>> mkgmap-dev mailing list
>> 

> mkgmap-dev at .org

>> http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>>    
> 
> 
> -- 
> Por favor, no me envíe documentos con extensiones .doc, .docx, .xls,
> .xlsx, .ppt, .pptx, .mdb, mdbx
> Instale LibreOffice desde http://es.libreoffice.org/descarga/
> LibreOffice es libre: se puede copiar, modificar y redistribuir
> libremente. Gratis y totalmente legal.
> LibreOffice está en continuo desarrollo y no tendrá que pagar por las
> nuevas versiones.
> 
> _______________________________________________
> mkgmap-dev mailing list

> mkgmap-dev at .org

> http://lists.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
View this message in context: http://gis.19327.n5.nabble.com/splitter-r261-performance-improvements-tp5740592p5740601.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.