[mkgmap-dev] splitter that generates problem list

Wed Nov 7 18:48:00 GMT 2012

WanMil wrote
>> The profiling data shows that the CPU is most of the time
>> busy with the pbf read and write routines, so I doubt that your disk
>> is the bottleneck.
> 
>  From my experience profiling is a very time consuming (=> CPU 
> consuming) task. So could it also be that the disk is not the bottleneck 
> while you profile the application but it is the bottleneck while you 
> don't profile it? (It's just a guess...)

This depends on the profiling tool. The overhead for 
-agentlib:hprof=cpu=samples,depth=20
is very small (a few percent)

I think that I can say "IO is no bottleneck" because the first real write
function
appears in position 56 with just 0.32% :
56  0.32% 74.98%    1154 301828 java.io.FileOutputStream.write

The read routine appear in position 3:
   3  5.21% 18.39%   18991 300527 java.io.FileInputStream.readBytes

but the pbf routines build most of the top time consumers:
 *  1  7.71%  7.71%   28080 300532 java.util.zip.Inflater.inflateBytes
   2  5.47% 13.18%   19936 301089 java.util.zip.Deflater.deflateBytes
 *  3  5.21% 18.39%   18991 300527 java.io.FileInputStream.readBytes
*   4  5.09% 23.48%   18563 301652 java.util.zip.Deflater.deflateBytes*
   5  3.42% 26.90%   12471 300813
uk.me.parabola.splitter.BinaryMapParser.parseWays
*   6  2.96% 29.86%   10776 301058 java.util.zip.Inflater.inflateBytes*
   7  2.75% 32.61%   10025 301108 uk.me.parabola.splitter.WriterGrid.get
   8  2.20% 34.82%    8032 300643
crosby.binary.Osmformat$Way$Builder.addRefs
   9  2.10% 36.92%    7646 301056 java.io.FileInputStream.readBytes
*  10  1.99% 38.91%    7266 300573
crosby.binary.Osmformat$DenseInfo$Builder.mergeFrom
  11  1.97% 40.88%    7169 300572
crosby.binary.Osmformat$DenseInfo$Builder.mergeFrom
  12  1.93% 42.81%    7030 300577
crosby.binary.Osmformat$DenseNodes$Builder.mergeFrom
  13  1.80% 44.61%    6577 300576
crosby.binary.Osmformat$DenseNodes$Builder.mergeFrom
  14  1.73% 46.34%    6298 300556
crosby.binary.Osmformat$DenseInfo$Builder.mergeFrom
  15  1.65% 47.99%    5998 300553
crosby.binary.Osmformat$DenseInfo$Builder.mergeFrom
  16  1.45% 49.43%    5271 300540
crosby.binary.Osmformat$DenseInfo$Builder.mergeFrom
*

So, I'd say: With pbf format, you don't see much improvement with a faster
disk, but with a faster CPU. 

Ciao,
Gerd

--
View this message in context: http://gis.19327.n5.nabble.com/splitter-that-generates-problem-list-tp5734014p5734710.html
Sent from the Mkgmap Development mailing list archive at Nabble.com.