[mkgmap-dev] Idea: map layers (multiple output tiles per input tile)

Fri Apr 15 22:23:03 BST 2011

On Sun, Apr 10, 2011 at 09:18:00PM +0200, WanMil wrote:
>> On Sun, Apr 10, 2011 at 07:42:53PM +0200, WanMil wrote:
>>> * The advantage to create multiple img files in one mkgmap run is that
>>> parsing and preparing of the OSM data must happen once only. Do you
>>> have numbers how many percent of the time is used for these steps?
>>
>> No, I haven't collected any profiling data yet. Which tool would you
>> recommend? For C and C++, which is what I have mainly been developing
>> in, I have been using OProfile and before that, gprof. Which Java tools
>> would come closest?
>
>I would add some time logging to the mkgmap source code. There are
>profiles for Java but I think they are not useful for such a job.

I made an experiment. Generating the full map with the default style 
from 9 osm.gz tiles on a dual-core machine took some 7 minutes, with 
assertions enabled.  With assertions disabled, generating a routes layer 
of the whole Finland takes about 2.5 minutes when parsing the whole 
country extract in osm.pbf (one input tile, thus one core used). I 
repeated this test for 5 layers, which I split from the currently rather 
useless --style=routes. These styles were routes-foot 
(route=foot|route=hiking), routes-bicycle, routes-bus, routes-rail 
(route=subway|route=train|route=tram), and routes-ferry. I did not 
generate routes-ski or routes-road.

Each of these 5 minimal styles would spit out a number of multipolygon 
error messages, even though the styles are this simple:

cat > relations << EOF
type=route & route=ferry
{
   add ref='${name}'; # if ref is missing, use name
   set ref='${network} ${ref}' | '${ref}' | '${network}';
   apply { set mkgmap:route='$(mkgmap:route),${ref}' | '${ref}' }
}
EOF
cat > lines << EOF
mkgmap:route=* { name '${mkgmap:route}' } [0x1b resolution 20]
EOF

I think that 2.5 minutes for these simple layers is too much. Because 
the processing time per layer did not vary much, no matter how much data 
was generated (13312 bytes .img of ferry routes, 986624 bytes of bus 
routes), I would guess that parsing is dominating the execution time, 
and the processing time would reduce from N*2.5 minutes to about 2.5 
minutes if we implemented a 'multiple output tiles from one input tile' 
feature.

Do you have any hints where to add the "poor man's profiler" timestamp 
printouts to the mkgmap source code?

	Marko