[mkgmap-dev] Idea: map layers (multiple output tiles per input tile)

Sat Apr 16 20:05:47 BST 2011

Am 15.04.2011 23:23, schrieb Marko Mäkelä:
> On Sun, Apr 10, 2011 at 09:18:00PM +0200, WanMil wrote:
>>> On Sun, Apr 10, 2011 at 07:42:53PM +0200, WanMil wrote:
>>>> * The advantage to create multiple img files in one mkgmap run is that
>>>> parsing and preparing of the OSM data must happen once only. Do you
>>>> have numbers how many percent of the time is used for these steps?
>>>
>>> No, I haven't collected any profiling data yet. Which tool would you
>>> recommend? For C and C++, which is what I have mainly been developing
>>> in, I have been using OProfile and before that, gprof. Which Java tools
>>> would come closest?
>>
>> I would add some time logging to the mkgmap source code. There are
>> profiles for Java but I think they are not useful for such a job.
>
> I made an experiment. Generating the full map with the default style
> from 9 osm.gz tiles on a dual-core machine took some 7 minutes, with
> assertions enabled.  With assertions disabled, generating a routes layer
> of the whole Finland takes about 2.5 minutes when parsing the whole
> country extract in osm.pbf (one input tile, thus one core used). I
> repeated this test for 5 layers, which I split from the currently rather
> useless --style=routes. These styles were routes-foot
> (route=foot|route=hiking), routes-bicycle, routes-bus, routes-rail
> (route=subway|route=train|route=tram), and routes-ferry. I did not
> generate routes-ski or routes-road.
>
> Each of these 5 minimal styles would spit out a number of multipolygon
> error messages, even though the styles are this simple:
>
> cat>  relations<<  EOF
> type=route&  route=ferry
> {
>     add ref='${name}'; # if ref is missing, use name
>     set ref='${network} ${ref}' | '${ref}' | '${network}';
>     apply { set mkgmap:route='$(mkgmap:route),${ref}' | '${ref}' }
> }
> EOF
> cat>  lines<<  EOF
> mkgmap:route=* { name '${mkgmap:route}' } [0x1b resolution 20]
> EOF
>
> I think that 2.5 minutes for these simple layers is too much. Because
> the processing time per layer did not vary much, no matter how much data
> was generated (13312 bytes .img of ferry routes, 986624 bytes of bus
> routes), I would guess that parsing is dominating the execution time,
> and the processing time would reduce from N*2.5 minutes to about 2.5
> minutes if we implemented a 'multiple output tiles from one input tile'
> feature.
>
> Do you have any hints where to add the "poor man's profiler" timestamp
> printouts to the mkgmap source code?
>
> 	Marko

I would like to add some profiling code to mkgmap.
My idea is, that it is possible to add a static class that collects 
profiling information from mkgmap classes. The classes can call:
Profiler.start(String stepname) and Profiler.stop(String stepname).

At the end of mkgmap it creates a statistic file how many time is spent 
in each step. Of course the profiler is active only if a specific option 
is set.

This could be used to track the processing time of mkgmap with 
subsequent commits.

Good idea? Bad idea? Any more ideas?

WanMil