[mkgmap-dev] Splitter question

Thu Nov 25 13:17:07 GMT 2010

I think I've been successfully working around this 'problem' without the 
need to hack Splitter, but Chris Miller did changed some things based on 
off-list communication (Details elude me at this moment though) to make 
this possible.

My process in splitting the planet is essentially a 2-step method:

1. Initial split to generate large tiles, some of which might be 
processed successfully. The ones that don't pass go into the second step.

2. A recursive process in which:
2a. Split the tile with a diminishing --max-nodes setting until Splitter 
returns at least two subtiles.
2b. The resulting subtiles can be processed by Mkgmap, or RoadMap in the 
Danny's case. If a subtile can't be processed then goto 2.

This way I can create tiles as large as possible but with varying 
amounts of nodes in an automated way, without the need for manually 
defined tile definitions.

The only prerequisite for this process is the ability to 'know' when the 
processing of a splitted tile is successful.

It is essentially the same process as the 'doit' and 'deeper' scripts 
but in an recursive way so that the depth of the process is variable 
depending on the actual needs instead of fixed 2 levels.

If there is interest in my script (PHP) then I can clean it up a bit and 
post it here.

On 2010-11-22 18:02, Scott Crosby wrote:
> On Sun, Nov 21, 2010 at 1:04 PM, Danny Backx <danny.backx at scarlet.be
> <mailto:danny.backx at scarlet.be>> wrote:
>
>     I've taken some time and played with the suggestions given.
>
>     I'm including two scripts. One (doit.nl <http://doit.nl>) is a
>     sample script to split the
>     OSM file for the Netherlands into manageable pieces, and then to convert
>     the OSM files into RoadMap format. This mostly works :
>     - I get 339 map files
>     - 9 maps turn out to be too big still for RoadMap
>
>     So I created another script (deeper.nl <http://deeper.nl>, used in a
>     subdirectory, you'll
>     notice that in the script) that'll take the files I put in the
>     subdirectory, and divide them in four smaller chunks.
>
>     I used deeper.nl <http://deeper.nl> and still had problems with 3 of
>     the smaller chunks so
>     I reran deeper.nl <http://deeper.nl> one more time, producing maps
>     that were (again) four
>     times smaller than the offending OSM files.
>
>
>
>     I realize that I'm working around something splitter is leaving
>     unsolved, and I'm partially automating this, my question is whether all
>     this is a good idea.
>
>
> If you're willing to do some work on the splitter to improve it, I think
> that yes, you could improve it in a few days of work to the point where
> this becomes a non-issue for you, and the splitter generates excellent
> and well-balanced tiles.
>
> The core problem is that the splitter needs to track the point density
> around the world very densely. It, divide the world into, say 8192 tiles
> horizontally and 8192 vertically. It goes through the planet it counts
> how many nodes there are for each tile, and then uses that summary
> 'density map' table to derive the areas for splitting, by, say. Your
> problem is that you want regions smaller than 1*1, because even a 1*1
> has too much data for your program. Increasing the density that stuff is
> tracked, to 32k*32ktiles across the world increases memory by 16x and
> might not solve your problem because the the minimum sized tile of 1*1
> may still have too many nodes, even if it is smaller in geographic size.
>
> The fix for you is that dividing Europe into, say 16k*16k tiles
> generates tiles with a smaller geographic extent than dividing the world
> into tiles of 32k*32k. You should be able to feed the geographic bounds
> data from the <bound> tag to configure the density map to only cover
> part of the world. There are a few catches in that you'll have to obey
> the alignment constraints that are required by mkgmap. (Which, alas, I
> do not know.). But that would solve your problem and might make the
> splitter more useful to others..
>
> There are also a few smaller-scale things that may help. For instance,
> the minimum are size in the splitter is 2*2 tiles, not 1*1 tiles.
>
> Scott
>
>
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev