[mkgmap-dev] Possible splitter bug

Thu Oct 22 18:48:33 BST 2009

Quoting Chris Miller <chris.miller at kbcfp.com>:

> I've been away for a few weeks and won't have much time to work on   
> the splitter
> for a bit longer yet but I've made a note to update the docs on the website
> to bring it in line with the latest splitter version.
>
> As for your original query, there's a few different scenarios so let me try
> to explain each of them:
>
> First off, when --cache isn't specified, the splitter scans through the osm
> file in an initial pass to find all the nodes so it can figure out how they
> are distributed, and hence generate the areas.list file. Because parsing
> the XML is quite slow, as an optimisation the splitter by default   
> stops looking
> for more nodes the moment it discovers the first <way> or <relation> tag
> because it assumes the <node> tags all appear at the top of the file. If
> you happen to have an osm file where the tags are mixed in with each other
> (as is the case with your contour file) you have to specify --mixed option
> to force the splitter to keep scanning the XML right to the very end so all
> the nodes are found on this first pass. Without that, the splitter will only
> find a few nodes and end up generating a bad areas.list file, thus messing
> up the rest of the split.
>
> When the --cache parameter is used, the splitter behaves quite differently.
> It first makes a pass through the *entire* osm file, building up a cache
> of all nodes/ways/rels (while still gathering the node information it needs
> to generate areas.list). Because the entire file is processed during the
> cache generation, it doesn't matter if the XML contains mixed nodes/ways/rels
> or not. So with --cache enabled, the --mixed parameter becomes redundant.
>
> Similarly (and as you point out), if you use --split-file then   
> --mixed becomes
> redundant too because the first pass isn't required.
>
> Hopefully that explains the behaviour you're experiencing. I'll try to get
> the docs (and code) updated to make it a bit clearer what's going on.
>
> Chris

Chris,

Thanks for explaining it...what you've written would be a fantastic  
addition to the wiki even if it just went in verbatim!

I guess one suggestion would be that if you need to use --mixed on  
certain types of data, it would be good if splitter told you that*,  
rather than it having a go but not really doing anything and then not  
obviously failing. In other words (and I stress imho) it's preferable  
from a user perspective for software to fail cleanly with an  
explanation of why it failed, rather than for it to seemingly continue  
to work but not do what it's supposed to do.

* Obviously, if it could do this, it ought to be able to assume  
--mixed implicitly and proceed anyway outputting a warning that this  
is what it was doing!

Charlie