[mkgmap-dev] Binary OSM file support?

Sun Sep 12 15:25:30 BST 2010

My internal repository has several independent development threads.
Only one of them is the binary format.  The other development threads
included patches that are already in and some have not been
benchmarked/tested thoroughly.

 * The binary format.
 * Improved double writing for XML output.
 * Alternate threading design.
 * Avoid sending each node to each osmwriter to do a bbox check.
(Depends on some of the binary format refactoring.)
 * too-many-areas. (already in)
 * Tag representation (partially in; not benchmarked)

Git knows which patch is in which thread.

The two critical ones for the binary format are the binary format
patches and the ULP patch in the double-writing thread.

The patches where I avoid sending each node to each OSM writer
improved the Big-O from O(n) to O(sqrt(n))), but the benchmarks seem
to be the identical performance. I think things are bottlenecked
updating the large shared arrays, which may also be why there's no
need to worry about the threading design making one thread per writer;
they're all idle waiting for the bottleneck to dribble out stuff to
write.

Scott

On Sun, Sep 12, 2010 at 8:25 AM, Chris Miller
<chris_overseas at hotmail.com> wrote:
> Hello Jeffrey,
>
> Thanks very much for this. It's likely to be a while before I get a chance
> to look at and apply these sorry, but hopefully will find some time next
> weekend. Perhaps in the meantime if you have a current build of the splitter
> that includes these patches you could make it available for others to test
> and provide some initial feedback on?
>
> The one thing I'm not sure about is the last one with the new thread design
> - I know it provides a decent performance boost, but I'm a little uneasy
> with the way it achieves this by using more threads than CPU cores. AFAIK
> that patch is incidental to the rest of the binary file support though so
> shouldn't affect this transition.
>
> Cheers,
> Chris
>
>> Here's a quick rebasing of the patches in Scott's splitter repository
>> to the current splitter trunk.  My Java dev system is down for some
>> upgrades at the moment so I haven't even compiled these yet but
>> hopefully I didn't do too bad of a job of resolving the conflicts.
>
>
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>