[mkgmap-dev] [PATCH v1] Experimental support for multi-threading

Sat May 9 16:22:15 BST 2009

Hi Folks,

I have recently gained access to a monster box containing 16 3GHz
cores and 32 G of memory so I thought it would make an ideal machine to
test a multi-threaded version of mkgmap.

As a lot of the processing has to be done in sequence, the
opportunities for using multiple threads are somewhat limited.
However, the attached patch parallelises:

1 - processing of ways from OSM form to MapLine/MapRoad.

2 - processing of cities

3 - processing of POIs

4 - processing of polylines and polyshapes within each division

At this time, the performance gain is useful but not stunning
(typically, 25-30% speedup when using at least 4 cores). There is a
small gain even when only using 2 cores.

With the patch in place, it will default to using as many threads as
you have cores. If you wish, you can explicitly specify the number of
threads to use with the --num-threads=N option. You may specify more
threads than you have cores but it won't make it any faster. Specifying
less threads than cores is useful for testing purposes or disabling the
parallel processing (by specifying just 1 thread).

One side effect of the multi-threading is that the elements are
written to the output file in a random order. If you specify the
(existing) --preserve-element-order option, the order will be preserved
but all of the threading will be disabled!

The patch has received some testing using mapsource and I think I have
removed the worst of the bugs but I would very much appreciate it if
people could try it out and see if it causes any breakage.

Obviously, I would also like to know how well it performs.

As it has only been tested under Linux any reports from other platforms
would be especially useful.

Cheers,

Mark
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mb-multithread-v1.patch
Type: text/x-patch
Size: 45271 bytes
Desc: not available
Url : http://lists.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20090509/6114f943/attachment.bin 