[mkgmap-dev] Patch to reduce memory usage by interning strings.
From WanMil wmgcnfg at web.de on Fri Apr 2 11:15:07 BST 2010
Am 02.04.2010 06:16, schrieb Scott A Crosby: > On Thu, 01 Apr 2010 12:10:22 +0100, Steve Ratcliffe<steve at parabola.me.uk> writes: > >> On 31/03/10 14:56, Scott A Crosby wrote: >>> I noticed that mkgmap does not intern any strings. In particular, this >> >> This is true. There is a pending patch that deals with excessive >> memory use in a slightly different way which is on the style-speed >> branch, with a pre-built one available at the bottom of the mkgmap >> download page. > > I pulled it out of SVN at 1569. > >> I drops all tags that are not used by the current style and as an >> extra feature it ensures that there is only copy of all the strings >> on the key side. >> >> Could you try it out please. It may be worth also interning the values >> but when I was looking at it there was much less benefit for the maps >> I was looking at (but it might vary with the input). >> I'd be happy to apply a patch that interned the values too if there >> was a decent memory saving without a significant performance drop. > > The style-speed branch builds the tile in question within 1gb of ram > using the default style. I suspect that that style isn't using the > problematic tags. A different style that did use the tags would likely > still blow up. This seems fragile. > > I benchmarked the problematic tile on the style-speed branch with and > without interning all keys and values in the Tag() constructor using > ordinary String.intern(). The 18 character change implementing > interning seems to increase performance by about a second, from 66 to > 65 seconds. > > I'd say to go for it. > > Scott Scott, thanks for posting hard performance values! That's good news because it makes possible to use a simple intern() solution. Before I read your post I compared some german tiles. The single threaded SVN r1625 took 328s with the default style. Using the intern() patch it took 335s. That seems to be an unsignificant change. I am not able to post multithreaded comparisons because my geriatric CPU has one core only... I also used the VisualGC plugin of JVisualVM to view the GC. The permanent space (where all the interned Strings are stored) did not exceed 11mb. So interning all Strings will probably not exceed the permanent space. However we might add a note to the README file that in some special cases it is necessary to increase the permanent size with -XX:MaxPermSize=128m (or more) to avoid OutOfMemoryExceptions. WanMil
- Previous message: [mkgmap-dev] Patch to reduce memory usage by interning strings.
- Next message: [mkgmap-dev] State of the coastline art
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the mkgmap-dev mailing list