[mkgmap-dev] Regression causing Heap space exhaustion?

Sat Aug 22 00:53:21 BST 2009

Morten Kjeldgaard <mok at bioxray.au.dk> writes:

> On 21/08/2009, at 23.26, Steve Ratcliffe wrote:
>
>> When support for relations was added the memory required increased.  I
>> think that was added along with routing, I believe, which was in  
>> version
>> 684 or thereabouts, so that will explain what you see.
>>
>> I don't use debian, but I just downloaded their package to have
>> a look at their mkgmap script.  If you add the option -Xmx512m after
>> the $JAVACMD in the line:
>>
>>   exec $JAVACMD -jar /usr/share/mkgmap/mkgmap.jar "$@"
>>
>> That should help.
>
> I've tried at 512M, 1024M, all the way up to 4096M, with the same  
> result. The only difference is at 4G it takes much longer before  
> mkgmap gives up. I don't think there's much point in going further  
> than 4G, the problem must lie elsewhere.

With splitter/mkgmap, the general strategy I use is:

  set heap size to a bit less than RAM, to avoid paging

  use --max-nodes to splitter to make small enough areas.

Here is the script I use (with a mac that has 4G physical ram):

#!/bin/sh

cd $HOME/OSM || exit 1

# rm all img/tdb/etc.
rm -f *.img *.tdb

INPUT=$*

## SPLIT INPUT MAPS
if [ "$INPUT" != "" -o ! -f template.args ]; then
    rm -f 63*.osm.gz

    if [ "$INPUT" = "" ]; then
	INPUT=`ls *.osm.bz2`
    fi

    rm -f areas.list template.args*
    echo "SPLITTING $INPUT:"
    time \
    java \
	-enableassertions \
	-Xmx2000m \
	-jar splitter.jar \
	--max-nodes=1600000 \
	$INPUT \
	> OUT.01.splitter 2>&1

    cp -pf template.args template.args.orig
    ed template.args <<EOF
g/^description:/d
w
EOF
fi

## BUILD MAPS
echo "BUILDING:"
time \
java \
    -enableassertions \
    -Xmx2048m \
    -jar mkgmap.jar \
    --description="OSM_gdt" \
    --country-abbr="US" \
    --country-name="United States" \
    --region-abbr="MA" \
    --region-name="Massachusetts" \
    --series-name="OSM_gdt_series" \
    --family-id=401 \
    --family-name="OSM_gdt_family" \
    --product-id=13 \
    --series-name="OSM_gdt_series" \
    --area-name="OSM_gdt_area" \
    --tdbfile \
    --remove-short-arcs=6 \
    --add-pois-to-areas \
    --route \
    -c template.args \
    --gmapsupp \
    > OUT.02.mkgmap 2>&1

echo OPTIONS not used: \
    --overview-mapname=63240000 - test default claim \
    --ignore-osm-bounds \

## PREPARE GMAPI FORMAT FOR ROADMAP
mkdir -p GMAPI
echo "GMAPI:"
time \
/Users/gdt/SOFTWARE/GMAPIBUILDER/gmapi-builder/gmapi-builder.py \
    -o GMAPI \
    -t 63240000.tdb \
    -b 63240000.img \
    -v 6324*.img \
    > OUT.03.gmapi 2>&1

echo "test-map:all-elements"
export BASE_LAT=42.0
export BASE_LONG=-70.0
time \
java \
    -enableassertions \
    -Xmx2048m \
    -jar mkgmap.jar \
    --overview-mapname=63500000 \
    --mapname=63500001 \
    --description="OSM_test" \
    --country-abbr="US" \
    --country-name="United States" \
    --region-abbr="MA" \
    --region-name="Massachusetts" \
    --series-name="OSM_test_series" \
    --family-id=402 \
    --family-name="OSM_test_family" \
    --product-id=14 \
    --series-name="OSM_test_series" \
    --area-name="OSM_test_area" \
    --tdbfile \
    test-map:all-elements
    > OUT.04.mkgmap 2>&1

mkdir -p GMAPI-TEST
echo "GMAPI-TEST:"
time \
/Users/gdt/SOFTWARE/GMAPIBUILDER/gmapi-builder/gmapi-builder.py \
    -o GMAPI-TEST \
    -t 63500000.tdb \
    -b 63500000.img \
    -v 6350*.img \
    > OUT.05.gmapi-test 2>&1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 193 bytes
Desc: not available
Url : http://lists.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20090821/35a4dc96/attachment.bin 