<!DOCTYPE html>
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi,</p>
<p>switching to my Ultrabook (6 GB) while travelling I recently
faced some kind of borderline condition with splitter. On the
first run it throws "OutOfMemoryError: Java heap space", on
closely subsequent runs without any modifications it does not.
Repeating the task after some delay fails again. I guess, there
might be some self-optimization involved for this.</p>
<blockquote>
<p><i>fail:<br>
...<br>
40.000.000 ways parsed... id=888262666<br>
Number of stored tile combinations in multiTileDictionary:
4.525<br>
41.000.000 ways parsed... id=929920953<br>
Exception in thread "main" java.lang.OutOfMemoryError: Java
heap space<br>
at
uk.me.parabola.splitter.tools.SparseLong2IntMap$ChunkMem.<init>(SparseLong2IntMap.java:189)<br>
at
uk.me.parabola.splitter.tools.SparseLong2IntMap.saveCurrentChunk(SparseLong2IntMap.java:627)<br>
at
uk.me.parabola.splitter.tools.SparseLong2IntMap.replaceCurrentChunk(SparseLong2IntMap.java:886)<br>
at
uk.me.parabola.splitter.tools.SparseLong2IntMap.put(SparseLong2IntMap.java:691)<br>
at
uk.me.parabola.splitter.SplitProcessor.processWay(SplitProcessor.java:149)<br>
at
uk.me.parabola.splitter.AbstractMapProcessor.consume(AbstractMapProcessor.java:84)<br>
at
uk.me.parabola.splitter.OSMFileHandler.execute(OSMFileHandler.java:157)<br>
at uk.me.parabola.splitter.Main.writeTiles(Main.java:542)<br>
at uk.me.parabola.splitter.Main.start(Main.java:132)<br>
at uk.me.parabola.splitter.Main.main(Main.java:81)<br>
Elapsed time: 8m 0s Memory: Current 1466MB (1339MB used,
127MB free) Max 1466MB</i></p>
<p><i>success:<br>
...<br>
48.000.000 ways parsed... id=1262369277<br>
Writing relations Tue Mar 19 10:50:36 CET 2024<br>
100.000 relations parsed... id=1783690<br>
200.000 relations parsed... id=4148045<br>
300.000 relations parsed... id=7895430<br>
400.000 relations parsed... id=11681672<br>
500.000 relations parsed... id=15581604<br>
coord Map: 312.851.126 stored long/int pairs require ca. 3
bytes per pair. 14.225.657 chunks are used, the avg. number of
values in one 64-chunk is 21.<br>
coord Map details: ~852 MB, including 88 array(s) with 8 MB<br>
<br>
way Map: 48.015.926 stored long/int pairs require ca. 3 bytes
per pair. 3.974.651 chunks are used, the avg. number of values
in one 64-chunk is 12.<br>
way Map details: ~123 MB, including 10 array(s) with 8 MB<br>
<br>
JVM Memory Info: Current 1466MB (1357MB used, 109MB free)
Max 1466MB<br>
Full Node tests: 62.230.523<br>
Quick Node tests: 282.354.912<br>
Thread worker-2 has finished<br>
...<br>
</i></p>
</blockquote>
<p>My main machine has 24 GB of main memory, and runs troublefree
using the following memory allocation on the same task:</p>
<blockquote>
<p><i>JVM Memory Info: Current 3342MB (2378MB used, 964MB free)
Max 6000MB</i><br>
</p>
</blockquote>
<p>Splitter 653 so far is involved without explicit memory
allocation (java -jar .../splitter-latest/splitter.jar ...), using<br>
</p>
<blockquote>
<p><i>java --version<br>
openjdk 11.0.22 2024-01-16<br>
OpenJDK Runtime Environment (build
11.0.22+7-post-Debian-1deb10u1)<br>
OpenJDK 64-Bit Server VM (build
11.0.22+7-post-Debian-1deb10u1, mixed mode, sharing)</i></p>
</blockquote>
<p>Following up on splitter tuning hints (areas.list gets generated
in each case) I reduced --max-areas= from 4096 to 2048 to 1024,
but to no avail (not even significantly on the runtimes), once I
figured out the effect above. It fails on all first runs and
succeeds on all shortly following next ones.<br>
</p>
<p>Unfortunately its not possible to increase main hardware memory
on the small machine, but system tools report only about 2...3 GB
being used anyway. <br>
Is it possible to tweak Java to overcome the problem without
hurting the maps, preferably by machine, to be able to run
identical scripts? Some pointers would be appreciated, also on how
to monitor the Java memory situation.</p>
<p>Thanks, Felix<br>
</p>
<p><br>
</p>
<p><br>
</p>
</body>
</html>