[mkgmap-dev] MapFailedException when generating map for Italy

Wed Aug 16 21:29:51 BST 2017

Hi Gerd,

I think I should have looked at the logfiles earlier,
but since I do not really understand the splitter log, I tend not to read it 
carefully.

I found out, that the problem was gone even with the old parameter for 
max-nodes.
The solution was just to splitt the area again.

I checked the original europe.o5m, it was correct,
I checked the old 50100031.o5m and it was wrong as you said.
I compared the 2 splitter logfiles that have generated the wrong and the 
correct version
(I have also copied them for your reference)
and I found the first importand difference at line 564

correct run:
dictionary contains now 1.000 entries
...
Number of stored area combis for nodes: 154.673.904
Number of stored area combis for ways: 16.877.672
Number of stored Integers for rels: 0
Number of stored combis in dictionary: 1.449
Number of detected problem ways: 64.970
Number of detected problem rels: 6.311
JVM Memory Info: Current 2558MB (1865MB used, 693MB free) Max 5461MB

wrong run:
dictionary contains now 1.000 entries
dictionary contains now 2.000 entries
...
dictionary contains now 34.000 entries
...
Number of stored area combis for nodes: 154.672.668
Number of stored area combis for ways: 11.580.442
Number of stored Integers for rels: 0
Number of stored combis in dictionary: 34.620
Number of detected problem ways: 608.980
Number of detected problem rels: 25.075
JVM Memory Info: Current 2555MB (1954MB used, 601MB free) Max 5461MB

Of course I don't know what this means.

Since I am executing all this via Scripts that I did not change between 
these 2 runs,
my only explanation is, that splitter (or my computer) did not had the best 
day last saturday.

Walter

-----Ursprüngliche Nachricht----- 
From: Gerd Petermann
Sent: Wednesday, August 16, 2017 10:11 AM
To: Development list for mkgmap
Subject: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

another possible way to verify the data in your europe.o5m:
extract a node that is corrupted in 50100031.o5m, e.g.
osmfilter europe.o5m --keep="@id=1965123604" > test.osm

When I do this with     50100031.o5m I see

        <node id="1965123604" lat="44.933073" lon="17.5296559" version="1"/>

while the correct data should look like this:
        <node id="1965123604" lat="37.6469658" lon="14.0636559" 
version="1"/>

Gerd
________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Gerd 
Petermann <GPetermann_muenchen at hotmail.com>
Gesendet: Mittwoch, 16. August 2017 08:12:18
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

The data in 50100031.o5m  looks very strange, maybe there is a special case 
in splitter.
So if you still have the same europe.o5m file, it would be great if you 
could try to reproduce the
problem. If you can, please send the splitter log and the areas.list.

Gerd
________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von 
Walter Schlögl <Walter.Schloegl-Resch at aon.at>
Gesendet: Dienstag, 15. August 2017 22:29:33
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Gerd,

I did not try to reproduce the corrupted file,
but I did not update the original italy.o5m file
that I used for splitting it with the lower maxnodes value.

My workflow has 4 steps.

1. update europe.o5m with daily diffs
2. cropping a country or region out of europe
3. splitter
4. mkgmap

Since I only repeated step 3 and 4,
I think that my splitter run has generated the corruption.
But since I remember also a corruption in step 1 some weeks ago,
this assumption could be wrong.

Walter

-----Ursprüngliche Nachricht-----
From: Gerd Petermann
Sent: Tuesday, August 15, 2017 7:19 PM
To: Development list for mkgmap
Subject: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

does that mean that you can still reproduce the corrupted file when using
the old maxnodes value?

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
Walter Schlögl <Walter.Schloegl-Resch at aon.at>
Gesendet: Dienstag, 15. August 2017 18:31:44
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Gerd,

thanks for this info.
This could happen by merging the daily diffs into europe.o5m
But since the working version was built out of the same file,
I could also happen during the splitting.

Very strange, but now I know what to look at if it happens again.

Walter

-----Ursprüngliche Nachricht-----
From: Gerd Petermann
Sent: Tuesday, August 15, 2017 3:07 PM
To: Development list for mkgmap
Subject: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

something is wrong with the data in 50100031.o5m
It contains a lot of very long lines which do not appear in OSM. I guess you
have merged different data sources
and they use similar OSM ids for different objects (typically this happens
with SRTM data). Please check your tool chain
and make sure that generated ids are much higher than those in real OSM
data. The node ids recently jumped over 5.000.000.000, see
https://osmstats.neis-one.org/?item=nodes

The tile in 50100025.o5m confirms that you have a tile that contains data
from italy and from Malta (which is a drive-on-left country).

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Gerd
Petermann <GPetermann_muenchen at hotmail.com>
Gesendet: Dienstag, 15. August 2017 14:01:15
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

I'll see if I can reproduce the crash. Reg. the remaining message:
I guess the tile contains also Malta?

Gerd
________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
Walter Schlögl <Walter.Schloegl-Resch at aon.at>
Gesendet: Dienstag, 15. August 2017 12:36:31
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Gerd,

many thanks for your hints, with this I could solve all issues.

I have changed the splitter parameter from --max-nodes=800000 to 600000
but I have not updated the osm data to see what happens.
The message "There is not enough room" is gone now as expected,
but also the NullPointerException is gone.

I have copied the tile anyway if you still see a need for analysis.

https://1drv.ms/f/s!AqiYxUeSl56qgnCes1X0pcV4NIQ9

The only message that I still get for Italy is this.
SCHWERWIEGEND (StyledConverter): 50100025.o5m: Attention: Tile contains both
drive-on-left (22469) and drive-on-right roads (25540)

I would expect such a message for other countries, but not for Italy.
If this message is harmless, I will just ignore it.

Walter

-----Ursprüngliche Nachricht-----
From: Gerd Petermann
Sent: Monday, August 14, 2017 10:02 AM
To: Development list for mkgmap
Subject: Re: [mkgmap-dev] MapFailedException when generating map for Italy

Hi Walter,

I think you have several problems here.
1) The message "There is not enough room in a single garmin map for all the
input data. The .osm file should be split into smaller pieces first."
means that you should use a smaller maxnodes value  in splitter (or change
the style to add fewer data)
2) in 50100031.o5m there seems to be a data error. If I got that right the
error was already fixed in the OSM database (way 195550142 was changed
recently),
so I probably cannot reproduce the problem. Maybe post a link to your input
file.
3) Finally we have the NullPointerException. I can add a check to avoid
that, but I'd like to find out why this happens.
It might be related to the problems in 2) , so again I'd like to have the
input file 50100031.o5m.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
Walter Schlögl <Walter.Schloegl-Resch at aon.at>
Gesendet: Montag, 14. August 2017 09:41:44
An: Development list for mkgmap
Betreff: [mkgmap-dev] MapFailedException when generating map for Italy

I am generating many maps without any error,
only the area of Italy is throwing an error which I cannot solve.

I hope, anybody is having an idea, what could be the solution.

The coordinates are: 4.4°O – 21.1°O and 35°N - 45°N

The error message:

SCHWERWIEGEND (MapFailedException): 50100004.o5m: (thrown in
BufferedImgFileWriter.ensureSize()) There is not enough room in a single
garmin map for all the input data. The .osm file should be split into
smaller pieces first.
SCHWERWIEGEND (MapFailedException): 50100009.o5m: (thrown in
BufferedImgFileWriter.ensureSize()) There is not enough room in a single
garmin map for all the input data. The .osm file should be split into
smaller pieces first.
SCHWERWIEGEND (MapFailedException): 50100017.o5m: (thrown in
BufferedImgFileWriter.ensureSize()) There is not enough room in a single
garmin map for all the input data. The .osm file should be split into
smaller pieces first.
SCHWERWIEGEND (MapFailedException): 50100018.o5m: (thrown in
BufferedImgFileWriter.ensureSize()) There is not enough room in a single
garmin map for all the input data. The .osm file should be split into
smaller pieces first.
SCHWERWIEGEND (MapFailedException): 50100021.o5m: (thrown in
BufferedImgFileWriter.ensureSize()) There is not enough room in a single
garmin map for all the input data. The .osm file should be split into
smaller pieces first.
SCHWERWIEGEND (MapFailedException): 50100023.o5m: (thrown in
BufferedImgFileWriter.ensureSize()) There is not enough room in a single
garmin map for all the input data. The .osm file should be split into
smaller pieces first.
SCHWERWIEGEND (StyledConverter): 50100025.o5m: Attention: Tile contains both
drive-on-left (16159) and drive-on-right roads (21405)
SCHWERWIEGEND (MapFailedException): 50100025.o5m: (thrown in
BufferedImgFileWriter.ensureSize()) There is not enough room in a single
garmin map for all the input data. The .osm file should be split into
smaller pieces first.
SCHWERWIEGEND (MapFailedException): 50100028.o5m: (thrown in
BufferedImgFileWriter.ensureSize()) There is not enough room in a single
garmin map for all the input data. The .osm file should be split into
smaller pieces first.
SCHWERWIEGEND (RouteArc): 50100031.o5m: Way SS116 Strada Statale 116
Randazzo - Capo d'Orlando (id 195550142) contains an arc whose length
(5717225 units) is too big to be encoded, using length 4194303
SCHWERWIEGEND (RouteArc): 50100031.o5m: Way SS116 Strada Statale 116
Randazzo - Capo d'Orlando (id 195550142) contains an arc whose length
(5717225 units) is too big to be encoded, using length 4194303
SCHWERWIEGEND (RouteArc): 50100031.o5m: Way SS116 Strada Statale 116
Randazzo - Capo d'Orlando (id 195550142) contains an arc whose length
(5717225 units) is too big to be encoded, using length 4194303
SCHWERWIEGEND (RouteArc): 50100031.o5m: Way SS116 Strada Statale 116
Randazzo - Capo d'Orlando (id 195550142) contains an arc whose length
(5717225 units) is too big to be encoded, using length 4194303
SCHWERWIEGEND (RouteArc): 50100031.o5m: Way SS116 Strada Statale 116
Randazzo - Capo d'Orlando (id 195550142) contains an arc whose length
(5327027 units) is too big to be encoded, using length 4194303
SCHWERWIEGEND (RouteArc): 50100031.o5m: Way SS116 Strada Statale 116
Randazzo - Capo d'Orlando (id 195550142) contains an arc whose length
(5327027 units) is too big to be encoded, using length 4194303
SCHWERWIEGEND (RouteArc): 50100031.o5m: Way SS116 Strada Statale 116
Randazzo - Capo d'Orlando (id 195550142) contains an arc whose length
(5327027 units) is too big to be encoded, using length 4194303
SCHWERWIEGEND (RouteArc): 50100031.o5m: Way SS116 Strada Statale 116
Randazzo - Capo d'Orlando (id 195550142) contains an arc whose length
(5327027 units) is too big to be encoded, using length 4194303
SCHWERWIEGEND (MapFailedException): 50100034.o5m: (thrown in
BufferedImgFileWriter.ensureSize()) There is not enough room in a single
garmin map for all the input data. The .osm file should be split into
smaller pieces first.
java.lang.NullPointerException
    at
uk.me.parabola.imgfmt.app.Coord.distanceInDegreesSquared(Coord.java:425)
    at uk.me.parabola.imgfmt.app.Coord.distance(Coord.java:409)
    at
uk.me.parabola.mkgmap.osmstyle.housenumber.HousenumberGroup.findSegment(HousenumberGroup.java:151)
    at
uk.me.parabola.mkgmap.osmstyle.housenumber.HousenumberRoad.detectGroups(HousenumberRoad.java:205)
    at
uk.me.parabola.mkgmap.osmstyle.housenumber.HousenumberRoad.buildIntervals(HousenumberRoad.java:104)
    at
uk.me.parabola.mkgmap.osmstyle.housenumber.HousenumberGenerator.generate(HousenumberGenerator.java:706)
    at
uk.me.parabola.mkgmap.osmstyle.StyledConverter.end(StyledConverter.java:624)
    at
uk.me.parabola.mkgmap.reader.osm.ElementSaver.convert(ElementSaver.java:246)
    at
uk.me.parabola.mkgmap.reader.osm.o5m.O5mBinMapDataSource.load(O5mBinMapDataSource.java:53)
    at
uk.me.parabola.mkgmap.reader.osm.OsmMapDataSource.load(OsmMapDataSource.java:138)
    at uk.me.parabola.mkgmap.main.MapMaker.loadFromFile(MapMaker.java:154)
    at uk.me.parabola.mkgmap.main.MapMaker.makeMap(MapMaker.java:52)
    at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:265)
    at uk.me.parabola.mkgmap.main.Main$1.call(Main.java:261)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Exiting - if you want to carry on regardless, use the --keep-going option

Walter Schlögl
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev 