[mkgmap-dev] Error writing overviewmap - continue at failure

Sun Feb 28 10:11:23 GMT 2021

Hi

When reading my error log again there was another error. Too many cities... Could it have caused my overview map too fail?

SEVERE (MapBuilder): C:\Garmin\Temp\8004 - JBM - Planet\Output Splitter\80041630.osm.pbf: Map contains 94484 Cities - more than 65535 might cause indexing problems and excess size. Suggest splitter with lower --max-nodes

I Investigated the area (which is more or less a dessert in Jemen :-) 
Overpass does not count a lot place = city.   Could somebody help to explain?
Could this halfway-error also be a reason to fail the overview map being created at the end?

Kind regards
Joris

KML - file of the area

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2">
<Document>
  <Style id="transWhitePoly">
    <LineStyle>
      <width>1.5</width>
    </LineStyle>
    <PolyStyle>
      <color>00ffffff</color>
      <colorMode>normal</colorMode>
    </PolyStyle>
  </Style>

  <Placemark>
    <name>80041630</name>
    <styleUrl>#transWhitePoly</styleUrl>
      <description>
        <![CDATA[YE-Sanaa]]>
      </description>
    <Polygon>
      <outerBoundaryIs>
        <LinearRing>
          <coordinates>
            43.637695,13.666992
            43.637695,16.962891
            45.571289,16.962891
            45.571289,13.666992
            43.637695,13.666992
          </coordinates>
        </LinearRing>
      </outerBoundaryIs>
    </Polygon>
  </Placemark>

  </Document>
</kml>

Met vriendelijke groeten,

Joris Bo
jorisbo at hotmail.com

-----Oorspronkelijk bericht-----
Van: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> Namens Joris Bo
Verzonden: zaterdag 27 februari 2021 18:40
Aan: Development list for mkgmap <mkgmap-dev at lists.mkgmap.org.uk>
Onderwerp: Re: [mkgmap-dev] Error writing overviewmap - continue at failure

Hi Mike & Gerd ,

My europe map (Europe-latest.osm.pbf  split of 24Gb) which works fine, has these img- files

80010001.img to 80011554.img (1554 map files) Osmmap.img of 7Mb Osmmap_mdr.img of 760Mb

I assumed the 7Mb file is the overview map? Not the Osmmap_mdr.img file?
For planet.osm (56Gb in stead of 24Gb) would then make an overview map of lets say approximately 20Mb My planet ovm work files together are 1.2Gb 😊 this is as pointed out, not a reference So I assume the real problem is that the overview map is loaded with too many objects.

I will study Gerd's proposal first (for now I get the concept but don't understand the exact procedure) But with a good backup not much can go wrong.

Thank you for the solutions and the effort off course, Kind regards Joris

Met vriendelijke groeten,

Joris Bo
jorisbo at hotmail.com

-----Oorspronkelijk bericht-----
Van: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> Namens Gerd Petermann
Verzonden: zaterdag 27 februari 2021 17:58
Aan: Development list for mkgmap <mkgmap-dev at lists.mkgmap.org.uk>
Onderwerp: Re: [mkgmap-dev] Error writing overviewmap - continue at failure

Hi Mike and Joris,

as Ticker pointer out before the overall sum of file sizes for the ovm*.img files is typically much larger than the resultiing overview map. Reason is that the ovm* files contain the full LBL section (all Strings that appear in the detailed map), only a few of them are used in the overview map.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Mike Baggaley <mike at tvage.co.uk>
Gesendet: Samstag, 27. Februar 2021 17:38
An: 'Development list for mkgmap'
Betreff: Re: [mkgmap-dev] Error writing overviewmap - continue at failure

Hi Joris,

I will soon be passing an updated patch to Gerd to improve this, as I had also been experiencing similar problems. The overview map is not built until all the tiles have been created (assuming the error is not in an overview tile), and I don’t think it uses the style file at that point, just merges all the ovm data, so using a new style will not help without restarting your run. The message produced by the patch when the overview file is too big is “The overview map is too big. The maximum size is 16777215. Try reducing the highest overview resolution or reducing the amount of information included in the overview.” If you build your own version of mkgmap, you can use the version of the patch I posted a few days ago. For the whole world, I expect the maximum resolution of overview to be pretty low. If I build a single UK county, I can have an overview at resolution 22, but only 20 for the UK. I haven’t tried anything bigger. You may get some idea how close you are to the limit by totalling the sizes of the ovm files. The maximum size may not necessarily be the max for the whole file, it may be the limit of a sub file though. That is certainly the case for ordinary map tiles, but I’m not sure on the overview.

I also have some code that I’m working on that adds support for a mkgmap:nooverview tag in the style file, so that if you have overlapping main and overview resolutions, you can direct your style to omit some of the information from the overview. Will be passing this patch over shortly as well.

I don’t think allowing a continuation after a failure in the final overview map would be helpful, as there is very little left to do at that point. There are some issues with the keep going handling and I’m also looking at that.

Cheers,
Mike

From: Joris Bo [mailto:jorisbo at hotmail.com]
Sent: 27 February 2021 14:49
To: Development list for mkgmap <mkgmap-dev at lists.mkgmap.org.uk>
Subject: [mkgmap-dev] Error writing overviewmap - continue at failure

Hi,

I think the below error message is for writing the overview map.
Of which I would not be surprised, it was a test on splitting and building a full-planet.osm map. So… no worries.

But now I’m wondering if there is a way in which I could continue testing with the already generated 4.426 img-files and 4.426 ovm_work files to obtain a .tdb file, mdx files, nsis installer etc.
It took 21 hours for mkgmap sofar, so it would be nice if that time spent was not useless.
I used splitter with maxnodes 1.700.000 which took another 9 hours and did not give errors when building a single .img file

I would like to simplify the style-file run mkgmap again on the previous output and try the create an overview map only (So without building all the img files again) and this way iterate until I found the breakpoint.
1)     Would that make sense?
2)     Could I enable some more error/verbose logging to obtain info about sizes/counts or whatever indicators that would help to see if my changes help in my search to a smaller overviewmap.
3)     Would It make sense to not generate a MapFailedException but a warning only? So mkgmap would continue finishing the last millimeters to the finish (off course without a working overviewmap)

Kind regards
Joris

SEVERE (MapFailedException): (thrown in BufferedImgFileWriter.ensureSize()) There is not enough room in a single garmin map for all the input data. The .osm file should be split into smaller pieces first.
uk.me.parabola.imgfmt.MapFailedException: There is not enough room in a single garmin map for all the input data. The .osm file should be split into smaller pieces first.
            at uk.me.parabola.imgfmt.app.BufferedImgFileWriter.ensureSize(BufferedImgFileWriter.java:275)
            at uk.me.parabola.imgfmt.app.BufferedImgFileWriter.put(BufferedImgFileWriter.java:238)
            at uk.me.parabola.imgfmt.app.trergn.Polyline.write(Polyline.java:142)
            at uk.me.parabola.imgfmt.app.trergn.RGNFile.addMapObject(RGNFile.java:140)
            at uk.me.parabola.imgfmt.app.map.Map.addMapObject(Map.java:273)
            at uk.me.parabola.mkgmap.build.MapBuilder$ShapeAddFilter.doFilter(MapBuilder.java:1459)
            at uk.me.parabola.mkgmap.build.LayerFilterChain.doFilter(LayerFilterChain.java:58)
            at uk.me.parabola.mkgmap.filters.LinePreparerFilter.doFilter(LinePreparerFilter.java:143)
            at uk.me.parabola.mkgmap.build.LayerFilterChain.doFilter(LayerFilterChain.java:58)
            at uk.me.parabola.mkgmap.filters.RemoveEmpty.doFilter(RemoveEmpty.java:59)
            at uk.me.parabola.mkgmap.build.LayerFilterChain.doFilter(LayerFilterChain.java:58)
            at uk.me.parabola.mkgmap.filters.RemoveObsoletePointsFilter.doFilter(RemoveObsoletePointsFilter.java:142)
            at uk.me.parabola.mkgmap.build.LayerFilterChain.doFilter(LayerFilterChain.java:58)
            at uk.me.parabola.mkgmap.filters.PolygonSplitterFilter.doFilter(PolygonSplitterFilter.java:81)
            at uk.me.parabola.mkgmap.build.LayerFilterChain.doFilter(LayerFilterChain.java:58)
            at uk.me.parabola.mkgmap.build.LayerFilterChain.startFilter(LayerFilterChain.java:73)
            at uk.me.parabola.mkgmap.build.MapBuilder.processShapes(MapBuilder.java:1260)
            at uk.me.parabola.mkgmap.build.MapBuilder.makeSubdivision(MapBuilder.java:901)
            at uk.me.parabola.mkgmap.build.MapBuilder.makeMapAreas(MapBuilder.java:813)
            at uk.me.parabola.mkgmap.build.MapBuilder.makeMap(MapBuilder.java:312)
            at uk.me.parabola.mkgmap.combiners.OverviewBuilder.writeOverviewMap(OverviewBuilder.java:192)
            at uk.me.parabola.mkgmap.combiners.OverviewBuilder.onFinish(OverviewBuilder.java:105)
            at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:675)
            at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:125)
            at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:144)
            at uk.me.parabola.mkgmap.main.Main.main(Main.java:115)

Met vriendelijke groeten,

Joris Bo
jorisbo at hotmail.com<mailto:jorisbo at hotmail.com>

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev