[mkgmap-dev] Tiles pruned in DEM map

Sun Dec 27 18:07:27 GMT 2020

I had removed --order-by-decreasing-area from my command for DEM maps, 
but --no-order-by-decreasing-area seems to fix the problem by now. By 
the way, this option is not documented in options file and I can't find 
it in the code. Is --no a general switch for all options or is it a 
particular case for order-by-decreasing-area? I don't remember having 
seen it before.

El 27/12/20 a las 10:36, Gerd Petermann escribió:
> Hi Carlos,
>
> I agree that --order-by-decreasing-area is to blame. This option seems to disturb the calculation of the overview map, and maybe the extreme size of the tile plays another role here.
>
> I hope Ticker can help here, I think the option supresses the merging of shapes, and this is probably not a good idea for the 0x4a shapes.
>
> As a work-around for you:
> Add option --no-order-by-decreasing-area at the end(!) of the parameter list so that the overview map is created without this option. This seems to fix the problem on my machine.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen at hotmail.com>
> Gesendet: Sonntag, 27. Dezember 2020 09:36
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> Hi Carlos,
>
> reg. the routing data: You can run display tool to check yourself, but the error is either in mkgmap or in display tool. My command looks like this :
> java -ea -cp d:\display\dist\display.jar;d:\mkgmap\dist\mkgmap.jar test.check.NodCheck  -vv --tab-zero=0 51145001.img > nodcheck.51145001.img 2> crash_nod
> It would be easier if you would post a link to your style.
>
> I am now able to reproduce the display error using this command
> java -Xmx6G -ea -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3 --dem-poly=e:\carlos\australia.poly  --overview-dem-dist=276160 --gmapi --show-profiles=1 --bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip --route --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area --location-autofill=is_in,nearest --drive-on=detect,left --road-name-config=d:\mkgmap\resources\roadNameConfig.txt  e:\carlos\51145001.o5m
>
> and the overview map has the same amount of 0x4a and 0x4b polygons as yours from 11-error.zip.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Carlos Dávila <carlos at alternativaslibres.org>
> Gesendet: Samstag, 26. Dezember 2020 19:12
> An: mkgmap-dev at lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>
> I'm sorry for the late response and for breaking the thread, but I
> didn't receive Gerd's last message, so I've had to copy it from mailing
> list archive. Reply inline.
>
> OK, I think I understand what problem you see. I used JaVaWa
> MapConverter to install the map in 11-error.zip and 12-OK.zip played
> with it. With 11-error.zip only the data from the overview map is
> displayed when I zoom to e.g. S21.71088 E114.92708, while 12-OK.zip
> shows more details (this is with MapSource, in Basecamp I see details in
> both maps).
>
> Yes, everything to the North of approx. S32.39125 is missing.
>
> I tried to analyse your files and I see three suspicious things so far:
>
> 1) The routing data seems to be wrong, NodCheck reports "Could not find
> node for road 38a61 nod2=124c8 " for both tiles. I don't see such an
> error with the default style.
>
> How can I check that? If both tiles are affected, probably this is not
> related with current issue, but other maps produced with the same style
> will also be wrong regarding routing.
>
> 2) The bad overview map contains a lot more 0x4a polygons. This is
> probably caused by the --order-by-decreasing-area, and I am not sure why
> this happens.
>
> Do you think the problem is in the overview map or may be in tile map?
> If I open tile in MapEdit++ the number of non polygon elements is almost
> the same in wrong and correct maps. For example, number of roads is
> exactly the same. It seems as if something is masking part of the tile
> in MapSource (also in BaseCamp in my case); elements are there, but you
> can't see them.
>
> 3) You use a special version of mkgmap, so please try also with the
> latest version.
>
> With latest mkgmap and default style problem persists.
>
> My first guess was that the bad NOD file may cause this but the problem
> doesn't disappear when I remove the file 51145001.NOD, so this is
> probably not to blame. Same for the 51145001.DEM file.
>
> I tried to reproduce the possible problem with the 0x4a tiles using the
> default style with this command java -Xmx4G -jar map.jar --route --gmapi
> --bounds=f:\osm\bounds.zip --precomp-sea=f:\osm\sea.zip
> --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
> --order-by-decreasing-area --location-autofill=is_in,nearest
> f:\dwnload\temp\51145001.o5m but my overview map contains the same
> number of 0x4a tiles as your good map.
>
> I cannot reproduce your DEM data because I don't know the polygon file
> (polygons/$pais.poly). Maybe I can reproduce the 0x4a problem when you
> create the test files again without --remove-ovm-work-files=true.
>
> You can download polygon file from here
> <https://alternativaslibres.org/tmp/australia.poly> and ovm from here
> <https://alternativaslibres.org/tmp/ovm_51145001.img>.
>
> El 22/12/20 a las 18:05, Carlos Dávila escribió:
>> I'm sorry, probably I didn't explain well enough.
>>
>> Overview is always correct, the problem affects only tiles. As you saw
>> in the screenshots of my first mail, they are different in size, but
>> they are generated from the same input files, so they should have the
>> same size. If you zoom in to an area that should be included in a
>> tile, only overview map is shown, no detailed map. Even more, when you
>> zoom in, at the given point where detailed map appears, tile boundary
>> jumps to the correct size, but nothing but overview is displayed
>> outside the "pruned" tile.
>>
>> You can download correct and wrong files from the links below and play
>> with them to get a better idea of the problem. They correspond to
>> first tile of Australia as shown in my screenshots.
>>
>> https://alternativaslibres.org/tmp/11-error.zip
>>
>> https://alternativaslibres.org/tmp/12-OK.zip
>>
>> Error was generated with java -Xmx27G -ea
>> -Dlog.config=logging.properties -jar mkgmap.jar
>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
>> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
>> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
>> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
>> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
>> --polygon-size-limits="24:12, 18:10, 16:0" --order-by-decreasing-area
>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>
>> OK was generated with java -Xmx27G -ea -Dlog.config=logging.properties
>> -jar mkgmap.jar
>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>> --dem-poly=polygons/$pais.poly --overview-dem-dist=276160 -c
>> opciones_comunes $CODEPAGE --gmapi --show-profiles=1
>> --bounds=bounds.zip --precomp-sea=sea.zip --route --family-name="OSM
>> $MAPA DEM" --family-id=5${FID} --overview-mapname=${ABR}5${FID}
>> --overview-mapnumber=${GRUPO}5${FID}000 --reduce-point-density=4
>> --polygon-size-limits="24:12, 18:10, 16:0"
>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON $LANGUAGE
>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>
>> Although removing --overview-dem-dist solved the issue in my first
>> test (see command below, correct output), after a lot of tests
>> removing options one by one from the original command, finally it
>> seems the problem is caused by option --order-by-decreasing-area (or
>> the combination of both).
>>
>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>> --dem=hgt/LIDAR-Sonny,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>> --dem-poly=polygons/$pais.poly -c opciones_comunes $CODEPAGE --gmapi
>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>> --overview-mapname=${ABR}5${FID}
>> --overview-mapnumber=${GRUPO}5${FID}000 --name-tag-list=$NAMETAG
>> --process-destination --process-exits --housenumbers
>> --reduce-point-density=4 --polygon-size-limits="24:12, 18:10, 16:0"
>> --order-by-decreasing-area --add-pois-to-areas --link-pois-to-ways
>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>> --check-roundabouts --check-roundabout-flares
>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>> --road-name-config=${CONFIG} --style=mio --remove-ovm-work-files=true
>> 51145001.o5m tmp/${ABR}5${FID}.txt
>>
>> El 22/12/20 a las 10:15, Gerd Petermann escribió:
>>> Hi Carlos,
>>>
>>> It seems I still don't understand what the problem is or how to
>>> reproduce it.
>>> I tried this command and the overview map looks OK:
>>> java -Xmx4G -jar d:\mkgmap\dist\mkgmap.jar --dem=f:\srtm3
>>> --overview-dem-dist=128000 --show-profiles=1 --gmapi
>>> f:\dwnload\temp\51145001.o5m
>>>
>>> If it is related to the DEM data I probably cannot help much. You may
>>> try a slightly different value for the --overview-dem-dist option or
>>> a modified polygon
>>> given with the --dem-poly option.
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
>>> von Carlos Dávila <carlos at alternativaslibres.org>
>>> Gesendet: Montag, 21. Dezember 2020 22:09
>>> An: mkgmap-dev at lists.mkgmap.org.uk
>>> Betreff: Re: [mkgmap-dev] Tiles pruned in DEM map
>>>
>>> The problem seems to be caused by overview DEM. If I remove
>>> --overview-dem-dist option tile is complete in MapSource. The issue can
>>> be reproduced with this tile
>>> <https://alternativaslibres.org/tmp/51145001.o5m> and HGT from
>>> vierfinderpanoramas3
>>>
>>> El 18/12/20 a las 11:48, Gerd Petermann escribió:
>>>> Hi Carlos,
>>>>
>>>> not sure if I understand what the problem is. The two screenshots
>>>> show completely different tile boundaries, so they were not created
>>>> from the same splitter output.
>>>>
>>>> Gerd
>>>>
>>>> ________________________________________
>>>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
>>>> von Carlos Dávila <cardavilam at gmail.com>
>>>> Gesendet: Donnerstag, 17. Dezember 2020 23:44
>>>> An: Development list for mkgmap
>>>> Betreff: [mkgmap-dev] Tiles pruned in DEM map
>>>>
>>>> I build several types of map (OSM, OSM+contour lines, maps for trucks
>>>> and OSM+DEM) for the same area, using same input files. I split
>>>> country.o5m with splitter and then use the same template.args output by
>>>> splitter for all maps, just changing mapname for the different types of
>>>> map. Given that, all resulting mapsets should have the same tiles, but
>>>> tiles in DEM map are smaller than in the other types. They share
>>>> part of
>>>> the boundaries (usually two of them) with other types, but other
>>>> boundaries are moved in, reducing displayed tile size. See attached
>>>> screenshots. However, file size is the same (discounting *.DEM) for
>>>> each
>>>> tile in *.gmap subfolders.
>>>>
>>>> Command is quite similar for OSM and OSM+DEM maps:
>>>>
>>>> java -Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar -c
>>>> opciones_comunes $CODEPAGE --gmapi --bounds=bounds.zip
>>>> --precomp-sea=sea.zip --route --country-name=$PAIS
>>>> --country-abbr=${ABR}
>>>> --area-name=$MAPA --family-name="OpenStreetMap $MAPA"
>>>> --family-id=1${FID} --product-version=$VERSION
>>>> --series-name="OSM-$MAPA"
>>>> --overview-mapname=${ABR}1${FID}
>>>> --overview-mapnumber=${GRUPO}1${FID}000
>>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>>> --report-similar-arcs --report-dead-ends=2 --link-pois-to-ways
>>>> --location-autofill=is_in,nearest --drive-on=detect,$DRIVEON
>>>> --check-roundabouts --check-roundabout-flares
>>>> --license-file=license_OSM
>>>> --copyright-file=copyright $LANGUAGE --road-name-config=${CONFIG}
>>>> --style=mio --check-styles -c $pais.args tmp/${ABR}1${FID}.txt
>>>>
>>>> java --Xmx27G -ea -Dlog.config=logging.properties -jar mkgmap.jar
>>>> --dem=hgt/LIDAR,hgt/ALOS,hgt/VIEW1,hgt/SRTM1,hgt/EU-DEM,hgt/VIEW3,hgt
>>>> --dem-poly=polygons/$pais.poly --dem-dists=3312,6624,9936,13248,26512
>>>> --overview-dem-dist=128000 -c opciones_comunes $CODEPAGE --gmapi
>>>> --show-profiles=1 --bounds=bounds.zip --precomp-sea=sea.zip --route
>>>> --country-name=$PAIS --country-abbr=${ABR} --area-name="$MAPA"
>>>> --family-name="OSM $MAPA DEM" --family-id=5${FID}
>>>> --product-version=$VERSION --series-name="OSM-$MAPA-DEM"
>>>> --overview-mapname=${ABR}5${FID}
>>>> --overview-mapnumber=${GRUPO}5${FID}000
>>>> --name-tag-list=$NAMETAG --process-destination --process-exits
>>>> --housenumbers --reduce-point-density=4 --polygon-size-limits="24:12,
>>>> 18:10, 16:0" --order-by-decreasing-area --add-pois-to-areas
>>>> --link-pois-to-ways --location-autofill=is_in,nearest
>>>> --drive-on=detect,$DRIVEON --check-roundabouts
>>>> --check-roundabout-flares
>>>> --license-file=license_OSM_${HGT} --copyright-file=copyright $LANGUAGE
>>>> --road-name-config=${CONFIG} --style=mio
>>>> --remove-ovm-work-files=true -c
>>>> ${pais}-DEM.args tmp/${ABR}5${FID}.txt
>>>>
>>>> Both OSM and OSM+DEM maps can be downloaded from
>>>> https://alternativaslibres.org/en/downloads.php#Oceania
>>>>
>>>> Any idea why this happens?
>>>>
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>> mkgmap-dev at lists.mkgmap.org.uk
>>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev at lists.mkgmap.org.uk
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev at lists.mkgmap.org.uk
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev