[mkgmap-dev] r4809 crashes by buildung mdr

Wed Oct 27 13:19:23 BST 2021

Hi Arndt,

thanks, I can now reproduce the crash.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Arndt Röhrig <arndt at speichenkarte.de>
Gesendet: Dienstag, 26. Oktober 2021 21:02
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] r4809 crashes by buildung mdr

Hi Ticker, hi Gerd,

https://speichenkarte.de/Arndt_NRWplus_mkgmapstyle.zip

I have tested only Netherland with default style and mkgmap stops:

Mkgmap version 4809
Time started: Tue Oct 26 20:36:49 CEST 2021
Number of MapFailedExceptions: 0
Exception in thread "main" java.lang.AssertionError: mdr20 value changed f=256561 t=256593 count=3890
at uk.me.parabola.imgfmt.app.mdr.Mdr5Record.setMdr20(Mdr5Record.java:134)
at uk.me.parabola.imgfmt.app.mdr.Mdr20.buildFromStreets(Mdr20.java:84)
at uk.me.parabola.imgfmt.app.mdr.MDRFile.writeSections(MDRFile.java:335)
at uk.me.parabola.imgfmt.app.mdr.MDRFile.write(MDRFile.java:270)
at uk.me.parabola.mkgmap.combiners.MdrBuilder.onFinishForDevice(MdrBuilder.java:353)
at uk.me.parabola.mkgmap.combiners.GmapsuppBuilder.onFinish(GmapsuppBuilder.java:174)
at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:690)
at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126)
at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:147)
at uk.me.parabola.mkgmap.main.Main.main(Main.java:118)

I use bounds and sea from mkgmap (thkukuk)
geoname-citys500 from here:
http://download.geonames.org/export/dump/

options:
copyright-message=(C) speichenkarte.de
product-id=1
license-file=.\Daten\LicenseMkgmap.txt
draw-priority=28
latin1
lower-case
max-jobs
merge-lines
allow-reverse-merge
line-types-with-direction=0x1e00
keep-going
remove-ovm-work-files
preserve-element-order
drive-on=detect,right
link-pois-to-ways
add-pois-to-areas
add-pois-to-lines
route
#check-roundabouts
#check-roundabout-flares
poi-address
location-autofill=is_in,nearest
housenumbers
process-destination
process-exits
bounds=.\Daten\bounds.zip
precomp-sea=.\Daten\sea.zip
generate-sea: land-tag=natural=background
show-profiles=1
polygon-size-limits=22:8,20:6,18:5,17:4,16:3,15:2,14:1,13:1
simplify-lines:22:4,20:6,18:8,17:9,16:10,15:11,14:12,13:13
simplify-polygons:22:4,20:6,18:8,17:9,16:10,15:11,14:12,13:13
improve-overview
dem=.\phyghtmap\hgt\Sonny1,.\phyghtmap\hgt\VIEW1,.\phyghtmap\hgt\Sonny3,.\phyghtmap\hgt\VIEW3,.\phyghtmap\hgt\SRTM1,.\phyghtmap\hgt\SRTM3
overview-dem-dist=90000
tdbfile
index
split-name-index
gmapi
add-boundary-nodes-at-admin-boundaries=4
x-no-force-end-nodes-routing-nodes
#report-routing-islands
max-routing-island-len=700
gmapsupp

Bauen.bat:
@echo off
cd %~dp0
setlocal EnableDelayedExpansion
::-----------------------------------------------------------------------------------------------------------------------------
::-----------------------------------------------------------------------------------------------------------------------------
echo %date:~0%
set Datum=%date:~8,2%%date:~3,2%%date:~0,2%
::set Datum=201111
echo %Datum%
for /l %%m in (4800,1,9999) do (
if exist .\tools\mkgmap-r%%m set Pfad=.\tools\mkgmap-r%%m
)
if /I "%Pfad%" == "" (
echo Kein mkgmap!
echo Taste druecken zum schliessen
pause >nul
exit
)
set Pfad=.\tools\mkgmap-r4809
::-----------------------------------------------------------------------------------------------------------------------------
::-----------------------------------------------------------------------------------------------------------------------------
:: ---mit oder ohne splitter---
color 9b
:oben

set Abfr=1

:weiter
:oben2
echo 1 mit runterfahren
echo 2 ohne ohne runterfahren
echo 3 Exit
set /P runter=
if /i "%runter%" == "1" goto weiter2
if /i "%runter%" == "2" goto weiter2
if /i "%runter%" == "3" goto exit 0
goto oben2
:weiter2

::timeout /t 2400
::-----------------------------------------------------------------------------------------------------------------------------
::-----------------------------------------------------------------------------------------------------------------------------
set dlog=-Dlog.config=.\Daten\dlogkomplett.txt
::set dlog=-Dlog.config=.\Daten\dlog.txt
set xmx=-Xmx12G
set xmx=-Xmx3G
set zln=0
set Name[1]=Speiche_mkgmap_NRW
set FamID[1]=6004
set MapID[1]=60040000
set OSM[1]=Speiche_NRW.osm.pbf
set Name[1]=Speiche_mkgmap_Fabrik
set FamID[1]=6002
set MapID[1]=60020000
set OSM[1]=Speiche_Fabrik.osm.pbf

set Name[2]=Speiche_mkgmap_NRWplus
set FamID[2]=6005
set MapID[2]=60050000
set OSM[2]=Speiche_NRWplus.osm.pbf
set Name[2]=Speiche_mkgmap_NL
set FamID[2]=6006
set MapID[2]=60060000
set OSM[2]=Speiche_NL.osm.pbf
set Name[3]=Speiche_mkgmap_Belgien
set FamID[3]=6007
set MapID[3]=60070000
set OSM[3]=Speiche_Belgien.osm.pbf
goto anf
set Name[2]=Speiche_mkgmap_Kanaren
set FamID[2]=6001
set MapID[2]=60010000
set OSM[2]=Speiche_Kanaren.osm.pbf
goto anf
set Name[3]=Speiche_mkgmap_Deutschland
set FamID[3]=6003
set MapID[3]=60030000
set OSM[3]=Speiche_Deutschland.osm.pbf
::goto anf
set Name[4]=Speiche_mkgmap
set FamID[4]=6000
set MapID[4]=60000000
set OSM[4]=Speiche_Europa.osm.pbf

:anf
set /a "zln=%zln%+1"
if "!Name[%zln%]!" == "" goto schluss
echo _______________________________________________________
:: ---osmconvert---
::if /i "%Abfr%" == "1" (
::color 6f
::echo %time% - osmconvert !Name[%zln%]!
::if exist .\OSM\!Name[%zln%]!_mit_SRTM.osm.pbf del .\OSM\!Name[%zln%]!_mit_SRTM.osm.pbf
::".\Tools\osmconvert" .\SRTM\SRTM_!Name[%zln%]!.pbf -o=.\OSM\2.o5m
::".\Tools\osmconvert" .\OSM\!Name[%zln%]!.osm.pbf .\OSM\2.o5m -o=.\OSM\!Name[%zln%]!_mit_SRTM.osm.pbf
::if exist .\OSM\2.o5m del .\OSM\2.o5m
::)
::goto ohnesplitter
:: ---Splitter---
if /i "%Abfr%" == "1" (
color 5f
echo %time% - splitter !Name[%zln%]!
if not exist .\Baustelle\!Name[%zln%]!_Splitter mkdir .\Baustelle\!Name[%zln%]!_Splitter
if exist .\Baustelle\!Name[%zln%]!_Splitter\60* del.\Baustelle\!Name[%zln%]!_Splitter\60*
::"java" -jar %xmx% .\Tools\splitter\splitter.jar --mapid=!MapID[%zln%]! --precomp-sea=.\Daten\sea.zip --geonames-file=.\Daten\cities500.txt --route-rel-values --max-areas=8192 --keep-complete=true --write-kml=splitter.kml --output-dir=.\Baustelle\!Name[%zln%]!_Splitter .\OSM\map > nul
"java" -jar %xmx% .\Tools\splitter\splitter.jar --mapid=!MapID[%zln%]! --precomp-sea=.\Daten\sea.zip --geonames-file=.\Daten\cities500.txt --route-rel-values --max-areas=8192 --keep-complete=true --write-kml=splitter.kml --output-dir=.\Baustelle\!Name[%zln%]!_Splitter .\OSM\!OSM[%zln%]! > nul
color 5f
)
:ohnesplitter
:: ---mkgmap---
color 4f
echo %time% - mkgmap %Pfad% !Name[%zln%]!
if exist .\Baustelle\!Name[%zln%]!_mkgmap rd .\Baustelle\!Name[%zln%]!_mkgmap /s /q
mkdir .\Baustelle\!Name[%zln%]!_mkgmap

"java" -jar %xmx% %dlog% %Pfad%\mkgmap.jar --family-id=!FamID[%zln%]! --latin1 --output-dir=.\Baustelle\!Name[%zln%]!_mkgmap .\Daten\mapnik_ohneLinien.txt > nul
"java" -jar %xmx% -enableassertions %dlog% %Pfad%\mkgmap.jar --output-dir=.\Baustelle\!Name[%zln%]!_mkgmap --style-file=%Pfad%\examples\styles\default --family-id=!FamID[%zln%]! --region-name=!Name[%zln%]! --series-name=!Name[%zln%]!_%Datum% --family-name=!Name[%zln%]! --area-name=!Name[%zln%]! --overview-mapname=!Name[%zln%]! --road-name-config=%Pfad%\examples\roadNameConfig.txt --dem-poly=.\Baustelle\!Name[%zln%]!_Splitter\areas.poly -c .\Daten\mkgmap_options_default.args -c .\Baustelle\!Name[%zln%]!_Splitter\template.args --description=!Name[%zln%]!_%Datum% %Pfad%\examples\typ-files\mapnik.txt > nul

::-----------------------------------------------------------------------------------------------------------------------------
::-----------------------------------------------------------------------------------------------------------------------------
:loeschen
::rd .\Baustelle\!Name[%zln%]!_splitter /s /q

:gmapnachmaps
::goto nogmapnachmaps
color 0b
echo %time% - xcopy !Name[%zln%]!.gmap nach C:\ProgramData\GARMIN\Maps\
if exist C:\ProgramData\GARMIN\Maps\!Name[%zln%]!.gmap rd C:\ProgramData\GARMIN\Maps\!Name[%zln%]!.gmap /s /q
xcopy .\Baustelle\!Name[%zln%]!_mkgmap\!Name[%zln%]!.gmap C:\ProgramData\GARMIN\Maps\!Name[%zln%]!.gmap\ /E > NUL
xcopy .\Baustelle\!Name[%zln%]!_mkgmap\mapnik_ohneLinien.typ C:\ProgramData\GARMIN\Maps\!Name[%zln%]!.gmap\ > nul
xcopy .\Baustelle\!Name[%zln%]!_mkgmap\!Name[%zln%]!_gmapsupp.img C:\ProgramData\GARMIN\Maps\!Name[%zln%]!.gmap\ > nul
:nogmapnachmaps
:loeschen2
::rd .\Baustelle\!Name[%zln%]!_mkgmap /s /q
color ac
echo %time% - Fertig mit !Name[%zln%]!
:: ---7zip---
:zippen
::if "!Name[%zln%]!" == "Speiche_Fabrik" goto nozip
color 3e
echo %time% - zippe gmap
if exist .\HP_Upload\!Name[%zln%]!*.zip if exist .\HP_Upload\!Name[%zln%]!\!Name[%zln%]!* del .\HP_Upload\!Name[%zln%]!\!Name[%zln%]!*
"C:\Program Files\7-Zip\7z.exe" a -tzip ".\HP_Upload\!Name[%zln%]!_gmap_%Datum%" "C:\ProgramData\GARMIN\Maps\!Name[%zln%]!.gmap" > nul
rename C:\ProgramData\GARMIN\Maps\!Name[%zln%]!.gmap\mapnik.typ _mapnik.typ >nul
rename C:\ProgramData\GARMIN\Maps\!Name[%zln%]!.gmap\mapnik_ohneLinien.typ mapnik.typ >nul
:nozip
echo _______________________________________________________
goto anf
:schluss
echo ________________________________________________________
color ac
echo %time% - Alles fertig
if /i "%runter%" == "1" shutdown -s -f
echo Taste druecken zum schliessen
pause >nul

Thanks again for your efforts!

Greetz

Arndt

Ticker Berkin <rwb-mkgmap at jagit.co.uk<mailto:rwb-mkgmap at jagit.co.uk>> hat am 26.10.2021 17:03 geschrieben:

Hi Gerd & Arndt

I've managed to reproduce the problem using splitter with
--split-file=arndt/splitter.kml

and lots of bits from SpeicheFabrik_Steuerdateien

When I tried with:
--style-file=SpeicheFabrik_Steuerdateien/styles_base
I got some tiles exceeding the maximum RGN size, and, even when running
with max-jobs=1, I ran out of memory on other tiles.
However, with default style I can build all the tiles and get the
assertion error.

The suspect names are:

SEVERE (global): 05CollDiff 0 1 Aische-En-Refail != Aische-en-Refail
SEVERE (global): 05CollDiff 0 1 Corroy-Le-Château != Corroy-le-Château
SEVERE (global): 05CollDiff 0 1 De Wijk != de Wijk
SEVERE (global): 05CollDiff 0 1 Mielen-Boven-Aalst != Mielen-boven-
Aalst
SEVERE (global): 05CollDiff 0 1 Nieuwer Ter Aa != Nieuwer ter Aa

and I'll now try and work out why.

Ticker

On Tue, 2021-10-26 at 11:41 +0200, Arndt Röhrig wrote:
 Hi Ticker, hi Gerd,

 i don´t use contour lines by my default-style-maps.
 bounds & sea is from the mkgmap page (thkukuk)

tonight i can upload the img
 greetz

 Arndt

 Gerd Petermann <gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>> hat am 26.10.2021
11:21 geschrieben:

 Hi Ticker & Arndt,

 Arndt posted a link to his scripts, see
https://speichenkarte.de/Steuerdateien/SpeicheFabrik_Steuerdateien211017.zip
 See BAUENbat and the various mkgmap_options*.args file. I think the
script concatenates OSM data with srtm data before splitting so you
really need eiher the areas.list or the splitter.kml file from Arndt
to be able to reproduce. The content of the bounds file might as well
be important.

 I think it's nearly impossible to reproduce the problem with the
scripts. Best would be if Arndt could post a link to the zipped *.img
files.

 Gerd

 ________________________________________
 Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag
von Ticker Berkin <rwb-mkgmap at jagit.co.uk<mailto:rwb-mkgmap at jagit.co.uk>>
 Gesendet: Dienstag, 26. Oktober 2021 11:07
 An: Development list for mkgmap
 Betreff: Re: [mkgmap-dev] r4809 crashes by buildung mdr

 Hi Gerd & Arndt

 I've also failed to reproduce this.

 I suspect the problem is relating to some slivers of
regions/countries
 around the main map area containing maybe 0/1 city and so it could
be
 very sensitive to the exact contents of the map.

 I used the splitter on europe-latest.osm.pbf with
 will have a slightly different content to cutting with a different
tool
 (osmosis) @Arndt - how do you do this? and also the areas.list

 It also might be sensitive to options like housenumbers, sea
 generation, bounds, location-autofill, name-tag-list, add-pois-to-
area,
 etc. @Arndt - can I have you option list.

 @Gerd - Now I have a set of .img files and the MDR, I'll try as you
 suggest and see what is the structures are around these indices.

 Ticker

 On Tue, 2021-10-26 at 07:17 +0000, Gerd Petermann wrote:

 Hi Ticker,

 I also failed to reproduce the crash with an older download of
 Europe. I'll try to download a new one today.
 An alternative might be to add more logging data.
 The assert message says
 changed f=369297 t=369312 count=7092
 Wouldn't it be enough to know the details of the entries between
the
 from (f) and the to (t) position in the corresponding list to get
a
 hint about the involved strings (or map tiles)?

 Gerd

 ________________________________________
 Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>> im
Auftrag
 von Ticker Berkin <rwb-mkgmap at jagit.co.uk<mailto:rwb-mkgmap at jagit.co.uk>>
 Gesendet: Montag, 25. Oktober 2021 16:37
 An: Development list for mkgmap
 Betreff: Re: [mkgmap-dev] r4809 crashes by buildung mdr

 Hi Arndt

 I am having a lot of trouble with disk space and download time
trying
 to get EUROPE.osm to cut with your poly. I've tried some of the
 individual countries / regions but don't get the error. At the
moment
 I'm trying to run the spitter on all the areas with your
 Speiche_NRWplus.poly, however it would be more reliable if I used
 your
 areas.list instead - do you still have this, can you send it to
me.

 Thanks
 Ticker

 On Sun, 2021-10-24 at 18:53 +0200, Arndt Röhrig wrote:

 Hi ticker,

 i use latin1

 NRW+ is mapthat cut from europa.osm NRW, BeNeLux, NDS, RLP, HES.
 (poly attached)

 the style ist here: (Steuerdateien)
 https://speichenkarte.de/

 default style doesn´t work too:

 Mkgmap version 4809
 Time started: Sun Oct 24 18:27:21 CEST 2021
 Number of MapFailedExceptions: 0
 Exception in thread "main" java.lang.AssertionError: mdr20 value
 changed f=369297 t=369312 count=7092
 at
 34
 )
 at
 at
 5)
 at uk.me.parabola.imgfmt.app.mdr.MDRFile.write(MDRFile.java:270)
 at
 ld
 er.java:353)
 at
 il
 der.java:174)
 at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:690)
 at
 ja
 va:126)
 at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:147)
 at uk.me.parabola.mkgmap.main.Main.main(Main.java:118)

 Greetz

 Arndt

 Ticker Berkin <ticker at jagit.co.uk<mailto:ticker at jagit.co.uk>> hat am 24.10.2021 18:18
 geschrieben:
 >

 Hi Arndt

 What code-page are you using?
 Does this happen with the default style?
 Where do you mean by nrw & countrys around?

 I'll try and reproduce

 Ticker

 On Sun, 2021-10-24 at 17:59 +0200, Arndt Röhrig wrote:

 Hi @all,

 mkgmap r4809 crashes with an error message. the error
happens
 at the
 end when generating the mdx/mdr/gmapsupp.
 r4808 works fine.
 the map "nrw & countrys around" crashes, but germany, canary
 islands
 and german countrys works.

 greetz

 Arndt

 15:44:03,39 - mkgmap .\tools\mkgmap-r4809 Speiche_NRWplus
 changed f=1228974 t=1229552 count=20428
 at
 )
 at
 at
 at
 er.java:353)
 at
 der.java:174)
 at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:690)
 at
 va:126)
 at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:147)
 at uk.me.parabola.mkgmap.main.Main.main(Main.java:118)
 _______________________________________________
 mkgmap-dev mailing list
 mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
 https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
 >

 _______________________________________________
 mkgmap-dev mailing list
 mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
 https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
 _______________________________________________
 mkgmap-dev mailing list
 mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
 https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
 >

 _______________________________________________
 mkgmap-dev mailing list
 mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
 https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
 _______________________________________________
 mkgmap-dev mailing list
 mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
 https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

 _______________________________________________
 mkgmap-dev mailing list
 mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
 https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
 _______________________________________________
 mkgmap-dev mailing list
 mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
 https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev