[mkgmap-dev] Exception in thread "main" java.lang.IllegalArgumentException . on Europe extract

Thu Dec 2 12:45:30 GMT 2021

Hi Felix,

I also wondered if the strings are case-sensitive, that's how I found out that your options don't work ;)

The match is done ignoring the case. I'll add this to the dok.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Felix Hartmann <extremecarver at gmail.com>
Gesendet: Donnerstag, 2. Dezember 2021 13:27
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Exception in thread "main" java.lang.IllegalArgumentException . on Europe extract

Oh yes, thanks you were right. Somehow I used semicolon instead of comma. It is still breaking however - difference being the temp file is 2.09GB instead of 2.33GB.
BTW is the mdr7-del case insensitive if used together with lower-case? I hope it is.

I cannot download the display tool - the link from here is not working:
https://files.mkgmap.org.uk/detail/472

(same for the older link).

On Thu, 2 Dec 2021 at 11:53, Gerd Petermann <gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>> wrote:
Hi Felix,

I've noticed that your arguments in --mdr7-del are not comma separated. They probably have no effect.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von Gerd Petermann <gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>
Gesendet: Mittwoch, 1. Dezember 2021 17:59
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Exception in thread "main" java.lang.IllegalArgumentException . on Europe extract

Hi Felix,

both --mdr7-del and --mdr7-excl are only evaluated when the global index is created. They have no effect on the tiles.
You can use MdrDisplay with a smaller sample map to check what is written to the index.

Garmin uses a compressed format for the string table (Mdr 15) but we don't know how the compression works.
I guess we could add code to omit the string table. It's probably not mandantory. On the other hand I see no need for a detailed map
of whole Europe anyway.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com>>
Gesendet: Mittwoch, 1. Dezember 2021 17:44
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Exception in thread "main" java.lang.IllegalArgumentException . on Europe extract

yes it is some sort of overflow - if I do the map in two runs (1500 tiles approx)
with 65500*
and 65501*
it compiles fine.
That's why I use a pretty long exclusion list - but this is not enough. The index without split-name-index is about half the size. I think the first list of options only matters on the actual mkgmap creation run - and  does not change anything anymore on the second step of creating the address index. Maybe I missed a couple - but most should be excluded.

--mdr7-del=pri;sec;ter;cw;min;unsf,uncl;living;pdstrn;trk;pth;ft;fp;brdlw;rd;byw;ser;cw;mrt;crt;Bklane;Bktrk;opp;Opptrk;g0;g1;g2;g3;g4;g5;g6;t1;t2;t3;t4;t5;t6;xbk;cr;mr;hr;mr0;mr1;mr2;mr3;mr4;mr5;mr0.;mr1.;mr2.;mr3.;mr4.;mr5.;mr6.;mr00;mr01;mr02;mr03;mr04;mr05;mr06;mr11;mr12;mr13;mr14;mr15;mr16;mr20;mr21;mr22;mr23;mr24;mr25;mr26;mr30;mr31;mr32;mr33;mr34;mr35;mr35;mr40;mr41;mr41;mr43;mr44;mr45;mr46;mr50;mr51;mr52;mr53;mr54;mr55;mr56;mr60;mr61;mr62;mr63;mr64;mr65;mr66;m0;m1;m2;m3;m4;m5;m0.;m1.;m2.;m3.;m4.;m5.;m6.;m00;m01;m02;m03;m04;m05;m06;m11;m12;m13;m14;m15;m16;m20;m21;m22;m23;m24;m25;m26;m30;m31;m32;m33;m34;m35;m35;m40;m41;m41;m43;m44;m45;m46;m50;m51;m52;m53;m54;m55;m56;m60;m61;m62;m63;m64;m65;m66;mr.0;mr.1;mr.2;mr.3;mr.4;mr.5;mr.6;m.0;m.1;m.2;m.3;m.4;m.5;m.6;m1.;m2.;m3.;m4.;m5.;m6.;lmr;mrx0;mrx1;mrx2;mrx3;mrx4;mrx5;mrx6;mx0;mx1;mx2;mx3;mx4;mx5;mx6;bridge;swing;aqueduct;viaduct;"ntnl bndry";admin_lv=4;"ntnl park";weir;dam;waterfall;rapids;stream;river;drain;ditch;"huge river";"hugest river";"medium river";"small river";"wide river";"hugest canal";"huge canal";"medium canal";"small canal";"big canal";canal;snowpark;Skitour-descent;Skipiste;Skislope;illuminated;Nordic-Skiing;skitour;platter;ropetow;tbar;chairlift;goods_lift;runway;taxiway;apron;VFS0;VFS1;VFS2;VFS3;VFS4;VFS5;VFS6;VFS7;usf;cstrn;steps;parking_aisle;res;cob;cobstn;public_footway;public_footpath --mdr7-excl=pri;sec;ter;cw;min;unsf,uncl;pdstrn;trk;pth;ft;fp;brdlw;rd;byw;ser;cw;mrt;crt;Bklane;Bktrk;opp;Opptrk;g0;g1;g2;g3;g4;g5;g6;t1;t2;t3;t4;t5;t6;xbk;cr;mr;hr;mr0;mr1;mr2;mr3;mr4;mr5;mr0.;mr1.;mr2.;mr3.;mr4.;mr5.;mr6.;mr00;mr01;mr02;mr03;mr04;mr05;mr06;mr11;mr12;mr13;mr14;mr15;mr16;mr20;mr21;mr22;mr23;mr24;mr25;mr26;mr30;mr31;mr32;mr33;mr34;mr35;mr35;mr40;mr41;mr41;mr43;mr44;mr45;mr46;mr50;mr51;mr52;mr53;mr54;mr55;mr56;mr60;mr61;mr62;mr63;mr64;mr65;mr66;m0;m1;m2;m3;m4;m5;m0.;m1.;m2.;m3.;m4.;m5.;m6.;m00;m01;m02;m03;m04;m05;m06;m11;m12;m13;m14;m15;m16;m20;m21;m22;m23;m24;m25;m26;m30;m31;m32;m33;m34;m35;m35;m40;m41;m41;m43;m44;m45;m46;m50;m51;m52;m53;m54;m55;m56;m60;m61;m62;m63;m64;m65;m66;mr.0;mr.1;mr.2;mr.3;mr.4;mr.5;mr.6;m.0;m.1;m.2;m.3;m.4;m.5;m.6;m1.;m2.;m3.;m4.;m5.;m6.;lmr;mrx0;mrx1;mrx2;mrx3;mrx4;mrx5;mrx6;mx0;mx1;mx2;mx3;mx4;mx5;mx6;"ntnl bndry"

On Wed, 1 Dec 2021 at 17:39, Gerd Petermann <gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>> wrote:
Hi Felix,

your index is probably simply too large (> 2GB).
The attached patch fixes another possible overflow, but I am not sure if such a file size is allowed for the index.

Maybe try to remove specific items from the index (see --mdr7-excl, --mdr7-del)
I think the reason for the huge size is your style which combines a lot of information in the road name, e.g. "102 Thw-Göttinger Weg T2 Pth Zentralalpenweg 02 Hr"
or "137 Mindener Jubiläumsweg T2 Pth Mindener Jubiläumsweg Hr"
(found these in omtb_austria_13.05.2021)
All this information is stored in the PC index (a string table), and with --split-name-index lots of entries are produced for strings like "Pth" or "T2" or "Hr"

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>>> im Auftrag von Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>
Gesendet: Mittwoch, 1. Dezember 2021 17:16
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Exception in thread "main" java.lang.IllegalArgumentException . on Europe extract

I found out that if I remove --split-name-index then it compiles fine. I'm just trying a minimal setup option wise and could look if it's a special .img tile or not.
I could upload all map tiles somewhere for further analysis - or try some more things before doing that (12GB data or so).

On Wed, 1 Dec 2021 at 13:08, Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>> wrote:
Hi Gerd,

here is the log - hope you can find out something with it - but I think it is missing the problem - there are only 3 lines before the crash - all else is many minutes before.

2021/12/01 12:55:08 INFO (ImgFS): bs=512, whole size=-1789952000, hb=-14501, fb=-3481499, blocks=-3496000

2021/12/01 12:55:08 INFO (ImgFS): bs=1024, whole size=-1786235904, hb=-3623, fb=-1740748, blocks=-1744371

2021/12/01 12:55:08 INFO (ImgFS): Best block size: 512 sizeInBlocks=-3496000, reserved=-14501

On Wed, 1 Dec 2021 at 08:57, Gerd Petermann <gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>>> wrote:
Hi Felix,

hm, in that case one of the two factors is already negative.
You can enable logging and run the combiner step again (no need to compile the tiles again)
Add this to the log config:
uk.me.parabola.imgfmt.sys.ImgFS.level=FINE

BTW: The index contains not only address search data, also POI.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>>>> im Auftrag von Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>>
Gesendet: Mittwoch, 1. Dezember 2021 02:27
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Exception in thread "main" java.lang.IllegalArgumentException . on Europe extract

Hi Gerd,

thanks for the quick response. Any other way I can try to get more debug information?

I tried with the patch but I still get:
gmapi-minimal: Skipping file E:\OpenMTBMap\contourlines20\europe\75500448.img
Exception in thread "main" java.lang.IllegalArgumentException
        at java.base/sun.nio.ch<http://sun.nio.ch><http://sun.nio.ch><http://sun.nio.ch>.FileChannelImpl.position(FileChannelImpl.java:355)
        at uk.me.parabola.imgfmt.sys.ImgFS.sync(ImgFS.java:249)
        at uk.me.parabola.imgfmt.sys.ImgFS.close(ImgFS.java:325)
        at uk.me.parabola.mkgmap.combiners.MdrBuilder.onFinish(MdrBuilder.java:334)
        at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:690)
        at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126)
        at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:147)
        at uk.me.parabola.mkgmap.main.Main.main(Main.java:118)

On Tue, 30 Nov 2021 at 14:48, Gerd Petermann <gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>>>> wrote:
Hi Felix,

might be an overflow. The code line is
file.position(param.getReservedDirectoryBlocks() * param.getBlockSize());

so two ints are multiplied.

Please try if the attached patch changes something.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>>><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>>>>> im Auftrag von Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>>>
Gesendet: Dienstag, 30. November 2021 13:23
An: Development list for mkgmap
Betreff: [mkgmap-dev] Exception in thread "main" java.lang.IllegalArgumentException . on Europe extract

Using mkgmap version 4808 - when compiling Europe. It went through fine on all geofabrik country extracts.
I then upgraded to mkgmap 4819 and yesterday downloaded a new europe extract (4 days newer) from geofabrik to see if I can reproduce it again - and the error message is identical.
I'm going to run with default style now again - to see if it makes a difference.

C:\openmtbmap\mkgmap.jar --max-jobs=11 --order-by-decreasing-area  "--generate-sea" --code-page=65001 "--precomp-sea=E:\OpenMTBMap\osmpbf_geofabrik\sea.zip" "--style-file=C:\openmtbmap\openmtbmap_style" --add-boundary-nodes-at-admin-boundaries --fix-roundabout-direction --improve-overview --drive-on=detect --allow-reverse-merge --lower-case --nsis --index --levels="0:24, 1:23, 2:22, 3:21, 4:20, 5:19, 6:18" --overview-levels="7:17, 8:16, 9:15, 10:14, 11:13, 12:12" --add-pois-to-areas --pois-to-areas-placement=entrance=main;entrance=yes;building=entrance;barrier=entrance --simplify-lines=23:2.6,22:4.2,21:5.4,20:6,19:7,18:7.5,17:4.5,16:5,15:5,14:6  --simplify-polygons=23:3.6,22:7,21:6,20:9,17:5.4 --add-boundary-nodes-at-admin-boundaries=2 --cycle-map --ignore-fixme-values --housenumbers --road-name-config=C:\openmtbmap\openmtbmap_svn\roadNameConfig.txt --split-name-index --link-pois-to-ways --ignore-turn-restrictions --polygon-size-limits="24:12, 23:14, 22:14, 21:20, 20:20, 19:20, 18:20
 , 17:20, 16:20, 15:20, 14:20, 13:25" --description=omtb_eu --show-profiles=1  --location-autofill=bounds,is_in,nearest --bounds=E:\OpenMTBMap\osmpbf_geofabrik\bounds.zip --route --country-abbr=eu --country-name=europe --mapname=65500000 --family-id=6550 --product-id=1 --series-name=omtb_europe_29.11.2021_UC_LOCAL --family-name=mtb_eu_29.11.2021_UC_LOCAL --tdbfile --x-gmapi-minimal --overview-mapname=mapsetc --keep-going --area-name="europe_29.11.2021_UC_LOCAL_omtb" -c D:\openmtbmap\maps\template.europe E:\OpenMTBMap\contourlines20\europe\7*.img typeu.TYP  1>NUL

Number of MapFailedExceptions: 0
gmapi-minimal: Writing freshly compiled file C:\openmtbmap\maps\.\65500000.img
gmapi-minimal: Writing freshly compiled file C:\openmtbmap\maps\.\65500001.img
gmapi-minimal: Writing freshly compiled file C:\openmtbmap\maps\.\65500002.img
gmapi-minimal: Writing freshly compiled file C:\openmtbmap\maps\.\65500003.img
gmapi-minimal: Writing freshly compiled file C:\openmtbmap\maps\.\65500004.img
................
skipped
.....................
long list of contourline files:
.........
gmapi-minimal: Skipping file E:\OpenMTBMap\contourlines20\europe\75500447.img
WARNING (global): Input file E:\OpenMTBMap\contourlines20\europe\75500448.img has different code page 1252
WARNING (global): Input file E:\OpenMTBMap\contourlines20\europe\75500448.img has different charset type 10
WARNING (global): Input files have different code pages
gmapi-minimal: Skipping file E:\OpenMTBMap\contourlines20\europe\75500448.img
.... 448 is the highest number contourlines file

............And here comes the error
it happens on writing the address index mdr file - mapset.img and mapset.tdb have already been created.

Exception in thread "main" java.lang.IllegalArgumentException
        at java.base/sun.nio.ch<http://sun.nio.ch><http://sun.nio.ch><http://sun.nio.ch><http://sun.nio.ch>.FileChannelImpl.position(FileChannelImpl.java:355)
        at uk.me.parabola.imgfmt.sys.ImgFS.sync(ImgFS.java:249)
        at uk.me.parabola.imgfmt.sys.ImgFS.close(ImgFS.java:325)
        at uk.me.parabola.mkgmap.combiners.MdrBuilder.onFinish(MdrBuilder.java:334)
        at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:690)
        at uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126)
        at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:147)
        at uk.me.parabola.mkgmap.main.Main.main(Main.java:118)

--
Felix Hartman - Openmtbmap.org & VeloMap.org

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>>>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
Felix Hartman - Openmtbmap.org & VeloMap.org

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
Felix Hartman - Openmtbmap.org & VeloMap.org

--
Felix Hartman - Openmtbmap.org & VeloMap.org

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
Felix Hartman - Openmtbmap.org & VeloMap.org

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
Felix Hartman - Openmtbmap.org & VeloMap.org