[mkgmap-dev] Exception in thread "main" java.lang.IllegalArgumentException . on Europe extract

Wed Dec 1 17:12:40 GMT 2021

Oh yeah the best solution would be if mkgmap kinda could create 4 maps of
Europe from all the Europe .img tiles with polygons/boxes created
manually/then provided as input file..

Splitting before and creating those 4 maps from scratch takes a lot more of
compilation time than the full Europe map alone.

On Wed, 1 Dec 2021 at 18:07, Felix Hartmann <extremecarver at gmail.com> wrote:

> It's still crashing also with patch v2.
> The full map of Europe serves one need - it is simpler than say creating 5
> maps of Europe with an Overlap. But yes I will simply create the continent
> maps of Europe and Asia without split-name-index (Asia has not crashed yet
> - but is its growing quickly).
>
> On Wed, 1 Dec 2021 at 17:44, Felix Hartmann <extremecarver at gmail.com>
> wrote:
>
>> yes it is some sort of overflow - if I do the map in two runs (1500 tiles
>> approx)
>> with 65500*
>> and 65501*
>> it compiles fine.
>> That's why I use a pretty long exclusion list - but this is not enough.
>> The index without split-name-index is about half the size. I think the
>> first list of options only matters on the actual mkgmap creation run - and
>> does not change anything anymore on the second step of creating the address
>> index. Maybe I missed a couple - but most should be excluded.
>>
>> --mdr7-del=pri;sec;ter;cw;min;unsf,uncl;living;pdstrn;trk;pth;ft;fp;brdlw;rd;byw;ser;cw;mrt;crt;Bklane;Bktrk;opp;Opptrk;g0;g1;g2;g3;g4;g5;g6;t1;t2;t3;t4;t5;t6;xbk;cr;mr;hr;mr0;mr1;mr2;mr3;mr4;mr5;mr0.;mr1.;mr2.;mr3.;mr4.;mr5.;mr6.;mr00;mr01;mr02;mr03;mr04;mr05;mr06;mr11;mr12;mr13;mr14;mr15;mr16;mr20;mr21;mr22;mr23;mr24;mr25;mr26;mr30;mr31;mr32;mr33;mr34;mr35;mr35;mr40;mr41;mr41;mr43;mr44;mr45;mr46;mr50;mr51;mr52;mr53;mr54;mr55;mr56;mr60;mr61;mr62;mr63;mr64;mr65;mr66;m0;m1;m2;m3;m4;m5;m0.;m1.;m2.;m3.;m4.;m5.;m6.;m00;m01;m02;m03;m04;m05;m06;m11;m12;m13;m14;m15;m16;m20;m21;m22;m23;m24;m25;m26;m30;m31;m32;m33;m34;m35;m35;m40;m41;m41;m43;m44;m45;m46;m50;m51;m52;m53;m54;m55;m56;m60;m61;m62;m63;m64;m65;m66;mr.0;mr.1;mr.2;mr.3;mr.4;mr.5;mr.6;m.0;m.1;m.2;m.3;m.4;m.5;m.6;m1.;m2.;m3.;m4.;m5.;m6.;lmr;mrx0;mrx1;mrx2;mrx3;mrx4;mrx5;mrx6;mx0;mx1;mx2;mx3;mx4;mx5;mx6;bridge;swing;aqueduct;viaduct;"ntnl
>> bndry";admin_lv=4;"ntnl
>> park";weir;dam;waterfall;rapids;stream;river;drain;ditch;"huge
>> river";"hugest river";"medium river";"small river";"wide river";"hugest
>> canal";"huge canal";"medium canal";"small canal";"big
>> canal";canal;snowpark;Skitour-descent;Skipiste;Skislope;illuminated;Nordic-Skiing;skitour;platter;ropetow;tbar;chairlift;goods_lift;runway;taxiway;apron;VFS0;VFS1;VFS2;VFS3;VFS4;VFS5;VFS6;VFS7;usf;cstrn;steps;parking_aisle;res;cob;cobstn;public_footway;public_footpath
>> --mdr7-excl=pri;sec;ter;cw;min;unsf,uncl;pdstrn;trk;pth;ft;fp;brdlw;rd;byw;ser;cw;mrt;crt;Bklane;Bktrk;opp;Opptrk;g0;g1;g2;g3;g4;g5;g6;t1;t2;t3;t4;t5;t6;xbk;cr;mr;hr;mr0;mr1;mr2;mr3;mr4;mr5;mr0.;mr1.;mr2.;mr3.;mr4.;mr5.;mr6.;mr00;mr01;mr02;mr03;mr04;mr05;mr06;mr11;mr12;mr13;mr14;mr15;mr16;mr20;mr21;mr22;mr23;mr24;mr25;mr26;mr30;mr31;mr32;mr33;mr34;mr35;mr35;mr40;mr41;mr41;mr43;mr44;mr45;mr46;mr50;mr51;mr52;mr53;mr54;mr55;mr56;mr60;mr61;mr62;mr63;mr64;mr65;mr66;m0;m1;m2;m3;m4;m5;m0.;m1.;m2.;m3.;m4.;m5.;m6.;m00;m01;m02;m03;m04;m05;m06;m11;m12;m13;m14;m15;m16;m20;m21;m22;m23;m24;m25;m26;m30;m31;m32;m33;m34;m35;m35;m40;m41;m41;m43;m44;m45;m46;m50;m51;m52;m53;m54;m55;m56;m60;m61;m62;m63;m64;m65;m66;mr.0;mr.1;mr.2;mr.3;mr.4;mr.5;mr.6;m.0;m.1;m.2;m.3;m.4;m.5;m.6;m1.;m2.;m3.;m4.;m5.;m6.;lmr;mrx0;mrx1;mrx2;mrx3;mrx4;mrx5;mrx6;mx0;mx1;mx2;mx3;mx4;mx5;mx6;"ntnl
>> bndry"
>>
>> On Wed, 1 Dec 2021 at 17:39, Gerd Petermann <
>> gpetermann_muenchen at hotmail.com> wrote:
>>
>>> Hi Felix,
>>>
>>> your index is probably simply too large (> 2GB).
>>> The attached patch fixes another possible overflow, but I am not sure if
>>> such a file size is allowed for the index.
>>>
>>> Maybe try to remove specific items from the index (see --mdr7-excl,
>>> --mdr7-del)
>>> I think the reason for the huge size is your style which combines a lot
>>> of information in the road name, e.g. "102 Thw-Göttinger Weg T2 Pth
>>> Zentralalpenweg 02 Hr"
>>> or "137 Mindener Jubiläumsweg T2 Pth Mindener Jubiläumsweg Hr"
>>> (found these in omtb_austria_13.05.2021)
>>> All this information is stored in the PC index (a string table), and
>>> with --split-name-index lots of entries are produced for strings like "Pth"
>>> or "T2" or "Hr"
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
>>> Felix Hartmann <extremecarver at gmail.com>
>>> Gesendet: Mittwoch, 1. Dezember 2021 17:16
>>> An: Development list for mkgmap
>>> Betreff: Re: [mkgmap-dev] Exception in thread "main"
>>> java.lang.IllegalArgumentException . on Europe extract
>>>
>>> I found out that if I remove --split-name-index then it compiles fine.
>>> I'm just trying a minimal setup option wise and could look if it's a
>>> special .img tile or not.
>>> I could upload all map tiles somewhere for further analysis - or try
>>> some more things before doing that (12GB data or so).
>>>
>>> On Wed, 1 Dec 2021 at 13:08, Felix Hartmann <extremecarver at gmail.com
>>> <mailto:extremecarver at gmail.com>> wrote:
>>> Hi Gerd,
>>>
>>> here is the log - hope you can find out something with it - but I think
>>> it is missing the problem - there are only 3 lines before the crash - all
>>> else is many minutes before.
>>>
>>> 2021/12/01 12:55:08 INFO (ImgFS): bs=512, whole size=-1789952000,
>>> hb=-14501, fb=-3481499, blocks=-3496000
>>>
>>> 2021/12/01 12:55:08 INFO (ImgFS): bs=1024, whole size=-1786235904,
>>> hb=-3623, fb=-1740748, blocks=-1744371
>>>
>>> 2021/12/01 12:55:08 INFO (ImgFS): Best block size: 512
>>> sizeInBlocks=-3496000, reserved=-14501
>>>
>>>
>>> On Wed, 1 Dec 2021 at 08:57, Gerd Petermann <
>>> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>
>>> wrote:
>>> Hi Felix,
>>>
>>> hm, in that case one of the two factors is already negative.
>>> You can enable logging and run the combiner step again (no need to
>>> compile the tiles again)
>>> Add this to the log config:
>>> uk.me.parabola.imgfmt.sys.ImgFS.level=FINE
>>>
>>> BTW: The index contains not only address search data, also POI.
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
>>> mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <
>>> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>
>>> Gesendet: Mittwoch, 1. Dezember 2021 02:27
>>> An: Development list for mkgmap
>>> Betreff: Re: [mkgmap-dev] Exception in thread "main"
>>> java.lang.IllegalArgumentException . on Europe extract
>>>
>>> Hi Gerd,
>>>
>>> thanks for the quick response. Any other way I can try to get more debug
>>> information?
>>>
>>> I tried with the patch but I still get:
>>> gmapi-minimal: Skipping file
>>> E:\OpenMTBMap\contourlines20\europe\75500448.img
>>> Exception in thread "main" java.lang.IllegalArgumentException
>>>         at java.base/sun.nio.ch<http://sun.nio.ch
>>> >.FileChannelImpl.position(FileChannelImpl.java:355)
>>>         at uk.me.parabola.imgfmt.sys.ImgFS.sync(ImgFS.java:249)
>>>         at uk.me.parabola.imgfmt.sys.ImgFS.close(ImgFS.java:325)
>>>         at
>>> uk.me.parabola.mkgmap.combiners.MdrBuilder.onFinish(MdrBuilder.java:334)
>>>         at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:690)
>>>         at
>>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126)
>>>         at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:147)
>>>         at uk.me.parabola.mkgmap.main.Main.main(Main.java:118)
>>>
>>> On Tue, 30 Nov 2021 at 14:48, Gerd Petermann <
>>> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com
>>> ><mailto:gpetermann_muenchen at hotmail.com<mailto:
>>> gpetermann_muenchen at hotmail.com>>> wrote:
>>> Hi Felix,
>>>
>>> might be an overflow. The code line is
>>> file.position(param.getReservedDirectoryBlocks() * param.getBlockSize());
>>>
>>> so two ints are multiplied.
>>>
>>> Please try if the attached patch changes something.
>>>
>>> Gerd
>>>
>>> ________________________________________
>>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
>>> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
>>> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
>>> mkgmap-dev-bounces at lists.mkgmap.org.uk>>> im Auftrag von Felix Hartmann
>>> <extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
>>> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>
>>> Gesendet: Dienstag, 30. November 2021 13:23
>>> An: Development list for mkgmap
>>> Betreff: [mkgmap-dev] Exception in thread "main"
>>> java.lang.IllegalArgumentException . on Europe extract
>>>
>>> Using mkgmap version 4808 - when compiling Europe. It went through fine
>>> on all geofabrik country extracts.
>>> I then upgraded to mkgmap 4819 and yesterday downloaded a new europe
>>> extract (4 days newer) from geofabrik to see if I can reproduce it again -
>>> and the error message is identical.
>>> I'm going to run with default style now again - to see if it makes a
>>> difference.
>>>
>>> C:\openmtbmap\mkgmap.jar --max-jobs=11 --order-by-decreasing-area
>>> "--generate-sea" --code-page=65001
>>> "--precomp-sea=E:\OpenMTBMap\osmpbf_geofabrik\sea.zip"
>>> "--style-file=C:\openmtbmap\openmtbmap_style"
>>> --add-boundary-nodes-at-admin-boundaries --fix-roundabout-direction
>>> --improve-overview --drive-on=detect --allow-reverse-merge --lower-case
>>> --nsis --index --levels="0:24, 1:23, 2:22, 3:21, 4:20, 5:19, 6:18"
>>> --overview-levels="7:17, 8:16, 9:15, 10:14, 11:13, 12:12"
>>> --add-pois-to-areas
>>> --pois-to-areas-placement=entrance=main;entrance=yes;building=entrance;barrier=entrance
>>> --simplify-lines=23:2.6,22:4.2,21:5.4,20:6,19:7,18:7.5,17:4.5,16:5,15:5,14:6
>>> --simplify-polygons=23:3.6,22:7,21:6,20:9,17:5.4
>>> --add-boundary-nodes-at-admin-boundaries=2 --cycle-map
>>> --ignore-fixme-values --housenumbers
>>> --road-name-config=C:\openmtbmap\openmtbmap_svn\roadNameConfig.txt
>>> --split-name-index --link-pois-to-ways --ignore-turn-restrictions
>>> --polygon-size-limits="24:12, 23:14, 22:14, 21:20, 20:20, 19:20, 18:20
>>>  , 17:20, 16:20, 15:20, 14:20, 13:25" --description=omtb_eu
>>> --show-profiles=1  --location-autofill=bounds,is_in,nearest
>>> --bounds=E:\OpenMTBMap\osmpbf_geofabrik\bounds.zip --route
>>> --country-abbr=eu --country-name=europe --mapname=65500000 --family-id=6550
>>> --product-id=1 --series-name=omtb_europe_29.11.2021_UC_LOCAL
>>> --family-name=mtb_eu_29.11.2021_UC_LOCAL --tdbfile --x-gmapi-minimal
>>> --overview-mapname=mapsetc --keep-going
>>> --area-name="europe_29.11.2021_UC_LOCAL_omtb" -c
>>> D:\openmtbmap\maps\template.europe
>>> E:\OpenMTBMap\contourlines20\europe\7*.img typeu.TYP  1>NUL
>>>
>>>
>>> Number of MapFailedExceptions: 0
>>> gmapi-minimal: Writing freshly compiled file
>>> C:\openmtbmap\maps\.\65500000.img
>>> gmapi-minimal: Writing freshly compiled file
>>> C:\openmtbmap\maps\.\65500001.img
>>> gmapi-minimal: Writing freshly compiled file
>>> C:\openmtbmap\maps\.\65500002.img
>>> gmapi-minimal: Writing freshly compiled file
>>> C:\openmtbmap\maps\.\65500003.img
>>> gmapi-minimal: Writing freshly compiled file
>>> C:\openmtbmap\maps\.\65500004.img
>>> ................
>>> skipped
>>> .....................
>>> long list of contourline files:
>>> .........
>>> gmapi-minimal: Skipping file
>>> E:\OpenMTBMap\contourlines20\europe\75500447.img
>>> WARNING (global): Input file
>>> E:\OpenMTBMap\contourlines20\europe\75500448.img has different code page
>>> 1252
>>> WARNING (global): Input file
>>> E:\OpenMTBMap\contourlines20\europe\75500448.img has different charset type
>>> 10
>>> WARNING (global): Input files have different code pages
>>> gmapi-minimal: Skipping file
>>> E:\OpenMTBMap\contourlines20\europe\75500448.img
>>> .... 448 is the highest number contourlines file
>>>
>>> ............And here comes the error
>>> it happens on writing the address index mdr file - mapset.img and
>>> mapset.tdb have already been created.
>>>
>>> Exception in thread "main" java.lang.IllegalArgumentException
>>>         at java.base/sun.nio.ch<http://sun.nio.ch><http://sun.nio.ch
>>> >.FileChannelImpl.position(FileChannelImpl.java:355)
>>>         at uk.me.parabola.imgfmt.sys.ImgFS.sync(ImgFS.java:249)
>>>         at uk.me.parabola.imgfmt.sys.ImgFS.close(ImgFS.java:325)
>>>         at
>>> uk.me.parabola.mkgmap.combiners.MdrBuilder.onFinish(MdrBuilder.java:334)
>>>         at uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:690)
>>>         at
>>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReader.java:126)
>>>         at uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:147)
>>>         at uk.me.parabola.mkgmap.main.Main.main(Main.java:118)
>>>
>>>
>>>
>>> --
>>> Felix Hartman - Openmtbmap.org & VeloMap.org
>>>
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
>>> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
>>> mkgmap-dev at lists.mkgmap.org.uk>>
>>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>
>>>
>>> --
>>> Felix Hartman - Openmtbmap.org & VeloMap.org
>>>
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
>>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>
>>>
>>> --
>>> Felix Hartman - Openmtbmap.org & VeloMap.org
>>>
>>>
>>>
>>> --
>>> Felix Hartman - Openmtbmap.org & VeloMap.org
>>>
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev at lists.mkgmap.org.uk
>>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>>
>>
>> --
>> Felix Hartman - Openmtbmap.org & VeloMap.org
>>
>>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
>

-- 
Felix Hartman - Openmtbmap.org & VeloMap.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20211201/d0f20006/attachment-0001.html>