[mkgmap-dev] Commit r4809: fix java.lang.AssertionError while building index from unicode tiles

Sun Oct 24 17:31:37 BST 2021

The reason for using code-pages other than 65001 is that many Garmin 
devices fail to load non original unicode maps. See Felix explanation 
here: 
https://openmtbmap.org/download/odbl/#Compatibility_-_Unicode_vs_Non_Unicode_cannot_authenticate_maps

El 24/10/21 a las 18:14, Ticker Berkin escribió:
> Hi Carlos
>
> When mkgmap doesn't have a resources/sort for the given code page, it
> defaults the sort to cp1252 (Western European).
>
> As part of building the the various indexes, it sorts counties,
> regions, cities, streets etc using this sort, but any characters that
> don't have a defined sort order are ignored in the ordering. The result
> of this is that, using cp1252 on Chinese, all names seem the same.
>
> I suspect that indexes are mostly empty and find is ignoring them.
>
> There is some logic that is differentiating the names in these
> structures on exact naming, and this inconsistency causes the assertion
> crash.
>
> The actual output in the map image is cp836, which Basecamp and
> Mapsource appear to handle. I don't know how well it is supported by
> Garmin devices.
>
> Is there a reason for using cp836 rather than cp65001/unicode?
>
> Ticker
>
> On Sun, 2021-10-24 at 16:22 +0200, Carlos Dávila wrote:
>> using copy from JOSM/paste into BaseCamp, I could test address
>> searches
>> and they seem to work.
>>
>> El 23/10/21 a las 23:50, Ticker Berkin escribió:
>>> Hi Carlos
>>>
>>> mkgmap doesn't have a resources/sort for code-page 936 (Microsoft's
>>> character encoding for simplified Chinese). I was surprised it
>>> doesn't
>>> give any warning about this. I'll look more closely tomorrow to see
>>> what happens when it doesn't find the resource file.
>>>
>>> I presume this didn't crash before, but did the index work?
>>>
>>> I suspect this will have many of the same problems as unicode sort
>>> had
>>> for unspecified characters.
>>>
>>> I'll also investigate the other change relating to collation
>>> strength.
>>>
>>> Ticker
>>>
>>> On Sat, 2021-10-23 at 22:26 +0200, Carlos Dávila wrote:
>>>> Hi devs.
>>>>
>>>> With this new version I get a new crash, but now with --code-
>>>> page=936,
>>>> not with unicode:
>>>>
>>>> Exception in thread "main" java.lang.AssertionError: mdr20 value
>>>> changed
>>>> f=5174 t=5180 count=2995
>>>>            at
>>>> uk.me.parabola.imgfmt.app.mdr.Mdr5Record.setMdr20(Mdr5Record.java
>>>> :134
>>>> )
>>>>            at
>>>> uk.me.parabola.imgfmt.app.mdr.Mdr20.buildFromStreets(Mdr20.java:8
>>>> 4)
>>>>            at
>>>> uk.me.parabola.imgfmt.app.mdr.MDRFile.writeSections(MDRFile.java:
>>>> 335)
>>>>            at
>>>> uk.me.parabola.imgfmt.app.mdr.MDRFile.write(MDRFile.java:270)
>>>>            at
>>>> uk.me.parabola.mkgmap.combiners.MdrBuilder.onFinish(MdrBuilder.ja
>>>> va:3
>>>> 31)
>>>>            at
>>>> uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:690)
>>>>            at
>>>> uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReade
>>>> r.ja
>>>> va:126)
>>>>            at
>>>> uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:147)
>>>>            at uk.me.parabola.mkgmap.main.Main.main(Main.java:118)
>>>>
>>>> mkgmap command: java -ea -jar mkgmap-r4809.jar --index
>>>> --bounds=bounds.zip --housenumbers --code-page=936 31177013.o5m
>>>>
>>>> https://files.mkgmap.org.uk/download/524/31177013.o5m
>>>>
>>>> El 22/10/21 a las 9:42, svn commit escribió:
>>>>> Version mkgmap-r4809 was committed by gerd on Fri, 22 Oct 2021
>>>>>
>>>>> fix java.lang.AssertionError while building index from unicode
>>>>> tiles
>>>>> mdrUnicode_v2.patch by Ticker Berkin
>>>>>
>>>>> http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4809
>>>>> _______________________________________________
>>>>> mkgmap-dev mailing list
>>>>> mkgmap-dev at lists.mkgmap.org.uk
>>>>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>> _______________________________________________
>>>> mkgmap-dev mailing list
>>>> mkgmap-dev at lists.mkgmap.org.uk
>>>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev at lists.mkgmap.org.uk
>>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk
>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev