[mkgmap-dev] Commit r4809: fix java.lang.AssertionError while building index from unicode tiles

Sun Oct 24 17:14:25 BST 2021

Hi Carlos

When mkgmap doesn't have a resources/sort for the given code page, it
defaults the sort to cp1252 (Western European).

As part of building the the various indexes, it sorts counties,
regions, cities, streets etc using this sort, but any characters that
don't have a defined sort order are ignored in the ordering. The result
of this is that, using cp1252 on Chinese, all names seem the same.

I suspect that indexes are mostly empty and find is ignoring them.

There is some logic that is differentiating the names in these
structures on exact naming, and this inconsistency causes the assertion
crash.

The actual output in the map image is cp836, which Basecamp and
Mapsource appear to handle. I don't know how well it is supported by
Garmin devices.

Is there a reason for using cp836 rather than cp65001/unicode?

Ticker

On Sun, 2021-10-24 at 16:22 +0200, Carlos Dávila wrote:
> using copy from JOSM/paste into BaseCamp, I could test address
> searches 
> and they seem to work.
> 
> El 23/10/21 a las 23:50, Ticker Berkin escribió:
> > Hi Carlos
> > 
> > mkgmap doesn't have a resources/sort for code-page 936 (Microsoft's
> > character encoding for simplified Chinese). I was surprised it
> > doesn't
> > give any warning about this. I'll look more closely tomorrow to see
> > what happens when it doesn't find the resource file.
> > 
> > I presume this didn't crash before, but did the index work?
> > 
> > I suspect this will have many of the same problems as unicode sort
> > had
> > for unspecified characters.
> > 
> > I'll also investigate the other change relating to collation
> > strength.
> > 
> > Ticker
> > 
> > On Sat, 2021-10-23 at 22:26 +0200, Carlos Dávila wrote:
> > > Hi devs.
> > > 
> > > With this new version I get a new crash, but now with --code-
> > > page=936,
> > > not with unicode:
> > > 
> > > Exception in thread "main" java.lang.AssertionError: mdr20 value
> > > changed
> > > f=5174 t=5180 count=2995
> > >           at
> > > uk.me.parabola.imgfmt.app.mdr.Mdr5Record.setMdr20(Mdr5Record.java
> > > :134
> > > )
> > >           at
> > > uk.me.parabola.imgfmt.app.mdr.Mdr20.buildFromStreets(Mdr20.java:8
> > > 4)
> > >           at
> > > uk.me.parabola.imgfmt.app.mdr.MDRFile.writeSections(MDRFile.java:
> > > 335)
> > >           at
> > > uk.me.parabola.imgfmt.app.mdr.MDRFile.write(MDRFile.java:270)
> > >           at
> > > uk.me.parabola.mkgmap.combiners.MdrBuilder.onFinish(MdrBuilder.ja
> > > va:3
> > > 31)
> > >           at
> > > uk.me.parabola.mkgmap.main.Main.endOptions(Main.java:690)
> > >           at
> > > uk.me.parabola.mkgmap.CommandArgsReader.readArgs(CommandArgsReade
> > > r.ja
> > > va:126)
> > >           at
> > > uk.me.parabola.mkgmap.main.Main.mainStart(Main.java:147)
> > >           at uk.me.parabola.mkgmap.main.Main.main(Main.java:118)
> > > 
> > > mkgmap command: java -ea -jar mkgmap-r4809.jar --index
> > > --bounds=bounds.zip --housenumbers --code-page=936 31177013.o5m
> > > 
> > > https://files.mkgmap.org.uk/download/524/31177013.o5m
> > > 
> > > El 22/10/21 a las 9:42, svn commit escribió:
> > > > Version mkgmap-r4809 was committed by gerd on Fri, 22 Oct 2021
> > > > 
> > > > fix java.lang.AssertionError while building index from unicode
> > > > tiles
> > > > mdrUnicode_v2.patch by Ticker Berkin
> > > > 
> > > > http://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=4809
> > > > _______________________________________________
> > > > mkgmap-dev mailing list
> > > > mkgmap-dev at lists.mkgmap.org.uk
> > > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev at lists.mkgmap.org.uk
> > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > 
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> 
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev