[mkgmap-dev] Fix and augment sort definitions

Thu Oct 12 15:14:16 BST 2023

Hi Gerd

The same fix was applied to cp1252.txt some time ago and this patch fixes the
rest of the sort/cp*.txt in the same way - making them do what the writer
intended.

I don't think a unit test for this is worth the time or effort.

Ticker 

On Thu, 2023-10-12 at 08:14 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > please can you provide a unit test for this?
> > 
> > Gerd
> > 
> > ________________________________________
> > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
> > Ticker
> > Berkin <rwb-mkgmap at jagit.co.uk>
> > Gesendet: Donnerstag, 12. Oktober 2023 09:18
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Fix and augment sort definitions
> > 
> > Hi Gerd
> > 
> > Last year it was reported that string ordering with the '#' character was
> > incorrect. This was
> > because, in the sort/cp*.txt files, the relevant line with the '#' was taken
> > as a comment.
> > 
> > I had a patch that fixed all the files, but it also attempted to do more
> > with
> > ß/ss and
> > dipthongs.
> > 
> > I've done another patch that doesn't have any contentious changes, just
> > fixes
> > the #, makes the
> > layout consistent between the files, increments the version/id2 values and
> > slight improvements
> > to the documentation.
> > 
> > Ticker
> > 
> > On Tue, 2022-01-11 at 14:00 +0000, Gerd Petermann wrote:
> > > > > > Hi Ticker,
> > > > > > 
> > > > > > if you don't mind I'd like to postpone this patch until the active
> > > > > > branches are merged into
> > > > > > trunk.
> > > > > > 
> > > > > > Gerd
> > > > > > 
> > > > > > 
> > > > > > ________________________________________
> > > > > > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> > > > > > von
> > > > > > Ticker Berkin
> > > > > > <rwb-mkgmap at jagit.co.uk>
> > > > > > Gesendet: Dienstag, 11. Januar 2022 11:25
> > > > > > An: Development list for mkgmap
> > > > > > Betreff: Re: [mkgmap-dev] Fix and augment sort definitions
> > > > > > 
> > > > > > Hi Gerd
> > > > > > 
> > > > > > Yes - gmapsupp builder gives a warning if id1/id2 are not consistent
> > > > > > in
> > > > > > all the .img files. It is just a warning and gmapsupp is built
> > > > > > anyway
> > > > > > and I think the warning can be ignored. gmapi doesn't notice.
> > > > > > 
> > > > > > Almost all of the significant sorting where the Garmin device...
> > > > > > needs
> > > > > > to know the sort details happens in Mdr, so this isn't a problem.
> > > > > > 
> > > > > > Other uses are mostly for de-duping/efficient processing, so these
> > > > > > shouldn't matter either.
> > > > > > 
> > > > > > However the LBL file does hold id1/id2 and many sections (Countries,
> > > > > > Regions, Cities, Zips, POIs) are sorted so the effect here is
> > > > > > unknown.
> > > > > > 
> > > > > > If using --latin2 / 1252, the only change in ordering is around
> > > > > > AE/OE
> > > > > > dipthongs.
> > > > > > 
> > > > > > Within the same commit or build as sortResource_v2, the attached
> > > > > > sortMashExp.patch should be applied, as it effects the binary SRT
> > > > > > file
> > > > > > and I don't want to increment all the id2's again. This patch
> > > > > > changes
> > > > > > the sort.expand TERTIARY mashing from 2 to 3, which is slightly more
> > > > > > consistent with the Garmin SRT binaries I've seen and allows
> > > > > > SrtDisplay
> > > > > > to show expansions with what looks like a meaningful case.
> > > > > > 
> > > > > > Ticker
> > > > > > 
> > > > > > On Tue, 2022-01-11 at 06:31 +0000, Gerd Petermann wrote:
> > > > > > > > > > Hi Ticker,
> > > > > > > > > > 
> > > > > > > > > > didn't try it: Will mkgmap complain when building an indexed
> > > > > > > > > > gmapi/gmapsupp
> > > > > > > > > > where some tiles where freshly compiled with the new version
> > > > > > > > > > and
> > > > > > > > > > others with
> > > > > > > > > > an older (like Felix and Carlos do)?
> > > > > > > > > > 
> > > > > > > > > > Gerd
> > > > > > > > > > 
> > > > > > > > > > ________________________________________
> > > > > > > > > > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im
> > > > > > > > > > Auftrag
> > > > > > > > > > von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> > > > > > > > > > Gesendet: Montag, 10. Januar 2022 12:04
> > > > > > > > > > An: Development list for mkgmap
> > > > > > > > > > Betreff: Re: [mkgmap-dev] Fix and augment sort definitions
> > > > > > > > > > 
> > > > > > > > > > Hi Gerd
> > > > > > > > > > 
> > > > > > > > > > What I meant was that keyboards/devices don't normally have
> > > > > > > > > > ways of
> > > > > > > > > > entering the single chars "…", "¼", "½", "¾", "™".
> > > > > > > > > > 
> > > > > > > > > > Names with these might be presented by Garmin software after
> > > > > > > > > > some
> > > > > > > > > > initial chars have been entered and you can then select the
> > > > > > > > > > complete
> > > > > > > > > > name that contains these chars.
> > > > > > > > > > 
> > > > > > > > > > I didn't see a good reason to remove the expand for these
> > > > > > > > > > and find
> > > > > > > > > > some
> > > > > > > > > > arbitrary sort PRIMARY for them. No one has complained about
> > > > > > > > > > them.
> > > > > > > > > > Also
> > > > > > > > > > cp65001 had over 1000 expands and I really don't want to
> > > > > > > > > > start
> > > > > > > > > > touching
> > > > > > > > > > these.
> > > > > > > > > > 
> > > > > > > > > > Ticker
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > On Mon, 2022-01-10 at 10:29 +0000, Gerd Petermann wrote:
> > > > > > > > > > > > > > Hi Ticker,
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > I've committed displaySrt_v2.patch .
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > I don't fully understand the comment
> > > > > > > > > > > > > > "Leave the above because no method of inputting them
> > > > > > > > > > > > > > anyway and
> > > > > > > > > > > > > > unlikely at start of names."
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > It is possible to enter these characters in
> > > > > > > > > > > > > > MapSource and I think
> > > > > > > > > > > > > > MapSource uses MDR12
> > > > > > > > > > > > > > when you type only a few characters for the name of
> > > > > > > > > > > > > > a POI and
> > > > > > > > > > > > > > don't
> > > > > > > > > > > > > > pick up an entry from the list.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > Gerd
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > ________________________________________
> > > > > > > > > > > > > > Von: mkgmap-dev
> > > > > > > > > > > > > > <mkgmap-dev-bounces at lists.mkgmap.org.uk> im
> > > > > > > > > > > > > > Auftrag
> > > > > > > > > > > > > > von
> > > > > > > > > > > > > > Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> > > > > > > > > > > > > > Gesendet: Montag, 10. Januar 2022 11:20
> > > > > > > > > > > > > > An: Development list for mkgmap
> > > > > > > > > > > > > > Betreff: Re: [mkgmap-dev] Fix and augment sort
> > > > > > > > > > > > > > definitions
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > Hi Gerd
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > I tried various approaches to fixing "Find" when the
> > > > > > > > > > > > > > fixed length
> > > > > > > > > > > > > > Mdr17
> > > > > > > > > > > > > > (maybe also Mdr12) prefix contains sort.expand chars
> > > > > > > > > > > > > > and couldn't
> > > > > > > > > > > > > > make
> > > > > > > > > > > > > > it work. I could documents these attempts in
> > > > > > > > > > > > > > Sort.java if you feel
> > > > > > > > > > > > > > this
> > > > > > > > > > > > > > is worthwhile.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > New patch attached that, for cp1252, leaves "ß" as
> > > > > > > > > > > > > > its own PRIMARY
> > > > > > > > > > > > > > after "s". Moved æ,Æ etc to be PRIMARIES on the
> > > > > > > > > > > > > > grounds that their
> > > > > > > > > > > > > > behaviour will be the same as "ß". Made cp1254
> > > > > > > > > > > > > > consistent as it
> > > > > > > > > > > > > > had
> > > > > > > > > > > > > > similar partial fixes.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > The main reason for the patch is to fix all the
> > > > > > > > > > > > > > other sort/cp*.txt
> > > > > > > > > > > > > > files that had line " > #" which was taken as a
> > > > > > > > > > > > > > comment, resulting
> > > > > > > > > > > > > > in
> > > > > > > > > > > > > > "#" being ignored in collation.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > With the Display patch (sent previously, but also
> > > > > > > > > > > > > > attached here),
> > > > > > > > > > > > > > it
> > > > > > > > > > > > > > can reproduce the resource/sort file from the binary
> > > > > > > > > > > > > > SRT section.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > Ticker
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > > > mkgmap-dev mailing list
> > > > > > > > > > > > > > mkgmap-dev at lists.mkgmap.org.uk
> > > > > > > > > > > > > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > _______________________________________________
> > > > > > > > > > mkgmap-dev mailing list
> > > > > > > > > > mkgmap-dev at lists.mkgmap.org.uk
> > > > > > > > > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > > > > > _______________________________________________
> > > > > > > > > > mkgmap-dev mailing list
> > > > > > > > > > mkgmap-dev at lists.mkgmap.org.uk
> > > > > > > > > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > > > > > 
> > > > > > _______________________________________________
> > > > > > mkgmap-dev mailing list
> > > > > > mkgmap-dev at lists.mkgmap.org.uk
> > > > > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > 
> > 
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev