[mkgmap-dev] Fix and augment sort definitions

Tue Jan 4 13:24:31 GMT 2022

Hi Gerd

I downloaded niedersachsen-latest.osm.pbf.
Built with current trunk, my resources/sort changes, option --latin
(but not --lower-case).
Loaded gmapsupp onto eTrex HCx
Find>Cities by name "VOSS" and "VOSSBERG" gives me 3 VOSSBERGs, all in
LOWER SAXONY, including the one you mention.
The eTrex name entry has a shift that allows entry of accented/eszet/ae
etc. "VOß" etc finds the same.

Rebuilding with --lower-case, Find>Cities "VOS", "VOSS", "VOSSB" ...
all work, showing the 3 "Voßberg"s.
"VOß", "VOßB" etc also work.

The only strangeness is that the name entry also has a lower case
shift, for both standard latin and the extra chars as mentioned.
Shifting to lower-case, all letters are disabled except "ß".

Find Address doesn't allow entry for this city because none of them
have any streets.

I've just noticed that my version of trunk has mdrUnicode_v9b.patch
applied, but the only significant difference is in MDR25 and will only
effect street cities rather then POI ones.

Will do the same testing on other eTrex

Ticker

On Tue, 2022-01-04 at 11:36 +0000, Gerd Petermann wrote:
> Hi Ticker,
> 
> OK, maybe you find more on that.
> BTW: Voßberg is very special as it probably also influences the MDR 17
> content.
> 
> I think I'll merge the faster-mp branch into trunk this afternoon and
> continue on
> the Huffman encoding later.
> 
> Gerd
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
> Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> Gesendet: Dienstag, 4. Januar 2022 12:01
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] Fix and augment sort definitions
> 
> Hi Gerd
> 
> I'm just building your area to test on my UK configured devices.
> 
> I'm not sure yet of the benefit of testing with the mdr2 branch.
> Actually, until we've better understood the indexing issues thrown up
> by use of --lower-case, the extra complications of ß seem too much.
> 
> Ticker
> 
> On Tue, 2022-01-04 at 09:50 +0000, Gerd Petermann wrote:
> > Hi Ticker,
> > 
> > I think you need option -g with svn log to see changes done in
> > branches.
> > 
> > Anyhow, I think I made a mistake back then because I didn't think of
> > devices which
> > are not configured to show a German keyboard (which has keys for äöü
> > and ß).
> > 
> > You are probably right that the expands are better for this case and
> > I don't
> > remember what problems I had with the ß. It is very likely that the
> > open
> > collator strength questions are the better approach.
> > 
> > I test with a tile around my hometown Wildeshausen
> > 55410043: 2447360,389120 to 2469888,407552
> > #       : 52.514648,8.349609 to 52.998047,8.745117
> > 
> > One problem that I found on the Oregon is the search for a
> > name that appears as a nearby city called "Voßberg"
> > https://www.openstreetmap.org/node/599127249
> > 
> > This name doesn't appear in the Oregons basemap.
> > I created a map with --latin and --lower-case.
> > When I search for Voß it is not found. Same when I search
> > for Voßber. Only the search for the full name works.
> > Voss is also not found, but Vossberg works.
> > 
> > Without the patch, search for Voß returns Voßberg,
> > search for Vossberg does that also, which is quite confusing
> > for me.
> > My basemap shows
> > --------- Description -----------------------------------------------
> > -----------
> > 00000035 | 41 53 43 49 49 20 53 6f | Description: ASCII Sort
> > 0000003d | 72 74 00                |
> > --------- Character table header ------------------------------------
> > -----------
> > 00000040 | 000000 | 5c 00                   | sub header len 92
> > 00000042 | 000002 | 01 00                   | id1 1
> > 00000044 | 000004 | 01 00                   | id2 1
> > 00000046 | 000006 | e4 04                   | codepage 1252
> > 
> > Maybe I should repeat those tests with the mdr2 branch?
> > 
> > Gerd
> > 
> > 
> > 
> > 
> > ________________________________________
> > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag
> > von Ticker Berkin <rwb-mkgmap at jagit.co.uk>
> > Gesendet: Dienstag, 4. Januar 2022 09:40
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] Fix and augment sort definitions
> > 
> > Hi Gerd
> > 
> > Sorry - I hadn't noticed these changes. They don't show up with
> > $ svn log resources/sort/cp1252.txt or cp1254.txt
> > 
> > All the other mkgmap sort files have all the expansions possible
> > including the eszett and diphthongs if applicable.
> > 
> > The two non-mkgmap sort files (0000848.SRT/Turkey and
> > I00003A0.SRT/adriatic TOPO) have expand for "ß" and some of "Œ"... so
> > I
> > presumed it was expected and reasonably supported.
> > 
> > In the binaries, the expand is expressed as a list of sortOrders
> > {primary,secondary,tertiary}. The secondary and tertiary are
> > disrupted
> > and don't match ones from actual characters (in the case of "ß", the
> > two s's get different secondaries). So these double chars will sort
> > after the real char and only match with PRIMARY.
> > 
> > As there many unknowns about how to make --lower-case indexing work
> > and
> > the setting, regarding collation strength, of the bit-flag indicating
> > same-name in some of the MDR sections, I feel that it is better to
> > have
> > the all the expands.
> > 
> > However, if you are against this, I'll redo cp1252 without these
> > expansions. I'm not sure of the basis of having the diphthongs as
> > alternate secondaries of their first character and the eszett as a
> > unique character.
> > 
> > Ticker
> > 
> > On Mon, 2022-01-03 at 11:44 +0100, Gerd Petermann wrote:
> > > Hi Ticker,
> > > 
> > > see
> > > https://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=3948
> > > https://www.mkgmap.org.uk/websvn/revision.php?repname=mkgmap&rev=3949
> > > 
> > > the sortResource.patch reverts these changes.
> > > 
> > > In Mapsource the results are a bit better with your patch.
> > > I'll try again with my Oregon later.
> > > 
> > > Gerd
> > > _______________________________________________
> > > mkgmap-dev mailing list
> > > mkgmap-dev at lists.mkgmap.org.uk
> > > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > 
> > 
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> 
> 
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev