[mkgmap-dev] Please try and report results of Huffman compression

Sun Jan 16 18:08:11 GMT 2022

Hi Gerd

A bit more:

Are the boundary conditions for tab1Width (ie the space for the shifted
minCode values) correct? eg if depth is 16, there is no minCode for
this level, so the maximum minCode appears at level 15, the range being
1..2^15-1 (I think)

I see a couple of problems with unicode:

1/ To fit the char into prefixTable, must be either 1 byte in utf8
representation or <= 8 bits in absolute value (or v0 is made 16 bits
instead). It could be that lookupBits is reduced until this isn't a
problem. What did example unicode mdrs have?

2/ If >256 chars not in prefix table, the offset per level in tab1
needs to be bigger than a byte.

I'm still surprised that it didn't appear to work when all the chars
fit into the prefix table. Maybe the top nibble of lookupBits is a flag
saying that tab1 is present, although the tab1length would do that
equally well. The 8 after 1stLevWithStruct and rowsIn1stTable is still
troubling, as I'd have expected it to be associated with the
prefixTable

Ticker

On Sun, 2022-01-16 at 12:37 +0000, Ticker Berkin wrote:
> Hi Gerd
> 
> I'm just starting to look at this, but won't be able to do much
> today,
> but some initial findings:
> 
> If just making gmapi from .img, unless --code-page is given it
> doesn't
> do compression, however the Mdr generated with the expected charset.
> 
> MdrCheck and MdrDisplay, including string dump, look good
> 
> Ticker
> 
> On Sat, 2022-01-15 at 15:59 +0000, Gerd Petermann wrote:
> > Hi all,
> > 
> > during the last weeks Ticker and I decoded Garmin's String
> > compression
> > of the global index for the PC (the *_mdr.img) .
> > In the mdr2 branch I've coded support for that in mkgmap and I
> > think
> > its ready for a first beta test.
> > 
> > It works with codepages 1250 ..1258 or 65001. 
> > I've no idea how the 2-byte pages work, but Garmin seems to support
> > that, too. Maybe I'll work on this later.
> > 
> > The branch version produces around 200 lines of diagnostic output
> > to
> > show what happens when compression is enabled.
> > This will be changed to debug level later.
> > 
> > The encoding is enabled by default, special option --x-no-mdr15-
> > compression can be used to turn it off.
> > 
> > Note that the size of the *_mdr.img file will be smaller (which is
> > esp.
> > intersting for those who get close to the 2G limit)
> > Note also that a zipped *_mdr.img can and will be larger since zip
> > cannot remove as much redundancy. 
> > Note also that this branch includes the changes from the faster-mp
> > branch and introduces a few changes 
> > which are relavant for --lower-case maps.
> > 
> > How to test?
> > Install the map in MapSource and check if you see any unexpected
> > characters in the result lists or maybe even crashes while
> > searching.
> > If so, please try the same input files with the --x-no-mdr15-
> > compression to verify if this is caused by compression or maybe
> > other
> > changes.
> > 
> > Please don't yet publish the maps, but let me know if you see
> > anything
> > unexpected. Some changes reg. lower-case in this branch are still
> > experimental.
> > 
> > The binary is here:
> > https://www.mkgmap.org.uk/download/mkgmap-mdr2-r4865.zip
> > 
> > If you see errors which are caused by compression, please attach
> > the
> > diagnostic output.
> > 
> > @Ticker: Please review the source.
> > 
> > Gerd
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> 
> 
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev