[mkgmap-dev] What's the maximum size of global index MDR size?

Sun Dec 12 11:07:39 GMT 2021

Hi all,

for now I've added a check to stop if the MDR subfile gets > 2G.
My findings reg. string compression in MDR 15:
If Mdr15 is compressed I also find an MDR 16 section which is rather small.
My first idea was that MDR15 might be the content of a zipped file, but offsets
into MDR15 never exceed the size of the MDR15 section, so it's more likely that
something simple like Huffman encoding is used and MDR 16 contains further
data (the frequencies of the Huffman tree?)

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <GPetermann_muenchen at hotmail.com>
Gesendet: Samstag, 11. Dezember 2021 11:06
An: mkgmap-dev at lists.mkgmap.org.uk
Betreff: [mkgmap-dev] What's the maximum size of global index MDR size?

Hi all,

does anybody know the actual limits?
The structure of the MDR file encodes offsets with 4 bytes.
We can assume those are interpreted as unsigned integers, so 0xffffffff (2^32 ~ 4G)
would be the highest possible offset, the section length is also encoded with 4 bytes,
so maybe we can even write a correct MDR that is close to 8G if the last section is the largest.
The MDR sub file for the PC is stored in the *_mdr.img file which also contains the SRT sub file.
As far as I know the *.img file can grow > 4G, so I hope that our routines to write large *.img files are OK.

As of now, the --gmapi option doesn't work with an MDR sub file > 2G, the corresponding folder will contain a *.MDR file with 0 bytes.
The display tool programs also fail to analyse such a file.

I see that both mkgmap and MapSource can handle an *_mdr.img written
by mkgmap file that is > 2G as long as the MDR sub file itself is < 2G.
Current mkgmap fails with different errors when an offset in the MDR
sub file gets > 2G. I started to fix those errors but MapSource crashes
with such an index file.
My problem: I don't know if it crashes because mkgmap still does something
wrong or because MapSource interprets the offset field as a signed int
(as mkgmap does so far in some places). With signed int the
values > 2G are interpreted as negative values and that cannot work.

Is it possible to have an MDR sub file > 2G in an *.img file?
If yes I may invest more time to find out what is wrong in mkgmap.
If no, we may skip the writing of sections
(e.g. Mdr 21 (streets sorted by region) or Mdr 22 (streets sorted by country)
or maybe even Mdr 15 (the string table).

Has anybody an idea how the string compression of Mdr15 section might work?

Gerd
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev