[mkgmap-dev] Performance with zipped hgt files

Mon Jan 8 14:46:56 GMT 2018

Hi Frank,

the current problem with zipped files is that I have to extract them into a buffer that is kept in Java Heap (managed by Garbage Collection)
while uncompressed files are manged by class MappedByteBuffer. These are not managed as Java Heap, they are somewhat virtual and
typically only require a small portion of the available memory, see
https://howtodoinjava.com/java-7/nio/java-nio-2-0-memory-mapped-files-mappedbytebuffer-tutorial/

I don't yet see a need to change splitter like that but I can do that as well.
I am now playing with code to extract zipped files into a temp dir.

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Frank Stinner <frank.stinner at leipzig.de>
Gesendet: Montag, 8. Januar 2018 15:30:08
An: mkgmap-dev at lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] Performance with zipped hgt files

Hi Gerd,

at last you need the data uncompressed in a buffer, right? That's why i see no difference between using compressed or uncompressed hgt files. Compressed files need a little bit more time for uncompressing, but after that the data need the same size in memory. That's why i see no vantage for mkgmap if the data are stored in an other form. It can only save disc space, no heap space.

What speaks against a limitation for the extent of a maptile? This limited the count of hgt's for one maptile too.

You have the option --max-areas for splitter. We need only an additional option like --maxextent or whatever. Perhaps we have then maptiles without points, lines or areas (on oceans, deserts or so), but why not?

Frank