[mkgmap-dev] DEM File Format and mkgmap

Mon Dec 4 15:02:27 GMT 2017

Hi Gerd

Thanks for letting me know

That is probably why eventually it stopped   with repeated plateau len 
64 messages on one of the dems builddemfile created

I have some 'old' non nt Garmin Topo maps with dem and will have a go 
using your display.jar

r

Nick


On 04/12/2017 13:49, Gerd Petermann wrote:
> Hi Nick,
>
> I just found out that display tool can not yet read DEM tiles with multiple zoom levels. Working on it now...
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von osm at pinns <osm at pinns.co.uk>
> Gesendet: Montag, 4. Dezember 2017 14:35:15
> An: mkgmap-dev at lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] DEM File Format and mkgmap
>
> Many thanks Gerd
>
> Will have a go at it today and let you know my findings, if any !
>
> Nick
>
>
> On 04/12/2017 13:27, Gerd Petermann wrote:
>> Hi Nick,
>>
>> My understanding is that the prediction starts with the values from the header (min and max height in the tile)
>> The first row is indeed rather easy to decode. Frank has described many more details in his pdf, and his program
>> seems to create valid DEM data since Basecamp displays it, I just try to reduce the number of special cases now.
>>
>> A compiled display.jar r505 is here:
>> http://files.mkgmap.org.uk/download/361/display.jar
>> If you also download and unzip this mkgmap package http://www.mkgmap.org.uk/download/mkgmap-r3997.zip
>> you can run e.g.
>> java -cp display.jar;mkgmap-r3997\mkgmap.jar test.display.DemDisplay 12345678.DEM > log.txt
>>
>> I am interested in those files created by BuildDemFile which create a run time exception. If you find one or more, I need the original *.TRE file(s)
>> which were the input for BuildDemFile .
>>
>> Gerd
>> ________________________________________
>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von osm at pinns <osm at pinns.co.uk>
>> Gesendet: Montag, 4. Dezember 2017 13:00:27
>> An: mkgmap-dev at lists.mkgmap.org.uk
>> Betreff: Re: [mkgmap-dev] DEM File Format and mkgmap
>>
>> Hi Gerd
>>
>> Presumably some hgt data gets stored in a tile from which other data is
>> predicted? Perhaps the first 'row'?
>>
>> Would be great if you could  compile the DemDisplay.java so we can do
>> ome testing as well.
>>
>> I'm glad you are almost 'getting there'!
>>
>> r
>>
>> Nick
>>
>>
>> On 03/12/2017 11:16, Gerd Petermann wrote:
>>> Hi all,
>>>
>>> I think I understand now much more about this format. I used Frank Stinners code for BuildDEMFile and his pdf to write a decoder in the display tool.
>>> I still fails to read some Garmin DEM files but seems to work with the files produced by  BuildDEMFile.
>>> My current work flow is to generate DEM files with BuildDEMFile and read them with my decoder. If the decoder fails I can find out why because I can trace step by step what BuildDEMFile does.
>>> I guess that I did not yet hit all special cases identified by Frank, but I am making good progress now, so I hope I'll be able to start implementing a DEM encoder in mkgmap soon.
>>>
>>> Reg. the format:
>>> It is quite different to those used in other GARMIN sub files like NOD, NET, RGN etc.  In short: The data is organized in tiles, typically 64*64 values. Instead of storing each value
>>> Garmin uses some statistics to predict the next value based on the already processed values. This predicted value is subtracted from the real value and only the delta is stored.
>>> The better the prediction, the closer to 0 is the delta. The prediction also calculates the error-probability of the prediction and uses this to select the number of bits which are
>>> needed to store the value.  If the delta is larger than expected the so called BigBin encoding is used.
>>> The format to store the delta is optimized for values close to 0, so that many deltas are stored with only 1 or 2 bits.
>>> A 2nd trick to save space is to use run length encoding for repeated values.
>>> So, the tricky part is to find out exactly how the prediction algo by Garmin works.
>>> Frank has done really great work here.
>>>
>>> For those who want to try the display tool:
>>> You have to compile on your own and you also need an installed mgkmap.jar.
>>> If that is available a command like this (replace d:\... with your paths) should produce a large txt file:
>>> java -ea -Xmx1000m -cp d:\display\dist\display.jar;d:\mkgmap\dist\mkgmap.jar test.display.DemDisplay test.DEM > dem_decode.txt
>>>
>>> I am now searching for DEM files produced by BuildDEMFile  which are not properly processed by display tool. When it fails you'll see a traceback like this
>>> Exception in thread "main" java.lang.RuntimeException: did not read all avalaible bytes
>>>            at test.display.DemDisplay$DemTile.decodeBitStreamData(DemDisplay.java:532)
>>>            at test.display.DemDisplay$DemSection.decodeBitstream(DemDisplay.java:831)
>>>            at test.display.DemDisplay.readFileHeader(DemDisplay.java:887)
>>>            at test.display.DemDisplay.print(DemDisplay.java:845)
>>>            at test.display.CommonDisplay.display(CommonDisplay.java:171)
>>>            at test.display.CommonDisplay.display(CommonDisplay.java:196)
>>>            at test.display.DemDisplay.main(DemDisplay.java:903)
>>>
>>> Gerd
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev at lists.mkgmap.org.uk
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev