[mkgmap-dev] Errors converting "mapnik.txt" into "mapnik.typ" (information only)

Tue Mar 31 17:11:10 BST 2020

Hi Eric

There is a lot here!

I don't want to spend a lot of time going through it point by point
here are some comments and some changes, helpful to TYPViewer users,
that could be made to mapnik.txt and mkgmap.

1/ DONE: remove a comma from a STRING default line. This is because
mkgmap and TYPViewer use different parsing methods for:
   String[#]=[lang#,]description
where [] indicates optional. mkgmap spots there is no language#

2/ PATCH in progress: Change a character used to represent a pixel
colour. These characters are arbitrary, but something, either the . or
the 1, causes TYPViewer to write an incorrect .typ file

3/ pixel characters can be chosen by the user; it is NOT an error to
use different characters per icon. TYPViewer changes them to the
characters it would use if it was reading a binary .typ file.

4/ Implement the [_comments] ... [end] section in mkgmap. These
comments wouldn't put it into the .typ file unlike TYPViewer, which
does. It is good for copyright and version information but not actual
comments. Having experimented with this I find that TYPViewer is
inconsistent when reading it back and sometimes reports a different
header length and doesn't pick up the comments.

5/ What I meant by "supposed Win 1252 Typ file" is that it is very easy
to get TYPViewer to generate a file where the Typ header says the
CodePage is 1252, but the strings are encoded as utf8 hence chars>127
will show incorrectly.
I was going try list some of the ways to get it correct and some that
get it wrong, but there are many variables that might have an effect on
this I'm finding this a time-consuming and pointless process.

6/ BOM and coding line. These lines are there so that there is a higher
chance of tools (editors, compilers...) getting the source coding
correct. Without these lines, some tools will scan the complete file
checking that all chars >127 are part of a valid utf-8 sequence, some
will just make assumptions and possibly get it wrong.

7/ TYPViewer recognises strings in mapnik.txt as being encoded as utf8
regardless of the method used to open the file or BOM/coding line (good
- see above). On forcing a change to be saved, it converts to the
specified CodePage, quietly mapping chars not in this CodePage to their
more generic form.

8/ mkgmap TypCompiler does check for BOM. Up until January 2020, just in the utf8 encoding, but since then in 16LE/16BE/32LE/32BE as well, also looking for "-*- coding:" near the start of the file.

9/ Change mkgmap so that the message severity for missing numbers is downgraded for:
   ProductCode=
   FID=
as these will be generated from other sources

10/ as per your recent mail, TYPViewer does change the Type hex numbers
in the _draworder section

11/ utf8 should be the standard for all source files. You suggest it is
not needed for 99% of usage, but you are ruling out most countries of
the world. There is little extra effort required to support utf8 until
we meet tools like TYPViewer.

12/ "Correct procedure" - most of this is fine for you, but I'd suggest
always using TYPViewer in utf8/65001 mode and never using it to
generate the .typ file. Then in mkgmap, use your normal --code
-page=1252 because 65001 makes the map bigger and isn't supported on
many Garmin devices.

13/ Actual translations in mapnik.txt: this is an ongoing effort by
anyone who wants to improve it. In the spreadsheet you've highlighted
quite a few default strings - I don't see what is wrong with them; they
were specified as described in the posting:

http://www.mkgmap.org.uk/pipermail/mkgmap-dev/2020q1/030254.html

You've also highlighted quite Dutch ones - could you do a patch for
these based on either the file I sent you or the latest svn source
file, or, failing that, just edit it and and sent it back to me.

Best wishes - hope everyone is healthy

Ticker

On Mon, 2020-03-30 at 21:59 +0200, eric_internet at casema.nl wrote:
> Hi Ticker, Gerd,
> 
> I have spent "some" time in investigating "mapnik.txt" vs.
> "TypViewer".
> Understatement...
> 
> Things got out of hand...
> 
> Therefore: two attachments...
> 
> Best regards,
> 
> Eric (AnkEric)