[mkgmap-dev] Binary OSM file support?

Mon Sep 13 17:55:00 BST 2010

On Mon, Sep 13, 2010 at 6:06 AM, Steve Ratcliffe <steve at parabola.me.uk> wrote:
> Hi Scott
>
> I am adding support for your binary format to mkgmap itself (not the
> splitter).

Excellent! Thank you.

>
> Now I've done enough refactoring of the XML reader to allow me to
> start I have a few questions.
>
> 1. Is there an final name for the format and the jar file?

Not yet. There are too many 'osm binary formats'. I asked for
suggestions on osm-dev yesterday and got 'protobuf binary format'. Do
you have any ideas?

>
> 2. Is there an 'offical' download location for a pre-built jar file?
>

No.

> 3. How do I recognise a file in your format.  Is there a conventional
>    file extension for it?

Those are good questions; I wish someone had asked them earlier.

I have not put in error checking to detect illegal/wrong inputs. I
have fixes in my local tree that throw exceptions on very ill-formed
inputs. They will be out in my next RC.

I have not thought about how one might detect the format. How
important is this?

For now, try feeding the data to the parser and see if there is an
exception? I can add a more robust check with magic at the start of
the file, but I won't have time to implement it for a while. I
designed a concatenable and streamable format. Magic at the start of a
file needs to also be a legal fileblock. I can specify and define such
a magic, as the static serialized contents of a '__Magic' fileblock
but implementing this may take a little while.

I have used *.bin as an extension, but I am open to suggestions.

>    Are the OSMHeader and OSMData blocks required file block types?
>

I am not sure what you are asking, but yes, both are required.

OSMHeader contains HeaderBlock::required_features, which must be
examined to confirm that your implementation can parse the file. You
may also want the contents of HeaderBlock::bbox.

> 4. Does the osmosis conversion from XML to binary keep the order of
>    the elements the same as they were in the XML file?

Yes. Absolutely.

Scott