[mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"

Wed Dec 23 06:29:06 GMT 2020

Hi Franco,

just noticed that I used o5m for output in both tests, so my numbers for pbf are probably a bit to small. Anyway, I think this just proves that it was a good idea to implement the o5m support ;)

Gerd

________________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Franco Bez <franco.bez at web.de>
Gesendet: Dienstag, 22. Dezember 2020 20:41
An: mkgmap-dev at lists.mkgmap.org.uk
Betreff: Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"

Hi Gerd,

thanks for your tests.

On your machine it's still more than double, on my machine almost
tripple the time for pbf compared to o5m.

Thanks for the hint with "ways missing nodes", I'll check this.

Ciao,

Franco

Am 22.12.20 um 10:51 schrieb Gerd Petermann:
> Hi Franco,
>
> the differences in my numbers are not that drastical. I tried with Germany  from geofabrik and got 5 minutes 21 seconds.
> for the o5m format and 12 min for pbf. Maybe the pbf reader could be improved to use multiple threads, but that's not my strength.
>
> BTW: Your logs show many warnings like "Sorry, way 4216363 is missing 29 node(s)." . Maybe you should check your tool chain if these ways are relavant for you.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Franco Bez <franco.bez at web.de>
> Gesendet: Samstag, 19. Dezember 2020 13:37
> An: mkgmap-dev at lists.mkgmap.org.uk
> Betreff: Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to "o5m"
>
> |Hi Gerd,|
>
> |here are the two log files.
> |
>
> |http://files.mkgmap.org.uk/download/491/logs.tgz|
>
> |Ciao,|
>
> |Franco
> |
>
> Am 19.12.20 um 12:09 schrieb Gerd Petermann:
>> Hi Franco,
>>
>> OK, I guess 2048 areas should be enough and memory is for sure enough. The major reason that I added o5m support to splitter was that this format was much faster to read, esp. when the file is read multiple times some passes only need relations or only ways. The pbf format didn't support this as well, but I thought that I also improved handling of that format.
>> If you like you can zip the two logs and upload them to http://files.mkgmap.org.uk .
>>
>> I'll do some tests on my own machine again.
>>
>> Gerd
>>
>> ________________________________________
>> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Franco Bez <franco.bez at web.de>
>> Gesendet: Samstag, 19. Dezember 2020 11:45
>> An: mkgmap-dev at lists.mkgmap.org.uk
>> Betreff: Re: [mkgmap-dev] splitter performance with "osm.pbf" compared to       "o5m"
>>
>> Hi Gerd,
>>
>> thanks for the fast reply.
>>
>> I tried "java -ea -Xmx16G -Xms16G" but this doen't make any difference
>> on my machine, AMD Ryzen 7 3700X 32GB Ram 2 SSDs 1TB each
>>
>> see the log extracts below.
>>
>> Ciao,
>>
>>   Franco
>>
>

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev