[mkgmap-dev] Error when running splitter with several input files, could there be keep-complete=single mode?

Tue Aug 31 11:23:37 BST 2021

That step is very easy.

You feed %FID% (if you use fid for first 4 numbers of mapid) and the new
template.args to mkgmap.jar. No need to recreate the .img files that are
okay. The broken ones - all 0byte ones - just delete them. If you wanna run
mkgmap again on the same o5m input files - you join the template.args file
from both runs - I always replace mapname with #notused so the
template.args only contains description and the actual input file. It
throws a warning that input files are missing - but that is allright. Of
course you need to add in your script to delete those input files that were
resplit.

I don't know what your "app" does - but the process is pretty simple. As
long as splitter can handle the several input files with --keep-complete
(and just runs keep-complete on each input file but not on the totality of
input files, plus does not try to join data from two files). And yes as
said - if you want to run splitter on each input file separately, that
creates loads of work - I don't know how to script that efficiently. Oh
yeah - and your script needs to read the highest number of the maps created
- add 1, and run mkgmap again with that number.  But yeah - scripting this
takes some time. But it is understandable that mkgmap cannot call
splitter.jar on it's own....The main thing in this is getting to run
splitter on several input files with keep-complete valid for each tile on
it's own.
But yeah I think osmconvert 1000000.o5m 100001.o5m 100010.o5m
-o=inputfile.o5m
and then calling mkgmap with the higher mapid, and *already_created.img
inputfile.o5m would suffice. So far I cannot see problems (as long as using
keep-complete=true)  - you can even do a third pass. You cannot do a third
pass if you use keep-complete=false overlap=2000
because the overlap would create a huge mess (or better objects ending up
twice in your map)

On Tue, 31 Aug 2021 at 11:33, Eric Internet <ankeric.osm at gmail.com> wrote:

> Also do not understand and on vacation now, so my options for
> understanding are very limited.
>
> But fwiw...
>
> Mkgmap should read files collection from contents files. One o5m file
> missing or 1 file extra (having different FID) is a show stopper. (Missing
> files is not tested, assumption only).
>
> I create 2-6 different maps each overnight from same dataset by simply:
> - Rename all o5m files by FID
> - Rename Reference in two content files by filename (changed by FID).
> This procedure takes seconds to run.
>
> All scripts (power management, splitter, OsmConvert, mkgmap, NSI) are
> executed by shell.run from within a "wrapper application".
> VB exe is: flexible, configurable, unlimited procedures and language
> elements.
>
> App UI checkboxes will select which datasets to download.
> App UI checkboxes will select which maps to be created.
> UI settings are saved in iNI file for Unattended execution.
> App will decide which datasets need to be merged by OsmConvert.
> App will check preconditions.
> App will run preprocess procedures (f.i. Rename by FID).
> App will wait for recent "latest" dataset.
> App will retry a failed download for n-times.
> App will decide what to do if a dataset is missing or outdated.
> App will verify each step to be successful (or not).
> App will install correctly created maps (read log files) into BaseCamp.
> Installer exe files and log files will be renamed by date (history,
> back-up and review options).
> App will run by Windows Scheduled Tasks.
>
> Recreating a specific map (because of style change) is as simple as
> clicking checkboxes.
>
> Time investment for mkgmap and Mapillary: one weekend development.
> Payback period in 2021 is already passed. Not by lead time (although
> Rename is faster than Merge) but most impressive by needed attention time.
> Don't shut down PC and tomorrow my new maps have been created somewhere
> between 06:00 and 09:00 AM or they failed.
>
> A one time problem is resolved by a new procedure for next occurrence.
> One time problems don't exist 😝
>
> AnkEric (Eric)
>
> On Tue, Aug 31, 2021, 10:15 Felix Hartmann <extremecarver at gmail.com>
> wrote:
>
>> yes - exactly what Ticker wrote is my intention. It seems fine using o5m
>> and merging with osmconvert however for right now.... If it's only a single
>> tile no problems, but if its 2-X files - then you can only use
>> keep-complete=false - or instead of using osm.pbf output in splitter o5m,
>> and then merge the failed o5m tiles into a new input2.o5m and process that
>> one again with splitter and lower max-nodes value. Instead if several input
>> files are given and keep-complete=true (or default which is true) - then
>> use keep-complete for each input file only.
>>
>> I do run into another problem there on the huge tiles, will have to check
>> on that later. Seems to be general about maps with very large tiles and not
>> related to splitter.
>>
>> On Tue, 31 Aug 2021 at 09:52, Ticker Berkin <rwb-mkgmap at jagit.co.uk>
>> wrote:
>>
>>> Hi
>>>
>>> My understanding of Felix's problem and a possible enhancement to solve
>>> it:
>>>
>>> Splitting a country... into a number of .osm.pfb files. Almost all of
>>> these are processable by mkgmap to produce tiles, but one or two exceed
>>> some .img limits.
>>>
>>> Rather that resplitting the whole country with a lower --max-nodes and
>>> trying again:
>>>
>>> Run the splitter again using the --split-file=area.list from the
>>> previous run (maybe with the good tiles deleted) and a new option that
>>> forces generation of 2 .osm.pbf files for each area. Probably just
>>> split in half, with some appropriate mapname augmentation. mkgmap
>>> should now be able to produce pairs of tiles to fill in the holes where
>>> the first run failed.
>>>
>>> Ticker
>>>
>>>
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev at lists.mkgmap.org.uk
>>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>>
>>
>>
>> --
>> Felix Hartman - Openmtbmap.org & VeloMap.org
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk
>> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

-- 
Felix Hartman - Openmtbmap.org & VeloMap.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20210831/cdd61af5/attachment-0001.html>