[mkgmap-dev] mkgmap doing excessive writing and converting to disk if used with --gmapi option

Thu Sep 16 13:11:19 BST 2021

I would prefer if I don't even need to link anything, just the concept that
only o5m and osm.pbf input is converted. I can keep the converted contour
lines on the server too, but it would be nicer if I don't even need to have
them for all maps where I have them as separate download.

If that is working all is perfect. I don't see a usecase for an exclude
list.

On Thu, 16 Sep 2021, 11:11 Gerd Petermann <gpetermann_muenchen at hotmail.com>
wrote:

> Hi all,
>
> OK, I think I can add code to detect the img files which were compiled in
> the same call of mkgmap. So, with option --gmapi-minimal
> - Files in this list always need to be written. It should be possible to
> evaluate a parameter list that specifies additional file patterns.
> - Files which don't yet exist in the gmapi folder are also written.
> - other files in the Product subfolder would not be touched
> - the Info.xml will contain all files listed as input
>
> So, either just --gmapi-minimal or --gmapi-minimal[=<patterns for files
> which should overwritten>]
>
> Would that work well for all?
>
> Gerd
>
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von
> Felix Hartmann <extremecarver at gmail.com>
> Gesendet: Montag, 13. September 2021 20:44
> An: Thomas Morgenstern; Development list for mkgmap
> Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to
> disk if used with --gmapi option
>
> .img files are always reused and never rewritten - it's only about the
> gmapi files (a gmapsupp.img has to be generated in one go anyhow - so it is
> not in question here). So I do not see the sense of making it more
> complicated? It would be fine with me too that way - but I think it's
> simpler to just assume all .img files already have the converted files -
> because you could do it when generating those .img files. The problem with
> your approach is - that when you have a long list of tiles that were
> crashing due to too little maxnodes - then regenerating it - will make it
> very complicated. It's much easier to assume that all .img files are
> already converted. All other input not. If you need to convert .img files
> too - then you just use the normal --gmapi option instead.
>
> On Mon, 13 Sept 2021 at 18:58, Thomas Morgenstern <webmaster at img2ms.de
> <mailto:webmaster at img2ms.de>> wrote:
> I suggest the new option should be named reuse=<comma separated list of
> files, wildcards enabled>. exampel  reuse=40000001.img, 40005*.img. If this
> option is used, then mkgmap should not overwrite the listed folders in the
> <name>gmap. Naturally mkgmap should write a new tdb, preview.img, mdr and
> idx. In most cases the reuse folders contains Contourlines or other static
> content.
>
> Thomas
>
> Am 13.09.2021 um 15:38 schrieb Felix Hartmann:
> Hi Gerd, yes exactly. I have a large collection of .img files of
> contourlines. When I realized that I trash my NVME disk very fast if I
> continue the way I did I got into thinking and analysing what happens. It
> was clear that it all comes down to the --gmapi option. I had believed that
> if I run several passes with .o5m and .img files as input - only the .o5m
> input is converted into new .gmapi folders - while the existing ones are
> not overwritten. Then I checked the created date/last modified date
> (because windows has in my eyes a bug that if you replace a file with a
> near identical file - it just shows a new modified date - but keeps the
> original created date - even though it was a full overwrite).
>
> That got me thinking that in order to save writes - I have to find a way
> to not only not recalculate the .img files - but also create a static set
> of .gmapi folders. Those I just mklink into the directory name that will be
> used on the next run of mkgmap.jar - I managed to do this by uncommenting
> this one line. Because I noticed that the symlinked (by mklink) files are
> not rewritten I changed my scripts to move them away and symlink them back.
> Then at the end delete all symlinks - and move the files back (or to the
> location that I will use for compressing). This step is a bit stupid if
> mkgmap could just have a --gmapi-minimal mode in which only those files are
> converted - that are also written out as .img files (if given --tdb-file
> option).
>
> I know that many people keep a static set of contourlines .img files (also
> containing DEM). You just add the show-profiles=1 option in case you
> include contourlines - and leave it out if not. But actually it does not
> matter if the contourlines files contain DEM or not.
> I think the easiest way is the principle - --gmapi-minimal only converts
> those files, it would write out as .img files if --tdb-files option were
> given (or is given). --gmapi on the other hand should convert the all input
> files to .gmapi format. Then mgkmap does not even need to test if those
> files are there or not. This not only saves a lot of writes - but also a
> lot of compile time.
>
> Because essentially if you only provide .img input files (including of
> course the ovm-img for the overview map if you want) you only create a new
> set of mapset files. The exception to this is the toolchain in which when a
> tile failed to compile - you resplit the input files - and parse them again
> with the same arguments so you have input of new o5m files and old .img
> files. But the principle stays the same. If on the rerun of the failed
> tiles now newy split with higher map-id you give --gmapi-minimal and
> tdb-file - only those new tiles are written - while the old .img and old
> ovm.img are supplied to create the correct mapset files. With this
> procedure you don't even need to put a symlink for the contourlines into
> the gmap folder. As the input data to create the contourlines rarely change
> - you can offer the contourlines as a separate download.
> Makes it much easier and faster.
>
> On the map compilation server you then do not even need to have a copy of
> the .gmapi contourlines files. You just need the new input data and the
> static contourlines .img files. Thomas Morgenstern for example had also not
> realized that he is writing the contourlines .gmapi files each time without
> any need. I think many/most providers of garmin maps who offer many
> regions/worldwide coverage put the contourlines separate. It's just a huge
> amount of compile time if you merge the contourlines in o5m format with the
> map data in o5m data each time before running mkgmap. The only actual
> advantage of this being that the contourlines do not overlap roads (as they
> are supplied as transparent .img files in another layer) - AND that when
> people select maps with a single click instead of drag in
> MapInstall/Mapsource they get all maps they think they get (though there is
> a different shading - each layer increases the darkness of the shading for
> selected). And of course that the very old Mapsource still has problems
> correctly showing the contourlines if they are in a separate layer. However
> nearly everyone moved on to Basecamp which is fully compatible with layered
> maps. The advantage of contourlines as separate layer is for the user he
> can switch them on / off independant of the maps and does not need to
> download and transfer them each time. So I think for most the advantages
> heavily outweigh the disadvantages - hence contourlines into a separate
> transparent layer. Same can be done of course for other things like
> buildings or vegetation - though the advantage here is much less on the
> compile side (while worldwide 10m coontourlines are 200GB of data - the
> same extracts are only 15GB in data of buildings - if buildings are shown
> only at resolution 24)
>
> On Mon, 13 Sept 2021 at 14:24, Gerd Petermann <
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>
> wrote:
> Hi Felix,
>
> I have no clue how exactly your scripts work, how you manage to reuse
> *.img files and so on. Also, I want to find a solution that works for all
> users, not just you.
>
> So, I expect that you have one step that compiles frequently changing OSM
> data and other steps which are used to compile static data like
> contourlines or DEM. I don't know if you do the latter for each country /
> continent or once for planet and I don't care as long as it works for you.
> My understanding is that you have a large collection of *.img files at
> some stage and that you run mkgmap multiple times with different
> combinations of those *.img files as input to produce different zip-files
> with gmapi format or gmapsupp format.
> I think that's the normal way to do it, so I try to support that way.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>
> Gesendet: Montag, 13. September 2021 12:28
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to
> disk if used with --gmapi option
>
> I move away the info.xml - and use a different name for the mapset files
> and then just have mkgmap any file that is input in osm.pbf / osm / o5m
> format. Gmapi-minimal should not convert any file that is supplied as .img
> - as it can be assumed that those exist already (if they do not - then
> create them with --gmapi). That is in my opinion the best approach. So
> mkgmap does not even try to convert them.
>
> Afterwards I distribute a gmapi folder that includes all the data - and by
> replacing the info.xml you can enable or disable contourlines. For big
> countries the contourlines would be a separate download anyhow - so the
> user only needs to download the maps (including mapset files) but not
> redownload the contourlines.  Same principle as in offering the
> contourlines as a separate gmapsupp.img file. so you have maps.img
> contourlines10m.img contourlines20m.img buildings.img ....
>
>
> On Mon, 13 Sept 2021 at 13:17, Gerd Petermann <
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com
> ><mailto:gpetermann_muenchen at hotmail.com<mailto:
> gpetermann_muenchen at hotmail.com>>> wrote:
> Hi Felix,
>
> sorry, the Data Deduplication as implemented in Windows Server would not
> help here. It works after data was written.
>
> And yes, files which are not just copies of the *.img are written as
> before. My understanding is that you have different product directories in
> the gmapi folder and that you write protect those files which should be
> kept.
> If you have a script to zip the gmapi directory you have to exclude the
> unwanted folders.
> Didn't try it. Does it make sense?
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>>> im Auftrag von Felix Hartmann <
> extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>
> Gesendet: Montag, 13. September 2021 12:09
> An: Development list for mkgmap
> Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to
> disk if used with --gmapi option
>
> Oh - but data was certainly written - A rename will not show as data
> written in both Task manager on Windows, as well as in the smart data (I'm
> using Windows 10 pro not Windows server however - maybe that functionality
> is limited to windows server?)
>
> On Mon, 13 Sept 2021 at 13:06, Felix Hartmann <extremecarver at gmail.com
> <mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>>>> wrote:
> Not sure if it makes it possible to use read only attribute instead of
> moving and mklink. Maybe yes - because that was not possible before. So it
> then would be set read only - instead of of move & mklink - and at the end
> remove read only instead of moving back.
>
> On Mon, 13 Sept 2021 at 12:59, Felix Hartmann <extremecarver at gmail.com
> <mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>>>> wrote:
> Thanks Gerd,
>
> but that is just removing the warning if it cannot overwrite, correct?
> If it can overwrite the gmap file it will still overwrite it as I see.. So
> in essence just a bit more intelligent then my disabling that line.
>
> I think it should not overwrite at all if present and --x-gmapi-minimal
> (then you don't have to move away the files and link them back to the
> original folder).
>
> On Mon, 13 Sept 2021 at 10:43, Gerd Petermann <
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com
> ><mailto:gpetermann_muenchen at hotmail.com<mailto:
> gpetermann_muenchen at hotmail.com>><mailto:gpetermann_muenchen at hotmail.com
> <mailto:gpetermann_muenchen at hotmail.com><mailto:
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com>>>>
> wrote:
> Hi all,
>
> attached is a quick patch that implements experimaltal option
> --x-gmapi-minimal.
>
> If used instead of --gmapi mkgmap will not fail if a write-protected
> output file exists in the gmapi output folder and mkgmap copies data from
> *.img. It should still crash when other files like Info.xml are written.
>
> BTW: no conversion is done when those files are written. I think mkgmap
> simply copies data from sub files in *.img into single files. So, the same
> sequence of Bytes exists two or more times on the Computer. Windows Server
> seems to support automatic data deduplication (1). I am sure Linux offers
> similar support. No idea how effective or reliable it is, but it might be
> worth trying.
>
> Gerd
> (1)
>
> https://docs.microsoft.com/en-us/windows-server/storage/data-deduplication/install-enable
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>>>> im Auftrag von Carlos Dávila <
> carlos at alternativaslibres.org<mailto:carlos at alternativaslibres.org
> ><mailto:carlos at alternativaslibres.org<mailto:
> carlos at alternativaslibres.org>><mailto:carlos at alternativaslibres.org
> <mailto:carlos at alternativaslibres.org><mailto:
> carlos at alternativaslibres.org<mailto:carlos at alternativaslibres.org>>>>
> Gesendet: Sonntag, 12. September 2021 22:45
> An: mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>>
> Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting to
> disk if used with --gmapi option
>
> I think it's a good idea if mkgmap checks the required files are present.
>
> El 12/9/21 a las 21:16, Gerd Petermann escribió:
> > Hi Felix,
> >
> > so you just want a new option so that mkgmap doesn't fail if it cannot
> overwrite (write-protected) files in the output directory, right? No need
> to verify the content of the exiting file(s)?
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>>>> im Auftrag von Felix Hartmann <
> extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>>
> > Gesendet: Sonntag, 12. September 2021 19:10
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting
> to disk if used with --gmapi option
> >
> > And well - it is burning SSD for the contourlines currently even if not
> calling multiple times. 130GB minimum per worldwide map update for all
> geofabrik extracts at 20m equidistance. 260GB at 10m. With calling multiple
> times and 10m and 20m I had about 2TB of useless writes per weekly map
> update. I've got rid of all of them with my uncommenting the line, plus
> saved about 500GB of writes by now doing all the gmapsupp.img and gmap
> stuff in Ramdisk So now instead of 4TB of disk writes per map update I'm
> down to 1TB.. (and yeah the resplitting of tiles added another 5TB of
> writes - but that could have been fixed easily by changing order of
> commands too).
> >
> > On Sun, 12 Sept 2021 at 20:04, Felix Hartmann <extremecarver at gmail.com
> <mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>>>>> wrote:
> > because you need to do it if you want to show contourlines. Now
> compiling the worldwide contourlines each time again - would burn SSD as
> well - besides taking longer than just compiling the maps. So everyone who
> offers maps for many countries as download puts the contourlines separate.
> If you want to offer the choice of showing contourlines or not - mkmgap
> will write the gmap contourlines once not needed, and once the maps not
> needed. If you don't want to offer that option - it's only the contourlines
> being written without need. Contourlines for all geofabrik extracts 20m
> equidistance are about 130GB, 10m would be 260GB.
> > If you offer 10m and 20m it will be even more not needed writes.
> >
> > The only solution is to go for a Ramdisk instead - However you likely
> will need 128GB of RAM to do that - because for Asia or Europe you need a
> 64GB Ramdisk. Same for North America extract (but few people offer that and
> just have Canada and the US divided into the 4-5 zones). For other maps you
> will get by with a 15-20GB Ramdisk (which I have resorted to now for all
> but the windows installers because I don't want to let the server wait for
> ages for the single thread NSIS wrapping up the data and instead start the
> next country in parallel). And yes going RAMDISK already using my patch and
> symlinking those files so mkgmap cannot overwrite them (would still be
> faster if mkgmap didn't try in first place I think). If you want to write
> out the contourlines as well besides the additional time - for Asia
> continent you may then go for a 90GB Ramdisk minimum if offering windows
> and gmap installers. In this case gmapsupp.img donwnloads aren't possible
> anyhow due to the huge data amounts. For sm
> >   aller countries I have them too....
> >
> > I coded around this now by using symlinks - but yeah that will be quite
> a lot of work and is prone to break - while I guess it's only 10 lines or
> so to add to mkgmap code to have a mode that does not write them out if
> they are present - or if you tell on commandline they are present already.
> For .img files they aren't overwritten either...
> >
> > On Sun, 12 Sept 2021 at 18:40, Gerd Petermann <
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com
> ><mailto:gpetermann_muenchen at hotmail.com<mailto:
> gpetermann_muenchen at hotmail.com>><mailto:gpetermann_muenchen at hotmail.com
> <mailto:gpetermann_muenchen at hotmail.com><mailto:
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com
> >>><mailto:gpetermann_muenchen at hotmail.com<mailto:
> gpetermann_muenchen at hotmail.com><mailto:gpetermann_muenchen at hotmail.com
> <mailto:gpetermann_muenchen at hotmail.com>><mailto:
> gpetermann_muenchen at hotmail.com<mailto:gpetermann_muenchen at hotmail.com
> ><mailto:gpetermann_muenchen at hotmail.com<mailto:
> gpetermann_muenchen at hotmail.com>>>>> wrote:
> > Hi Felix,
> >
> > did not read all the posts in detail. I understand that mkgmap is
> neither burning SSDs nor doing any excessive writing unless you call it
> multiple times for the same input files. So the question is: Why do you do
> that?
> >
> > Gerd
> >
> > ________________________________________
> > Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>>><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk><mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:
> mkgmap-dev-bounces at lists.mkgmap.org.uk>>>>> im Auftrag von Felix Hartmann
> <extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com><mailto:
> extremecarver at gmail.com<mailto:extremecarver at gmail.com>>>>>
> > Gesendet: Freitag, 10. September 2021 00:02
> > An: Development list for mkgmap
> > Betreff: Re: [mkgmap-dev] mkgmap doing excessive writing and converting
> to disk if used with --gmapi option
> >
> > Well I think the .tmp files are just building up - and the renamed. So
> they are not causing any actual excessive write.
> > As for the gmap - it would be cool if there is a mode to not write them.
> >
> > Actually it would be great if mkgmap could write all in one go. Because
> the thing that takes so much time - is the address search - and that is
> always the same. The differences are tiny (just because MapInstall is
> crashing when files are missing) you need to compile them separately.
> >
> > but maybe there could be a mode where mkgamp writes all in one go.
> > So family-name / family-name1 / family-name2
> > description / description1 / description2
> > input input1 input2
> > family-name..
> > show-profiles
> > overview-mapname
> > product-id
> > (and maybe I missed some options are those that would need to be given
> for each set of input tiles. And then just an option where you tell mkgmap
> files starting with which first 4 numbers are relevant for address search.
> No need to analyze if those other supplied .img (e.g. buildings or
> contourlines) need to be added to address search.). I know coded around the
> problem of the gmap files causing excessive writes. But yeah that is
> actually really complicated be it on windows or linux...
> >
> > On Wed, 8 Sept 2021 at 17:07, Felix Hartmann <extremecarver at gmail.com
> <mailto:extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>>>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com><mailto:extremecarver at gmail.com<mailto:
> extremecarver at gmail.com>>>>>> wrote:
> > Well I could give it 20 GB ram disk, maybe 32 but then I need to render
> on less than 12 processes 64GB ram available). But that is not enough for
> Asia continent map, and I guess super tight for Europe...
> >
> > Mkgmap could definitely keep those .tmp files in memory. But the
> important bit is the gmap files not needeed to be written.... Also would
> save quite some CPU time.
> >
> > On Wed, 8 Sep 2021, 14:31 ael <witwall3 at disroot.org<mailto:
> witwall3 at disroot.org><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org>><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org>>><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org>><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org>>>><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org>><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org>>><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org>><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org><mailto:witwall3 at disroot.org<mailto:
> witwall3 at disroot.org>>>>>> wrote:
> > On Wed, Sep 08, 2021 at 02:10:52PM +0300, Felix Hartmann wrote:
> >> Yes on an nvme disk you barely notice the conversion - it's really
> quick.
> >> BUT it is not needed if you have the files and even more - it burns your
> >> NVME SSD disk.
> > +1. I always use an old spinning rust disk when using mkgmap to save ssd
> > write cycles, even without contours and such. It seems to be profligate
> > in its use of disk cycles. I did try using RAM disk, but even with 16GB
> > on a laptop, that was soon exhausted.
> >
> > ael
> >
> >
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> >>><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk>><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>>>><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> >><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk>>><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>>>>
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >
> >
> > --
> > Felix Hartman - Openmtbmap.org & VeloMap.org
> >
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> >>><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk>><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>>>>
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >
> >
> > --
> > Felix Hartman - Openmtbmap.org & VeloMap.org
> >
> >
> >
> > --
> > Felix Hartman - Openmtbmap.org & VeloMap.org
> >
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>>
> > https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>><mailto:mkgmap-dev at lists.mkgmap.org.uk
> <mailto:mkgmap-dev at lists.mkgmap.org.uk><mailto:
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>>>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk
> ><mailto:mkgmap-dev at lists.mkgmap.org.uk<mailto:
> mkgmap-dev at lists.mkgmap.org.uk>>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
>
>
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> https://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20210916/327d1ca9/attachment-0001.html>