<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
if that doesn't seriously (more than 30-40%) slow down the splitter,
I assume it would be much better...<br>
<div class="moz-cite-prefix">On 30.04.2014 14:06, Gerd Petermann
wrote:<br>
</div>
<blockquote cite="mid:DUB112-W402302662FD5B679DF5CAB9E410@phx.gbl"
type="cite">
<div dir="ltr">
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style>
<div dir="ltr">Hi,<br>
<br>
if I got that right the number of nodes is not <br>
highly correlated to the img size, so the max-nodes<br>
value is not a good estimate.<br>
<br>
I assume the reason is that nodes which belong to<br>
roads produce a lot more bytes<br>
in the img file compared to nodes which are parts<br>
of shapes or other non-routable ways, not talking<br>
about nodes which are simply ignored by the style.<br>
<br>
So, a possible solution in splitter could be to parse <br>
the ways before reading the nodes and save all nodeids<br>
which belong to ways with highway=*.<br>
If these nodes are refered by more than one way with highway=*<br>
we assume that they will be routing nodes.<br>
These special nodes could be counted e.g. 10 times to <br>
give a better estimate.<br>
<br>
Gerd<br>
<br>
<div>> Date: Wed, 30 Apr 2014 13:36:19 +0200<br>
> From: <a class="moz-txt-link-abbreviated" href="mailto:osm@na1400.info">osm@na1400.info</a><br>
> To: <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@lists.mkgmap.org.uk">mkgmap-dev@lists.mkgmap.org.uk</a><br>
> Subject: Re: [mkgmap-dev] mkgmap ToDo list<br>
> <br>
> Multithreading the tile rendering for a single map is
indeed a bit <br>
> difficult and I gave it up because you need to keep
track which image <br>
> id's are already in use. Since I provide multiple maps
the work-around <br>
> is running a few scripts parallel, which is also a
crude form of <br>
> multithreading.<br>
> <br>
> The script language is PHP and it doesn't run on
Windows without some <br>
> changes ('/' vs '\' in paths, 'rm -rf', that sort of
stuff). Never tried it.<br>
> <br>
> To get a better optimum in file size, using the process
I described <br>
> earlier, you could start off with a huge --max-nodes
setting and then <br>
> 'search' for the highest --max-nodes that works for
each specific area.<br>
> <br>
> On 30/04/2014 11:49, Felix Hartmann wrote:<br>
> > I would love if there was a possibility that you
pass the used max-nodes<br>
> > value to mkgmap.<br>
> > When mkgmap is compiling the maps, then after the
.img is created it<br>
> > should check<br>
> > a) did it crash due to too many max-nodes<br>
> > b) for me not important - but for others with very
old GPS, etrex 10,<br>
> > ---> is tile bigger than X (usually 8) MB.<br>
> ><br>
> > if a) or b) true, then pass the file back to
splitter and split with 60%<br>
> > of maxnodes - and compile the resulting .img files
again. Should it fail<br>
> > again, use 40%, again 25%... Sometimes there are
awful tiles, that need<br>
> > supersmall max-nodes till they compile, however
lately (last 1-2 years)<br>
> > I never encountered them anymore. I think that
happened rather due to a<br>
> > but in splitter/mgkmap that is fixed now.<br>
> ><br>
> > okay, you could also do this with a script, but it
gets rather<br>
> > complicated to multithread it (you need to wait
till mgkmap finished<br>
> > compiling all .img files - and run mkgmap first
without address index to<br>
> > save time) and do some clever routines on making
sure that the map id<br>
> > (e.g. 6340????.img) stay correct. Even more
complicated to have<br>
> > consequent map id...<br>
> ><br>
> > For europe with a fixed max-node I get tiles from
1.9MB up to 18MB.<br>
> > That's factor 9 - so it's huge...<br>
> > If I could narrows that down easily to 8-18MB -
without getting tiles<br>
> > crashing due to too high max-nodes values, that
would be sweet.<br>
> ><br>
> ><br>
> ><br>
> > As for the scripts - would they run on windows
too? - What programming<br>
> > language are they in?<br>
> ><br>
> ><br>
> > On 29.04.2014 21:39, <a class="moz-txt-link-abbreviated" href="mailto:osm@na1400.info">osm@na1400.info</a> wrote:<br>
> >> Oh, and ofcourse anyone interested can get my
scripts, send an email.<br>
> >> They'll be on Github someday anyway.<br>
> >><br>
> >><br>
> >> On 2014-04-29 20:37, Gerd Petermann wrote:<br>
> >>> Hi Lambertus,<br>
> >>><br>
> >>> okay, if I got that right you finally get
*.img files with a size<br>
> >>> near (but below) 8MB, so maybe Henning can
use that script, too.<br>
> >>><br>
> >>> If you do that for e.g. Germany, how small
is tpically the smallest<br>
> >>> *.img file ?<br>
> >>> Is it probably near 4 MB?<br>
> >>><br>
> >>> Gerd<br>
> >>><br>
> >>>> To: <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@lists.mkgmap.org.uk">mkgmap-dev@lists.mkgmap.org.uk</a><br>
> >>>> Date: Tue, 29 Apr 2014 20:30:27 +0200<br>
> >>>> From: <a class="moz-txt-link-abbreviated" href="mailto:osm@na1400.info">osm@na1400.info</a><br>
> >>>> Subject: Re: [mkgmap-dev] mkgmap ToDo
list<br>
> >>>><br>
> >>>> These are the direct results from
Splitter. The format is o5m, both<br>
> >>>> input as output. Splitter version is:
r321.<br>
> >>>><br>
> >>>> For this test I split the original
source with --max-nodes=8000000.<br>
> >>> Then<br>
> >>>> I render the initial tiles, when the
result is larger than 8MB it's<br>
> >>>> subsplit again with
--max-nodes=(8000000-(attempt*100000)). The<br>
> >>> initial<br>
> >>>> source files are ~70MB (o5m) and after
several subsplits the two<br>
> >>> *.img<br>
> >>>> are < 8MB. During this process
--max-nodes has been reduced to e.g.<br>
> >>>> 7500000 and the source file is split
up in two o5m files of about<br>
> >>> 37MB.<br>
> >>>><br>
> >>>> I can upload an example source file
and it's two subsplit siblings<br>
> >>> if<br>
> >>>> you want.<br>
> >>>><br>
> >>>><br>
> >>>> On 2014-04-29 19:38, GerdP wrote:<br>
> >>>> > Hi Lambertus,<br>
> >>>> ><br>
> >>>> > that's interesting. Are these the
img file sizes or the osm file<br>
> >>> sizes?<br>
> >>>> ><br>
> >>>> > Gerd<br>
> >>>> ><br>
> >>>> ><br>
> >>>> > Lambertus wrote<br>
> >>>> >> Unfortunately I cannot
confirm that. Below is a bit of logging<br>
> >>> from my<br>
> >>>> >> script:<br>
> >>>> >> Original: 97000020
(70551453), New: 0 (35684445), New: 1<br>
> >>> (36852845)<br>
> >>>> >> Original: 97000001
(74621042), New: 0 (37522992), New: 1<br>
> >>> (37222739)<br>
> >>>> >> Original: 97000002
(73391358), New: 0 (37679505), New: 1<br>
> >>> (38098627)<br>
> >>>> >> Original: 97000003
(77862567), New: 0 (39075311), New: 1<br>
> >>> (39261197)<br>
> >>>> >><br>
> >>>> >> The original files above
contain contour data, the filesize is<br>
> >>> between<br>
> >>>> >> brackets. As you can see both
resulting file are approximately<br>
> >>> the<br>
> >>>> >> same<br>
> >>>> >> size.<br>
> >>>> >><br>
> >>>> >> On 2014-04-29 15:39, Gerd
Petermann wrote:<br>
> >>>> >>> Hi Lambertus,<br>
> >>>> >>><br>
> >>>> >>> and I guess that even
after this optimization you will<br>
> >>>> >>> see a factor 3 or higher
between the largest tile and the<br>
> >>> smallest.<br>
> >>>> >>> Can you confirm that?<br>
> >>>> >>><br>
> >>>> >>> Gerd<br>
> >>>> >>><br>
> >>>> >>>> Date: Tue, 29 Apr
2014 15:32:38 +0200<br>
> >>>> >>>> From:<br>
> >>>> ><br>
> >>>> >> osm@<br>
> >>>> ><br>
> >>>> >>>> To:<br>
> >>>> ><br>
> >>>> >> <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@.org">mkgmap-dev@.org</a><br>
> >>>> ><br>
> >>>> >>>> Subject: Re:
[mkgmap-dev] mkgmap ToDo list<br>
> >>>> >>>><br>
> >>>> >>>> Num-tiles=x would
indeed be better for this specific need.<br>
> >>>> >>>><br>
> >>>> >>>> It is my experience
that it regularly takes multiple calls to<br>
> >>>> >>> Splitter<br>
> >>>> >>>> to get 2+ sub-tiles
when you reduce the max-nodes by 100k for<br>
> >>> each<br>
> >>>> >>>> sub-split attempt.
This is what I currently do to get an<br>
> >>> optimum in<br>
> >>>> >>>> tile-size vs total
number of tiles.<br>
> >>>> >>>><br>
> >>>> >>>><br>
> >>>> >>>><br>
> >>>> >>>> On 29/04/2014 15:09,
Gerd Petermann wrote:<br>
> >>>> >>>> > Hi Lambertus,<br>
> >>>> >>>> ><br>
> >>>> >>>> > that sounds like
a possible change in splitter:<br>
> >>>> >>>> > Instead of
specifying max-nodes you may specify --num-tiles=x<br>
> >>>> >>>> > and splitter
will try to find a split that produces excactly<br>
> >>> x<br>
> >>>> >>> tiles<br>
> >>>> >>>> > which are not
too narrow and have a node number which is not<br>
> >>>> >>>> > too far from the
average (but still aligned to a multiple of<br>
> >>> map<br>
> >>>> >>> units<br>
> >>>> >>>> > as now).<br>
> >>>> >>>> > So, for your
script that means you don't have to find the<br>
> >>>> >>> max-nodes<br>
> >>>> >>>> > value.<br>
> >>>> >>>> ><br>
> >>>> >>>> > I'll think about
this again...<br>
> >>>> >>>> ><br>
> >>>> >>>> > Gerd<br>
> >>>> >>>> ><br>
> >>>> >>>> > > Date: Tue,
29 Apr 2014 14:59:36 +0200<br>
> >>>> >>>> > > From:<br>
> >>>> ><br>
> >>>> >> osm@<br>
> >>>> ><br>
> >>>> >>>> > > To:<br>
> >>>> ><br>
> >>>> >> <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@.org">mkgmap-dev@.org</a><br>
> >>>> ><br>
> >>>> >>>> > > Subject:
Re: [mkgmap-dev] mkgmap ToDo list<br>
> >>>> >>>> > ><br>
> >>>> >>>> > > While this
possibly can be solved in Splitter or Mkgmap, it<br>
> >>>> >>> could also<br>
> >>>> >>>> > > be solved
by your build-script when you add a maximum tile<br>
> >>> size<br>
> >>>> >>> check<br>
> >>>> >>>> > > and
re-split (with a lower number of max-nodes) until you<br>
> >>> get<br>
> >>>> >>> two or<br>
> >>>> >>>> > > more
sub-tiles. Granted, this adds complexity to the script<br>
> >>> but<br>
> >>>> >>> it works<br>
> >>>> >>>> > > well for
me.<br>
> >>>> >>>> > ><br>
> >>>> >>>> > > On
25/04/2014 21:54, Henning Scholland wrote:<br>
> >>>> >>>> > > > Hi
Gerd,<br>
> >>>> >>>> > > ><br>
> >>>> >>>> > > > I
would like to have img-tiles which have globally nearly<br>
> >>> the<br>
> >>>> >>> same<br>
> >>>> >>>> > > >
filesize, so that they use the space of devices like<br>
> >>> eTrex 10.<br>
> >>>> >>>> > > ><br>
> >>>> >>>> > > > With
my actual map I use globally the same value for<br>
> >>>> >>> max-nodes. But the<br>
> >>>> >>>> > > > size
of the img-tiles differ more then factor 2. Eg. a<br>
> >>> tile in<br>
> >>>> >>> Germany<br>
> >>>> >>>> > > > is
between 2 and 5 mb where a tile in China is about 10<br>
> >>> mb. If<br>
> >>>> >>> I remove<br>
> >>>> >>>> > > >
details, this difference will increase, because in<br>
> >>> Germany<br>
> >>>> >>> more objects<br>
> >>>> >>>> > > > will
be removed from the img-tile then in China.<br>
> >>>> >>>> > > ><br>
> >>>> >>>> > > >
Henning<br>
> >>>> >>>> > > ><br>
> >>>> >>>> > > >
_______________________________________________<br>
> >>>> >>>> > > >
mkgmap-dev mailing list<br>
> >>>> >>>> > > ><br>
> >>>> ><br>
> >>>> >> <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@.org">mkgmap-dev@.org</a><br>
> >>>> ><br>
> >>>> >>>> > > >
<a class="moz-txt-link-freetext" href="http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev">http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
> >>>> >>>> > ><br>
> >>>> >>>> > >
_______________________________________________<br>
> >>>> >>>> > > mkgmap-dev
mailing list<br>
> >>>> >>>> > ><br>
> >>>> ><br>
> >>>> >> <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@.org">mkgmap-dev@.org</a><br>
> >>>> ><br>
> >>>> >>>> > >
<a class="moz-txt-link-freetext" href="http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev">http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
> >>>> >>>> ><br>
> >>>> >>>> ><br>
> >>>> >>>> >
_______________________________________________<br>
> >>>> >>>> > mkgmap-dev
mailing list<br>
> >>>> >>>> ><br>
> >>>> ><br>
> >>>> >> <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@.org">mkgmap-dev@.org</a><br>
> >>>> ><br>
> >>>> >>>> >
<a class="moz-txt-link-freetext" href="http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev">http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
> >>>> >>>> ><br>
> >>>> >>>><br>
> >>>> >>>>
_______________________________________________<br>
> >>>> >>>> mkgmap-dev mailing
list<br>
> >>>> >>>><br>
> >>>> ><br>
> >>>> >> <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@.org">mkgmap-dev@.org</a><br>
> >>>> ><br>
> >>>> >>>>
<a class="moz-txt-link-freetext" href="http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev">http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
> >>>> >>><br>
> >>>> >>>
_______________________________________________<br>
> >>>> >>> mkgmap-dev mailing list<br>
> >>>> >>><br>
> >>>> ><br>
> >>>> >> <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@.org">mkgmap-dev@.org</a><br>
> >>>> ><br>
> >>>> >>>
<a class="moz-txt-link-freetext" href="http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev">http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
> >>>> >>
_______________________________________________<br>
> >>>> >> mkgmap-dev mailing list<br>
> >>>> ><br>
> >>>> >> <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@.org">mkgmap-dev@.org</a><br>
> >>>> ><br>
> >>>> >>
<a class="moz-txt-link-freetext" href="http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev">http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
> >>>> ><br>
> >>>> ><br>
> >>>> ><br>
> >>>> ><br>
> >>>> ><br>
> >>>> > --<br>
> >>>> > View this message in context:<br>
> >>>> ><br>
> >>>
<a class="moz-txt-link-freetext" href="http://gis.19327.n5.nabble.com/mkgmap-ToDo-list-tp5803388p5804588.html">http://gis.19327.n5.nabble.com/mkgmap-ToDo-list-tp5803388p5804588.html</a><br>
> >>>> > Sent from the Mkgmap Development
mailing list archive at<br>
> >>> Nabble.com.<br>
> >>>> >
_______________________________________________<br>
> >>>> > mkgmap-dev mailing list<br>
> >>>> > <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@lists.mkgmap.org.uk">mkgmap-dev@lists.mkgmap.org.uk</a><br>
> >>>> >
<a class="moz-txt-link-freetext" href="http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev">http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
> >>>>
_______________________________________________<br>
> >>>> mkgmap-dev mailing list<br>
> >>>> <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@lists.mkgmap.org.uk">mkgmap-dev@lists.mkgmap.org.uk</a><br>
> >>>>
<a class="moz-txt-link-freetext" href="http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev">http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
> >>><br>
> >>>
_______________________________________________<br>
> >>> mkgmap-dev mailing list<br>
> >>> <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@lists.mkgmap.org.uk">mkgmap-dev@lists.mkgmap.org.uk</a><br>
> >>>
<a class="moz-txt-link-freetext" href="http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev">http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
> >>
_______________________________________________<br>
> >> mkgmap-dev mailing list<br>
> >> <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@lists.mkgmap.org.uk">mkgmap-dev@lists.mkgmap.org.uk</a><br>
> >>
<a class="moz-txt-link-freetext" href="http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev">http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
> ><br>
> <br>
> _______________________________________________<br>
> mkgmap-dev mailing list<br>
> <a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@lists.mkgmap.org.uk">mkgmap-dev@lists.mkgmap.org.uk</a><br>
> <a class="moz-txt-link-freetext" href="http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev">http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a><br>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
mkgmap-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:mkgmap-dev@lists.mkgmap.org.uk">mkgmap-dev@lists.mkgmap.org.uk</a>
<a class="moz-txt-link-freetext" href="http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev">http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev</a></pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
keep on biking and discovering new trails
Felix
openmtbmap.org & <a class="moz-txt-link-abbreviated" href="http://www.velomap.org">www.velomap.org</a></pre>
</body>
</html>