[mkgmap-dev] Option to output polygons in size order

Sun Jul 31 05:01:24 BST 2016

Hi Felix,

sorry, I cannot reproduce that result. I used your input file (13998319 bytes) and created a style as you described,

and used your options.

As expected I see a part of the Chiemsee and three empty areas in it (at zoom level 3 and below)

So I guess you are looking at out-aged data or the input file changed in the mean time.

BTW: The tile boundary of the input file goes straight through the Chiemsee (relation 32246),

that's why I only see a part of it.

So, please double check your process and if that doens't help:

The only other input file is the bounds file. I cannot think of a good reason why that would

change the result unless it is somehow corrupted.

Gerd

________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Felix Hartmann <extremecarver at gmail.com>
Gesendet: Samstag, 30. Juli 2016 23:37:03
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Option to output polygons in size order

I'm dumbstruck - I tried everything and reduced the full style-file to 1 line. Still it's not working. Here is how I create the map:

osmconvert.exe --drop-version e:\openmtbmap\osmpbf_geofabrik\bayern-latest.osm.pbf -o=C:\openmtbmap\osmpbf_geofabrik\bayern.o5m
java -Xms4000m -Xmx12400m -jar C:\openmtbmap\splitter.jar "--precomp-sea=C:\openmtbmap\maps\sea.zip" --max-nodes=1400000 --max-threads=8 --output=pbf "--keep-complete" --max-areas=1524  --geonames-file=cities5000 --description=bayern --mapid=65260000 C:\openmtbmap\osmpbf_geofabrik\bayern.o5m

java -jar -XX:StringTableSize=100003 -Xms6000M -Xmx13300M C:\openmtbmap\mkgmap.jar --max-jobs=8 --code-page=1252 "--style-file=C:\openmtbmap\openmtbmap_style" --nsis --index --levels="0:24, 1:23, 2:22, 3:21, 4:20, 5:19, 6:18" --overview-levels="7:17, 8:16, 9:15, 10:14, 11:13, 12:12" --add-pois-to-areas --reduce-point-density=3.4 --reduce-point-density-polygon
=6 --housenumbers --link-pois-to-ways --ignore-turn-restrictions --polygon-size-limits="24:16, 23:14, 22:12, 21:11, 20:10, 19:9, 18:8, 17:7, 16:6, 15:5, 14:4, 13:3, 12:2, 11:0, 10:0" --description=openmtbmap_dby --show-profiles=1  --location-autofill=bounds,is_in,nearest  --bounds=C:\openmtbmap\maps\bounds.zip --route --country-abbr=dby --country-name=bayern --mapname=65260000 --family-id=6526 --p
roduct-id=1 --series-name=openmtbmap_bayern_30.07.2016 --family-name=mtbmap_dby_30.07.2016 --tdbfile --overview-mapname=mapsetc --keep-going --area-name="bayern_30.07.2016_openmtbmap.org<http://bayern_30.07.2016_openmtbmap.org>" e:\openmtbmap\maps\65260031.osm.pbf

I have uploaded 65260031.osm.pbf
here: https://openmtbmap.org/65260031.osm.pbf

My style-file is reduced to:
Polygons:
natural=water {set 20resolution=yes} [0x3c resolution 18-21]
Lines and Points empty for quicker bug checking.

mkgmap.jar up to date downloaded from the website not self compiled. Splitter self compiled but stock without patches.
Ends up with the lake with the island cut out - but the island put in as 0x3c too!

bayer.osm.pbf downloaded three days ago from Geofabrik. But I'm sure the data is fine - why else should it be cut out otherwise?
Where can this bug come from?

On 29 July 2016 at 13:55, Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com>> wrote:
well I have no other rules with 0x3c and resolution 18-21 in my style. So I don't know how else it can happen. And yes - mapdata is up to date from geofabrik as of yesterday. Strangely the europe map is fine, Germany and Alps are not fine - so it's kinda random.
Oh yeah - the name for the water is not Chieemsee but Herreninsel.

However for island the only rule that should fit is:
place=island & layer!=*     & 20resolution!=yes & name:en!="North Island" & name:en!="South Island" & natural!=coastline & area_size() < 2000000 [0x0d resolution 20]

but it clearly puts 0x3c and starts at resolution 18. The actual island does not end up in the map - as I guess 20resolution=yes is set further up in the style.

On 29 July 2016 at 13:40, Gerd Petermann <GPetermann_muenchen at hotmail.com<mailto:GPetermann_muenchen at hotmail.com>> wrote:

Hi Felix,

I have no idea why the Herreninsel http://www.openstreetmap.org/way/4605746

should  trigger any of these rules. Are you sure that your input file contains the current

OSM data?

To make sure I suggest to download the relation http://www.openstreetmap.org/relation/32246

in JOSM, safe it to a file and use that as input for mkgmap.

Next, you can enable logging so that you see what happens in mkgmap.

Gerd

________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com>>
Gesendet: Freitag, 29. Juli 2016 13:12:05

An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Option to output polygons in size order

I guess the area_size() function kinda checks if it's big enough - and in turn does not check anymore if it is part of the multipolygon or not. Else I really cannot explain the flooding. I can change all lines which have 0x3c to a unique number - and check which line it is. Will do so tomorrow. However there is clearly some bug.

On 29 July 2016 at 12:05, Gerd Petermann <GPetermann_muenchen at hotmail.com<mailto:GPetermann_muenchen at hotmail.com>> wrote:

Hi Felix,

I suggest to use echotags to find out if a rule fires or not. What kind of error

do you expect with the area_size() function?

Gerd

________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com>>
Gesendet: Freitag, 29. Juli 2016 11:45:47

An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Option to output polygons in size order

Nope - it's not the splitter - it's fully within one tile. I guess there is a bug in the style parser together with area_size() filter and continue command?

I actually looked at it with gpsmapedit - and I see the island is cut out - but additionally to the cut out water is put into it. The only lines in my polygons file that can be related to it are as follows:
(note the flooded island also exists at resolution 18 as 0x3c - so all other lines are only relevant for the key 20resolution!=yes).
I put those lines which should not fire in grey. Chiemsee is natural=water & water=lake.

( natural=forest | landuse=wood | natural=forrest | landuse=forrest ) {set landuse=forest}
( landuse=forest | natural=wood ) {set 20resolution=yes} [0x50 resolution 18-20 continue with_actions]
( natural=glacier | landuse=glacier ) & 20resolution!=yes {set 20resolution=yes} [0x4d resolution 18-21 continue with_actions]

( natural=lake | ( natural=water & water=lake)) & 20resolution!=yes {set 20resolution=yes} [0x3c resolution 18-21 continue with_actions]
natural=water & water=oxbow     & 20resolution!=yes {set 20resolution=yes} [0x3c resolution 20-21 continue with_actions]
natural=water & ( water=cove | water=lagoon )    & 20resolution!=yes {set 20resolution=yes} [0x3c resolution 21-21 continue with_actions]
natural=water & water=* & water!=lake & water!=reservoir & water!=canal & water!=river & water!=yes & water!=Cove & water!=bay & water!=Lake  & area_size() < 1000         & 20resolution!=yes {set 20resolution=yes}
natural=water & water=reflecting_pool     & 20resolution!=yes {set 20resolution=yes}
natural=water & water=lock     & 20resolution!=yes {set 20resolution=yes}
natural=water & water=moat     & 20resolution!=yes {set 20resolution=yes}
natural=water & water=wastewater     & 20resolution!=yes {set 20resolution=yes}
natural=water & ( water=shallow | water=drain | water=well | water=salt_pool | water='Salt_pool' )     & 20resolution!=yes {set 20resolution=yes}
natural=water & ( water=intermittent | intermettent=yes )     & 20resolution!=yes {set 20resolution=yes}
natural=water & ( water=reservoir | water=canal )    & 20resolution!=yes {set 20resolution=yes} [0x3c resolution 19-21 continue with_actions]
natural=water & 20resolution!=yes & area_size() > 2000 {set 20resolution=yes} [0x3c resolution 18-21 continue with_actions]

Is there maybe a bug related to the area_size() filter and Multipolygons?

(not I did not use yet this mornings patch). Right now my server is blocked - I can try out things only from tomorrow onwards.

On 29 July 2016 at 07:07, Gerd Petermann <GPetermann_muenchen at hotmail.com<mailto:GPetermann_muenchen at hotmail.com>> wrote:

Hi Felix,

okay. In case that you also used splitter to calculate new tiles a possible explanation

might be a special case at tile boundaries, although the --keep-complete option should

avoid that.

Gerd

________________________________
Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com>>
Gesendet: Freitag, 29. Juli 2016 00:11:45
An: Development list for mkgmap
Betreff: Re: [mkgmap-dev] Option to output polygons in size order

Well - recompiled and this time the Chieemsee is fine. Really do wonder why it missed the islands. Next time someone reports somethink like this - or I notice a problem somewhere I'll report on time...

On 27 July 2016 at 16:08, Thorsten Kukuk <kukuk at suse.de<mailto:kukuk at suse.de>> wrote:
On Wed, Jul 27, Felix Hartmann wrote:

> Ah - I guess the Chieemsee will be taken from the sea input files - won't
> it?

No, it does not. Lakes are natural=water and most of the time multipolygones.

> I never really now what water features are taken from which input.

Quite easy: coastline, and only coastlines, are taken from the sea input.
Lakes are no sea.

> If
> not I would really wonder why all islands in the Chieemsee are flooded for
> me. The Chieemsee was updated last time 20 days ago - so I should have the
> correct data if it is taken from the normal osm.pbf file.

On my map, the Chiemsee including the three islands, is correct, no flooding
anywhere.

  Thorsten

> I used them since 10.06.2016 without update. I just uploaded the version I
> use here: https://openmtbmap.org/sea.zip
>
>
> Felix
>
>
>
> On 27 July 2016 at 15:14, Gerd Petermann <GPetermann_muenchen at hotmail.com<mailto:GPetermann_muenchen at hotmail.com>>
> wrote:
>
> > Hmm,
> >
> >
> > the way 4605746 is an inner member of mp-relation
> > https://www.openstreetmap.org/relation/32246
> >
> > I see no problems with the default style. Do you still have the 18.07.
> > data ?
> >
> >
> > Gerd
> >
> >
> > ------------------------------
> > *Von:* mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag von
> > Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com>>
> > *Gesendet:* Mittwoch, 27. Juli 2016 14:59:51
> >
> > *An:* Development list for mkgmap
> > *Betreff:* Re: [mkgmap-dev] Option to output polygons in size order
> >
> > Oh - check the Herreninsel Chieemsee. It was flooded based on 18.07 data
> > and already flodded in June. Was fine in March though.
> >
> > http://www.openstreetmap.org/way/4605746
> >
> > It should be a multipolygon but it's not. It's much smaller than the lake
> > however. Basically right now in my map the whole forest is flooded. Also
> > Fraueninsel flooded. Mapnik get'r it right however!
> >
> > On 27 July 2016 at 14:52, Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com>> wrote:
> >
> >> Know - sadly not. Usually such places are fixed up sooner or later - and
> >> then sometimes destroyed again. It's kinda hard to find them too - because
> >> you will either give lake or water preference (or give it same
> >> draw-priority and end up with chance). I just know since I implemented a
> >> limited layer approach - complaints about something "missing" are much more
> >> rare.
> >>
> >> On 27 July 2016 at 14:44, Gerd Petermann <GPetermann_muenchen at hotmail.com<mailto:GPetermann_muenchen at hotmail.com>
> >> > wrote:
> >>
> >>> Hi Felix,
> >>>
> >>>
> >>> okay, I like the idea reg. layer, but I was not yet able to find an
> >>> example in OSM.
> >>>
> >>> I assume the problem appears only in specific regions wheres such an
> >>> unexperienced
> >>>
> >>> mapper is active. Do you know such a region?
> >>>
> >>>
> >>> Gerd
> >>>
> >>>
> >>> ------------------------------
> >>> *Von:* mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag
> >>> von Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com>>
> >>> *Gesendet:* Mittwoch, 27. Juli 2016 14:31:28
> >>>
> >>> *An:* Development list for mkgmap
> >>> *Betreff:* Re: [mkgmap-dev] Option to output polygons in size order
> >>>
> >>> Well the smaller polygon in usual is the one that people expect to end
> >>> up on top. HOWEVER - before even checking for size - there could be a check
> >>> for the layer tag. It is still commonly used by people who do not
> >>> understand how to use multipolygons.
> >>>
> >>> So an approach could be - take polygon overlap check for values defined
> >>> in "overlap" style-file - after multipolgyon overlap is gone.
> >>> Check if layer tag is present on either of the polygons. If yes - then
> >>> cut out according to layer.
> >>> If not - cut out the smaller from the bigger. Usually it's the smaller
> >>> polygon that should appear.
> >>>
> >>> I guess it needs to happen quite late therefore. Why smaller - well
> >>> quite often people contacted me about islands missing/flooded or similar -
> >>> and usually it was the smaller polygon that should have been on top. I
> >>> guess with layer tag however 90% of all cases can already be resolved. (I
> >>> do this in a very limited way already - by having some polygons like water
> >>> and forest in several versions with different priority based on layer tag -
> >>> this did help a lot)
> >>>
> >>> Felix
> >>>
> >>> On 27 July 2016 at 13:41, Gerd Petermann <
> >>> GPetermann_muenchen at hotmail.com<mailto:GPetermann_muenchen at hotmail.com>> wrote:
> >>>
> >>>> Hi Felix,
> >>>>
> >>>>
> >>>> okay, maybe I'll add this as an experimental option as well.
> >>>>
> >>>> One big question here is: At what point would the cutting
> >>>>
> >>>> happen? Before style processing (as we do with mp-relations)
> >>>>
> >>>> or maybe as a new stage before the img data is written.
> >>>>
> >>>>
> >>>> What I don't yet understand is the idea that a smaller
> >>>>
> >>>> polygon is more important. Do you have examples for that,
> >>>>
> >>>> esp. cases where shapes do only partially overlap?
> >>>>
> >>>>
> >>>> Gerd
> >>>> ------------------------------
> >>>> *Von:* mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk<mailto:mkgmap-dev-bounces at lists.mkgmap.org.uk>> im Auftrag
> >>>> von Felix Hartmann <extremecarver at gmail.com<mailto:extremecarver at gmail.com>>
> >>>> *Gesendet:* Mittwoch, 27. Juli 2016 13:24:37
> >>>> *An:* Development list for mkgmap
> >>>> *Betreff:* Re: [mkgmap-dev] Option to output polygons in size order
> >>>>
> >>>>
> >>>> On 27 July 2016 at 09:29, Gerd Petermann <
> >>>> GPetermann_muenchen at hotmail.com<mailto:GPetermann_muenchen at hotmail.com>> wrote:
> >>>>
> >>>>> reg. the idea of "cutting out overlaps": I guess it would consume
> >>>>> quite a lot of CPU and it would heavily increase the img size
> >>>>>
> >>>>> because we would have to write many more points. Think of a shape for
> >>>>> "place=village" with hundreds of holes for each building
> >>>>>
> >>>>> shape. Up to now we save the shape for the village and the shapes for
> >>>>> the buildings. With cutting we have to calculate what
> >>>>>
> >>>>> remains of the village shape, this would be a very complex shape with
> >>>>> many holes, so it would have many points.
> >>>>>
> >>>>> I don't think that would be a good idea.
> >>>>>
> >>>>>
> >>>> Well that's why I wrote we will need an additional file in the
> >>>> style-file for this. So only for certain polygons this should be done.
> >>>> Prime examples are: any kind of forest, most kind of water, and maybe a
> >>>> handful more. However definitely not buildings or for example poygons you
> >>>> can put semi-transparent.
> >>>>
> >>>> I'm quite sure with this limited approach 90% of problems would be
> >>>> gone. And mapsize only a couple percent bigger. However I have no clue
> >>>> about complexity and CPU cycles for such a limited approach.
> >>>>
> >>>>
> >>>> --
> >>>> Felix Hartman - Openmtbmap.org & VeloMap.org
> >>>> Schusterbergweg 32/8
> >>>> 6020 Innsbruck
> >>>> Austria - Österreich
> >>>>
> >>>> _______________________________________________
> >>>> mkgmap-dev mailing list
> >>>> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
> >>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Felix Hartman - Openmtbmap.org & VeloMap.org
> >>> Schusterbergweg 32/8
> >>> 6020 Innsbruck
> >>> Austria - Österreich
> >>>
> >>> _______________________________________________
> >>> mkgmap-dev mailing list
> >>> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
> >>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >>>
> >>
> >>
> >>
> >> --
> >> Felix Hartman - Openmtbmap.org & VeloMap.org
> >> Schusterbergweg 32/8
> >> 6020 Innsbruck
> >> Austria - Österreich
> >>
> >
> >
> >
> > --
> > Felix Hartman - Openmtbmap.org & VeloMap.org
> > Schusterbergweg 32/8
> > 6020 Innsbruck
> > Austria - Österreich
> >
> > _______________________________________________
> > mkgmap-dev mailing list
> > mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
> > http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> >
>
>
>
> --
> Felix Hartman - Openmtbmap.org & VeloMap.org
> Schusterbergweg 32/8
> 6020 Innsbruck
> Austria - Österreich

> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
Thorsten Kukuk, Senior Architect SLES & Common Code Base
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
Felix Hartman - Openmtbmap.org & VeloMap.org
Schusterbergweg 32/8
6020 Innsbruck
Austria - Österreich

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
Felix Hartman - Openmtbmap.org & VeloMap.org
Schusterbergweg 32/8
6020 Innsbruck
Austria - Österreich

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
Felix Hartman - Openmtbmap.org & VeloMap.org
Schusterbergweg 32/8
6020 Innsbruck
Austria - Österreich

_______________________________________________
mkgmap-dev mailing list
mkgmap-dev at lists.mkgmap.org.uk<mailto:mkgmap-dev at lists.mkgmap.org.uk>
http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev

--
Felix Hartman - Openmtbmap.org & VeloMap.org
Schusterbergweg 32/8
6020 Innsbruck
Austria - Österreich

--
Felix Hartman - Openmtbmap.org & VeloMap.org
Schusterbergweg 32/8
6020 Innsbruck
Austria - Österreich
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20160731/698c36ed/attachment-0001.html>