[mkgmap-dev] nearby POIs

Sat May 2 21:28:40 BST 2020

Hi Mike, hi Gerd,

thanks for the nearby option, I like the new way to unclutter the map.

To my args-file i´ve added:
nearby-poi-rules=*/named:10,*/unnamed:30,0x2f00-0x2f1f:30,0x3200-0x331f:50,0x13700-0x1370d:30

A lot of the pois seem to be reduced (deleted) as expected, but I stumbled on tourism=picnic_site and leisure=picnic_table here:
https://www.openstreetmap.org/way/498671746
https://www.openstreetmap.org/way/108892873

These 2 polygon tourism=picnic_site contain picnic pois, tourism in the one and leisure in the other.
My rule combines the pois in 0x2f0b:
tourism=picnic_site | leisure=picnic_table [0x2f0b resolution 24]

As long as tourism and leisure are collected in one poi, both poi will not be reduced.
If separated to different pois, tourism is reduced, leisure not.
I can´t get the option to work on both.

Regards
Jan

> Am 29.04.2020 um 18:06 schrieb Gerd Petermann <gpetermann_muenchen at hotmail.com>:
> 
> Hi Mike,
> 
> OK, did not see this line
>  fileRules.addAll(Arrays.asList(rules));
> 
> So, I think I'll commit the patch tomorrow if nobody else suggests changes.
> 
> Gerd
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Mike Baggaley <mike at tvage.co.uk>
> Gesendet: Mittwoch, 29. April 2020 17:52
> An: 'Development list for mkgmap'
> Betreff: Re: [mkgmap-dev] nearby POIs
> 
> Hi Gerd,
> 
> I've replaced the trim statements with replaceAll in the attached updated
> patch. Having thought a bit more about caching the rules, I've realised in
> the multithreaded environment, there would need to be object locks and
> synchronisation, which would be unnecessarily complicated.
> 
> Regarding the two options - I had 30 odd rules in my command line, so the
> config file is better for me, but I expect that most users will probably
> just use one or two rules. Your range suggestion has allowed me to reduce
> the number of rules to about 25. The config file option doesn't override the
> inline option; if both are provided then both sets of rules are applied -
> hence I don't think the documentation needs changing.
> 
> Regards,
> Mike
> 
> -----Original Message-----
> From: Gerd Petermann [mailto:gpetermann_muenchen at hotmail.com]
> Sent: 29 April 2020 07:28
> To: 'Development list for mkgmap' <mkgmap-dev at lists.mkgmap.org.uk>
> Subject: Re: [mkgmap-dev] nearby POIs
> 
> Hi Mike,
> 
> regarding error handling I think we can wait for someone to complain. Each
> solution has its pros and cons.
> regarding option handling:
> - I have no idea what users do. It is possible to create very different maps
> with a single execution of mkgmap. I can't think of any good reason to do
> this, but I also see no way to find out if anybody does. I presume that many
> users don't read this list.
> - If you see a simple solution to cache the results which works fine with
> multiple threads using different options go ahead.
> - When I suggested --nearby-poi-rules-config=filename I thought that is
> should replace --nearby-poi-rules. Having both options simply complicates
> everything. If you think that it is unlikely to have more than two rules we
> should better remove --nearby-poi-rules-config=filename, else it should be
> documented that --nearby-poi-rules-config=filename overwrites
> --nearby-poi-rules.  Sorry for suggesting it in the first place.
> - all those trim() statements in combination with substring() look error
> prone. If I got that right you could use rule.replaceAll("\\s+", "") once to
> remove all whitespace characters?
> 
> Gerd
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Mike
> Baggaley <mike at tvage.co.uk>
> Gesendet: Dienstag, 28. April 2020 22:06
> An: 'Development list for mkgmap'
> Betreff: Re: [mkgmap-dev] nearby POIs
> 
> Hi Gerd, please find attached an updated patch that allows for # comments
> either as whole lines or as ends of a line. It also allows whitespace
> between the components of the rules.
> 
> At the moment, if an error is found in a rule, this is reported as an error
> and that rule is ignored. This does result in duplicated error messages, one
> from each tile. I'm happy to change this to stop processing if you think
> that is better. I'm not keen on ignoring all rules if an error is found in a
> rule. If the rules file is not found then this will also just report an
> error and continue. Perhaps this one at least would be better if it caused
> termination?
> 
> I did think about loading the rules at the beginning, prior to processing
> the tiles, but realised this would break the 'option before tile' convention
> which allows different options to be passed to each tile (does anyone do
> this for anything other than the description?). It would be pretty
> straightforward to cache the options and resulting rules and if the options
> are the same for the next tile, use the cached rules instead of
> reprocessing. What do you think?
> 
> Regards,
> Mike
> 
> -----Original Message-----
> From: Gerd Petermann [mailto:gpetermann_muenchen at hotmail.com]
> Sent: 28 April 2020 09:21
> To: 'Development list for mkgmap' <mkgmap-dev at lists.mkgmap.org.uk>
> Subject: Re: [mkgmap-dev] nearby POIs
> 
> Hi Mike,
> 
> I did not try it, but it seems the configuration file doesn't allow the
> typical "#" as comment. This makes it rather useless. We should at least
> allow comment lines starting with a # to be ignored.
> I've added code to handle this, also untested.
> 
> - What should happen when an error was found in the rules?
> + Ignore only the bad rule(s)?
> + Ignore all rules?
> + stop processing?
> We have this problem with other options like --style-option as well, so it's
> not directly related to the patch. I think It would be good to check this
> before any tile is processed, but the current implementation of the option
> handling makes this really difficult.
> 
> Gerd
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Mike
> Baggaley <mike at tvage.co.uk>
> Gesendet: Montag, 27. April 2020 16:06
> An: 'Development list for mkgmap'
> Betreff: Re: [mkgmap-dev] nearby POIs
> 
> Hi Gerd, please find attached an updated patch that addresses the points
> below.
> 
> Cheers,
> Mike
> 
> -----Original Message-----
> From: Mike Baggaley [mailto:mike at tvage.co.uk]
> Sent: 22 April 2020 17:55
> To: 'Development list for mkgmap' <mkgmap-dev at lists.mkgmap.org.uk>
> Subject: Re: [mkgmap-dev] nearby POIs
> 
> Hi Gerd,
> 
> To answer your points:
> 
> 1) 0x2f1f was just a POI in my style which I have as a bus stop, and as bus
> stations often have all individual stops mapped it can be useful to remove
> duplicates (I don't use the default style). Looking at the default style I
> see that it has both bus stops and taxi as 0x2f17. I'm happy with either the
> example changing to 0x2f17 or the default style changing to 0x2f1f for bus
> stops (I have 0x2f17 just for taxi). The sample doesn't have to match the
> default style, but it would make sense for it to do so.
> 
> 2) The code sorts the rules into POI type order, but doesn't sort by
> distance. It would better handle the example you have given if it sorted
> rules with the same POI type by distance. I'll look into this.
> 
> 3) Allowing a range of POI types seems like it could be useful. I see that
> --poi-excl-index uses a hyphen to specify a range and propose we use the
> same format. I'll look into this as well.
> 
> Regards,
> Mike
> 
> -----Original Message-----
> From: Gerd Petermann [mailto:gpetermann_muenchen at hotmail.com]
> Sent: 22 April 2020 07:07
> To: 'Development list for mkgmap' <mkgmap-dev at lists.mkgmap.org.uk>
> Subject: Re: [mkgmap-dev] nearby POIs
> 
> Hi Mike,
> 
> okay, I started to find out if the code works as documented. Here is a first
> set of questions:
> 1) I started with the given sample
> --nearby-poi-rules=*/named:10,*/unnamed:25,0x2f1f:30
> and found out the the default style doesn't use type 0x2f1f. Is that
> intended?
> 2) Is it allowed to use multiple rules for the same POI type?
> Two rules like
> --nearby-poi-rules=0x2f1f:10:merge-at-mid-point,0x2f1f:1
> are accepted, but I am not sure what to expect when this is done. The rules
> are applied in the given order, so the second will never match, right?
> 3) The option --poi-excl-index allows to specify ranges of POI types.
> Wouldn't it be good to use the same logic for --nearby-poi-rules?
> 
> Gerd
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Mike
> Baggaley <mike at tvage.co.uk>
> Gesendet: Mittwoch, 22. April 2020 01:03
> An: 'Development list for mkgmap'
> Betreff: Re: [mkgmap-dev] nearby POIs
> 
> HI Gerd, the mods look fine to me. I don't think documenting a config file
> in a sample file is a good idea. I think it is much better if all the
> documentation is in one place. I also think it make sense to keep the
> --nearby-poi-rules option as I expect some users would just want something
> like -- nearby-poi-rules=*:25, and it would be overkill to have to create a
> config file to do this. I have fixed the formatting issues in the options
> file in the attached version of the patch.
> 
> Regards,
> Mike
> 
> -----Original Message-----
> From: Gerd Petermann [mailto:gpetermann_muenchen at hotmail.com]
> Sent: 21 April 2020 08:46
> To: 'Development list for mkgmap' <mkgmap-dev at lists.mkgmap.org.uk>
> Subject: Re: [mkgmap-dev] nearby POIs
> 
> Hi Mike,
> 
> attached is a modified version of the patch. I've removed some of the
> changes in StyledConverter and did some cleanup on the NearbyPoiHandler.
> 
> I think you got me wrong regarding the config file. With similar to
> --road-name-config=filename I really meant that the config file should be
> provided as an example containing the detailed documentation.
> The option --nearby-poi-rules should be removed, it just adds complexity.
> This will also fix some formatting problems in file options.
> 
> Gerd
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Mike
> Baggaley <mike at tvage.co.uk>
> Gesendet: Montag, 20. April 2020 16:00
> An: 'Development list for mkgmap'
> Betreff: Re: [mkgmap-dev] nearby POIs
> 
> Hi Gerd, the logging changes were actually part of another patch that is
> aimed at tidying up logging, which causes me significant problems due to the
> mixed use of System.err.println and log.xxxx. If you want to remove the
> System.error.println change from the patch then that is fine.
> 
> Regards,
> Mike
> -----Original Message-----
> From: Gerd Petermann [mailto:gpetermann_muenchen at hotmail.com]
> Sent: 20 April 2020 07:25
> To: 'Development list for mkgmap' <mkgmap-dev at lists.mkgmap.org.uk>
> Subject: Re: [mkgmap-dev] nearby POIs
> 
> Hi Mike,
> 
> I agree about the changes to debug level because the remaining log.info()
> messages are still meaningful.
> 
> You changed some System.err.println(...) to log.info(...).
> This has two effects:
> 1) severity is changed from error to info (which is wrong in these cases),
> the user should always see these messages.
> 2) the normal logging procedure prints the path to the current tile but the
> tile is not relevant and thus may be confusing.
> 
> I am not 100% sure about the messages log.warn("road has < 2
> points",way.getId(),"(discarding)");
> You probably only see them when you disable WrongAngleFixer by modfying the
> code. What was the reason for the change?
> 
> Gerd
> 
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Mike
> Baggaley <mike at tvage.co.uk>
> Gesendet: Sonntag, 19. April 2020 15:24
> An: 'Development list for mkgmap'
> Betreff: Re: [mkgmap-dev] nearby POIs
> 
> Hi Gerd, the logging changes seem to me to be correct (though unrelated to
> the nearby POI change), reducing the severity of some messages to match
> those elsewhere. I accept that these things are open to interpretation, but
> would consider messages about entering and leaving functions and similar
> static code location messages to be debug level messages (e.g. "Maintaining
> highway counters"), those that give information that the code is doing
> something that does not need any action to be informational messages (e.g.
> "Pruning point xxxx"), those that might warrant investigation would be
> warning (e.g. Oneway road xxx goes nowhere" ) and those that indicate some
> sort of real problem would be error (e.g. "shape is not closed with
> identical points ").
> 
> Regards,
> Mike
> 
> -----Original Message-----
> From: Gerd Petermann [mailto:gpetermann_muenchen at hotmail.com]
> Sent: 19 April 2020 08:41
> To: 'Development list for mkgmap' <mkgmap-dev at lists.mkgmap.org.uk>
> Subject: Re: [mkgmap-dev] nearby POIs
> 
> Hi Mike,
> 
> thanks, will look at it during the next days. Some of the changes to the
> logging in StyledConverter are probably not OK.
> 
> Gerd
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Mike
> Baggaley <mike at tvage.co.uk>
> Gesendet: Sonntag, 19. April 2020 08:45
> An: 'Development list for mkgmap'
> Betreff: Re: [mkgmap-dev] nearby POIs
> 
> Hi Gerd, please find attached an updated patch that provides a config file
> option and puts the nearby POI code into a new class.
> 
> Cheers,
> Mike
> 
> -----Original Message-----
> From: Mike Baggaley [mailto:mike at tvage.co.uk]
> Sent: 08 April 2020 14:02
> To: 'Development list for mkgmap' <mkgmap-dev at lists.mkgmap.org.uk>
> Subject: RE: [mkgmap-dev] nearby POIs
> 
> HI Gerd, I'll look into having a config file option and separating the code
> into a new class. Probably will be a couple of days before I get back to you
> with an updated version of the patch.
> 
> Cheers,
> Mike
> 
> -----Original Message-----
> From: Gerd Petermann [mailto:gpetermann_muenchen at hotmail.com]
> Sent: 07 April 2020 09:07
> To: Development list for mkgmap <mkgmap-dev at lists.mkgmap.org.uk>
> Subject: Re: [mkgmap-dev] nearby POIs
> 
> Hi Mike,
> 
> the syntax for the option parameters is extremely complex. I think it would
> be better to use a config file for that, similar to the option
> --road-name-config=roadNameConfig.txt we could have a
> --nearby-poi-rules=nearbyPoiConfig.txt
> What do you think?
> 
> Gerd
> 
> 
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Mike
> Baggaley <mike at tvage.co.uk>
> Gesendet: Montag, 6. April 2020 20:24
> An: 'Development list for mkgmap'
> Betreff: [mkgmap-dev] nearby POIs
> 
> Hi Gerd, please find attached a patch for handling the removal of duplicate
> POIs that are near to each other but not coincident. The existing code
> handles the removal of coincident duplicates, but leaves other duplicates in
> place. It is very common in OSM for buildings to be tagged as amenities with
> a single node also being tagged as the same amenity. If using
> --add-pois-to-areas, this can lead to a lot of duplication. The patch
> provides a new command line option with the following syntax:
> 
> --nearby-poi-rules=type[/all|/named|/unnamed]:max-distance[:delete-poi|delet
> e-name|merge-at-mid-point][,...]
> 
> This defines a set of rules to follow when a POI is near to another of the
> same type. Each rule consists of three parts separated by colons. The
> first two parts must be provided; the last part can be defaulted.
> 
> The first part of the rule is the Garmin POI type code with an optional
> suffix; it determines when the rule is triggered. The type code may be
> specified in decimal or hexadecimal (e.g. 0x2c0b). A rule is triggered
> when processing a POI if the type code of the POI matches the rule type,
> providing there is also a match in the POI name and the first part
> suffix. If the suffix is '/all' (the default) then the match is only made
> on the type. If the suffix is '/named' then the rule is only triggered if
> the POI has a name. If the suffix is '/unnamed' then the rule is only
> triggered if the POI has no name. A wildcard of an asterisk character may
> be used to match any type code. The wildcard may also be combined with a
> suffix to allow separate processing of named and unnamed POIs.
> 
> The second part of the rule is the distance in metres which an already
> processed POI must be within for it to be considered to be nearby and
> hence trigger the action part of the rule.
> 
> The third part of the rule is the action part and provides three options:
> ::delete-poi - the POIS are considered to be duplicates and the
> duplicate is deleted. This is the default.
> ::delete-name - the POIS are not duplicates, but only a single
> name needs to be displayed.
> ::merge-at-mid-point - the POIS are considered to be duplicates
> but the location of the existing point is moved to a point
> midway between the two points.
> 
> Wildcard rules are only applied if no other rule is applicable.
> 
> For example:
> :
> --nearby-poi-rules=*/named:50,*/unnamed:25,0x2f1f/named:50:delete-name,0x2f1
> f:3,0x641d:400:merge-at-mid-point
> 
> This has the following effect:
> :       If no other rule applies, a POI with the same name and type and
>        within 50m of one already processed will be deleted
>        If no other rule applies, a POI having no name and of the same type
>        and within 25m of one already processed will be deleted
>        A POI of type 0x2f1f that is within 50m of another POI with the same
> name and type will have its name deleted
>        A POI of type 0x2f1f that is within 3m of another POI with the same
> type will be deleted
>        A POI of type 0x641d that is within 400m of another POI with the
> same
> type will be deleted and the existing point moved
> 
> Note: a POI that matches another in type, name and exact location is always
> considered a duplicate and deleted.
> 
> Regards,
> Mike
> 
> 
> 
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> 
> 
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> 
> 
> 
> 
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev