[mkgmap-dev] [PATCH v6] Automatic location completion

Thu Mar 10 18:17:14 GMT 2011

Now *with* my answer:

Your style file must contain rules which assign the country, region and 
city name. So something like this:
mkgmap:region!=*&  mkgmap:admin_level6=* { set
  mkgmap:region='${mkgmap:admin_level6}' }
mkgmap:region!=*&  mkgmap:admin_level5=* { set
   mkgmap:region='${mkgmap:admin_level5}' }
mkgmap:region!=*&  mkgmap:admin_level4=* { set
  mkgmap:region='${mkgmap:admin_level4}' }
mkgmap:region!=*&  mkgmap:admin_level3=* { set
  mkgmap:region='${mkgmap:admin_level3}' }

These rules are contained in the default stlye of the branch. So you can 
copy them and modify as you like it.

The warning "SEVERE (LocationHook): philippines.osm: Cannot process 
location element because it contains no name tag: 27751466" tells you 
that there are boundaries without a name tag. They are useless for the 
index because a city without a name cannot be searched. The id is the 
way id, so you can fix that by modifying the way 
http://www.openstreetmap.org/browse/way/27751466

At the moment roads are not splitted on borders. This is on the TODO list.

WanMil

> Thanks for jar file.  Now to report my test results:
> I uploaded the map via Garmin MapInstall search for "Address" and
> "Intersections".  I get a "No Data Available"
>
> Using: mkgmap-locator-r1892.jar
>
> Args.list:
> ode-page=1252
> tdbfile
> latin1
> country-abbr=PHL
> country-name=PHILIPPINES
> remove-short-arcs=5
> route
> add-pois-to-areas
> family-id=639
> family-name=OSM_PHIL
> overview-mapname=40000001
> series-name=OSM_PHIL
> description=OSM_PHIL
> style-file=/home/maning/Downloads/osm/routable_garmin/git/osmphgps/styles/default
> generate-sea=multipolygon,extend-sea-sectors,close-gaps=1000
> index
> make-poi-index
> adjust-turn-headings
> drive-on-right
> report-dead-ends
> ignore-maxspeeds
> link-pois-to-ways
> location-autofill=0
>
>
> Some error reports:
> SEVERE (LocationHook): philippines.osm: Element lists created after 230 ms
> SEVERE (LocationHook): philippines.osm: Quadtree created after 23798 ms
> SEVERE (LocationHook): philippines.osm: Location admin_level=1
> processed after 23815 ms
> SEVERE (LocationHook): philippines.osm: Location admin_level=2
> processed after 23864 ms
> SEVERE (LocationHook): philippines.osm: Start assigning  admin_level=2
> after 23962 ms
> SEVERE (LocationHook): philippines.osm: Special admin_level=2 handling
> processed after 24627 ms
> SEVERE (LocationHook): philippines.osm: Cannot process location
> element because it contains no name tag: 4611686018427393428
> [boundary=administrative,admin_level=3,mkgmap:stylefilter=polygon,mkgmap:admin_level2=PHL]
> SEVERE (LocationHook): philippines.osm: Location admin_level=3
> processed after 40069 ms
> SEVERE (LocationHook): philippines.osm: Cannot process location
> element because it contains no name tag: 61462633
> [boundary=administrative,admin_level=4,mkgmap:admin_level2=PHL]
> SEVERE (LocationHook): philippines.osm: Cannot process location
> element because it contains no name tag: 4611686018427387951
> [boundary=administrative,admin_level=4,mkgmap:stylefilter=polygon,mkgmap:admin_level2=PHL]
> SEVERE (LocationHook): philippines.osm: Cannot process location
> element because it contains no name tag: 27751466
> [boundary=administrative,admin_level=4,mkgmap:admin_level2=PHL]
> SEVERE (LocationHook): philippines.osm: Location admin_level=4
> processed after 40335 ms
> SEVERE (LocationHook): philippines.osm: Location admin_level=5
> processed after 40336 ms
> SEVERE (LocationHook): philippines.osm: Cannot process location
> element because it contains no name tag: 28045303
> [mkgmap:admin_level6=Makati,boundary=administrative,admin_level=6,mkgmap:admin_level2=PHL,mkgmap:admin_level3=Metro
> Manila]
> SEVERE (LocationHook): philippines.osm: Location admin_level=6
> processed after 42682 ms
> SEVERE (LocationHook): philippines.osm: Location admin_level=7
> processed after 42683 ms
> SEVERE (LocationHook): philippines.osm: Location admin_level=8
> processed after 42823 ms
> SEVERE (LocationHook): philippines.osm: Location admin_level=9
> processed after 42836 ms
> SEVERE (LocationHook): philippines.osm: Location admin_level=10
> processed after 44427 ms
> SEVERE (LocationHook): philippines.osm: Location admin_level=11
> processed after 44427 ms
> SEVERE (LocationHook): philippines.osm: Location postal_code processed
> after 44427 ms
> SEVERE (LocationHook): philippines.osm: Location hook finished in 44427 ms
>
>
> Next I tried using the default style and now I can search for streets
> via "Address".  There must be something wrong with my own style
> Moving on, for areas with proper relation/multipolygon I get good
> results for streets within villages (admin_level=10
> http://www.openstreetmap.org/browse/relation/371327) but I can't get
> streets within tows/cities (admin_level=6
> http://www.openstreetmap.org/browse/relation/146949)
>
> One question, does your code split the road when it crosses another admin_level?
>
>
>
> 2011/3/10 Carlos Dávila<cdavilam at orangecorreo.es>:
>> El 08/03/11 23:07, WanMil escribió:
>>
>> Next round of the location improvements:
>> * The algorithm that searched which elements were contained within a
>> boundary was (is) wrong. I updated some parameters in the Quadtree so I the
>> probability is very much lower that an element is not assigned correctly.
>>
>> In case they can help you improve the algorithm, here you have two examples
>> of nodes wrongly assigned (according to my styles below): nodes 945168390
>> and 943005606, that are within boundary relation 348994 (admin_level=6) are
>> assigned region=Extremadura (boundary relation 349050, admin_level=4). I can
>> provide more examples if necessary.
>> Extract of the styles:
>> mkgmap:region!=*&  is_in:province=* { set mkgmap:region='${is_in:province}'
>> }
>> mkgmap:region!=*&  mkgmap:admin_level6=* { set
>> mkgmap:region='${mkgmap:admin_level6}' }
>> mkgmap:region!=*&  mkgmap:admin_level5=* { set
>> mkgmap:region='${mkgmap:admin_level5}' }
>> mkgmap:region!=*&  mkgmap:admin_level4=* { set
>> mkgmap:region='${mkgmap:admin_level4}' }
>> mkgmap:region!=*&  mkgmap:admin_level3=* { set
>> mkgmap:region='${mkgmap:admin_level3}' }
>>
>> * The mkgmap:country, addr:country and is_in:country tag is now always
>> assigned with the three letter country code. This makes it possible to
>> define country specific rules in the style files.
>>
>> With no changed on your mkgmap:country rules in styles I have (among others)
>> Esp and España (ESP) in the list of countries. If I select Esp, a full list
>> of places is found, but if I select España (ESP), nothing in found.
>>
>> A note ot Carlos:
>> With this patch you can add debug rules to the style file. So if you don't
>> understand why a specific region/country is used add the following line to
>> your style files:
>> mkgmap:region=* { echo "R=${mkgmap:region} 3=${mkgmap:admin_level3}
>> 4=3=${mkgmap:admin_level4} 5=${mkgmap:admin_level5} 6=${mkgmap:admin_level6}
>> is_in=${is_in:region}" }
>>
>> How can I send the echo output to a file? I tried java -jar mkgmap.jar
>> options input_files>  textfile with no success
>>
>> I think your rules are ok.
>>
>> By the way: Should I create a branch for my changes? Maybe this makes it
>> easier for more people to test and to play with?
>>
>> WanMil
>>
>> El 06/03/11 19:40, WanMil escribió:
>>
>> I implemented Minkos idea using the style file to set the city,
>> region, zip and country names.
>>
>> How does it work?
>> The new AddressHook adds special tags to each element that is inside a
>> known boundary:
>> mkgmap:admin_level2
>> mkgmap:admin_level3
>> mkgmap:admin_level4
>> ..
>> mkgmap:admin_level11
>>
>> If an admin_level is not known the relevant tag is not set. The same
>> for the zip tag mkgmap:postalcode.
>>
>> The style file contains a new rule block (in each file):
>> mkgmap:country!=*&  addr:country=*
>>     { set mkgmap:country='${addr:country}' }
>> mkgmap:country!=*&  is_in:country=*
>>     { set mkgmap:country='${is_in:country}' }
>> mkgmap:country!=*&  mkgmap:admin_level2=*
>>     { set mkgmap:country='${mkgmap:admin_level2}' }
>>
>> mkgmap:region!=*&  is_in:county=*
>>     { set mkgmap:region='${is_in:county}' }
>> mkgmap:region!=*&  mkgmap:admin_level6=*
>>     { set mkgmap:region='${mkgmap:admin_level6}' }
>> ...
>>
>> mkgmap:city!=*&  openGeoDB:name=*
>>     { set mkgmap:city='${openGeoDB:name}' }
>> mkgmap:city!=*&  mkgmap:admin_level8=*
>>     { set mkgmap:city='${mkgmap:admin_level8}' }
>> ...
>>
>> mkgmap:postal_code!=*&  addr:postcode=*
>>     { set mkgmap:postal_code='${addr:postcode}' }
>> mkgmap:postal_code!=*&  mkgmap:postcode=*
>>     { set mkgmap:postal_code='${mkgmap:postalcode}' }
>>
>> These rules set the mkgmap tags:
>> mkgmap:country
>> mkgmap:region
>> mkgmap:city
>> mkgmap:postal_code
>>
>> Only these tags are used to set the country, the region, the city and
>> the zip code. No more rules are applied automatically (there are still
>> some in the Locator class but they will be removed).
>>
>> This enables you to configure which admin_level information you want
>> to use for the city name, the region names etc. Also country specific
>> rules are possible (e.g. add a mkgmap:country=NLD rule).
>>
>> Todo: The country normalization must be performed by the AddressHook
>> so that one can be sure that the Netherlands are always tagged with
>> mkgmap:country=NLD (or mkgmap:country=Nederland?)
>>
>> The rules are not perfect up to now but please feel free to post some
>> good rules for your country.
>>
>> I have changed my styles as follows,
>>
>> mkgmap:region!=*&  is_in:region=* { set mkgmap:region='${is_in:region}' }
>> mkgmap:region!=*&  mkgmap:admin_level4=* { set
>> mkgmap:region='${mkgmap:admin_level4}' }
>> mkgmap:region!=*&  mkgmap:admin_level6=* { set
>> mkgmap:region='${mkgmap:admin_level6}' }
>> mkgmap:region!=*&  mkgmap:admin_level5=* { set
>> mkgmap:region='${mkgmap:admin_level5}' }
>> mkgmap:region!=*&  mkgmap:admin_level3=* { set
>> mkgmap:region='${mkgmap:admin_level3}' }
>>
>> to adapt them to what I think is best for Spain, but, if I understood
>> well, something is not working as expected: places tagged as villages
>> that have no is_in:region information and are within admin_level=4 and
>> admin_level=6 polygons take the region from the a_l=6 polygon, instead
>> of a_l=4 as it should be from the style above.
>>
>> WanMil
>>
>>
>> Attached is the next step for the automatic location completion:
>> * Some speed improvements (it's still slow but not as slow as before ;-)
>> * boundary=postal_code is now supported.
>> * The Locator does no longer overwrite already set attributes. This
>> seems to be a better solution. I think the Locator must be rewritten
>> completely. Up to now I have commented some lines only without deep
>> understanding what it is really doing.
>>
>> Some things that don't work:
>> * Lot's of POIs are assigned with the default country although the
>> MapElements are assigned with the correct country. Example:
>> - Spain splitted with --overlap=3000 (more overlap is better for my
>> patch)
>> - Cafe Antxi (http://www.openstreetmap.org/browse/node/629554219) is
>> correctly assigned with country=ESP but in the POI search it is
>> displayed with the default country.
>> I have no idea why.
>>
>> * Zips are only applied to POIs. Streets seems to be not yet supported
>> by the MDR creation. (=>  Steve?)
>>
>> Things to do:
>> * Speed improvement (At the moment the location search takes as much
>> time as the rest of the map creation)
>> * A better mapping of city, region to the admin-levels
>> * Make use more OSM information (like residential areas, place=city POIs
>> in residential areas etc.)
>> * Clean up of the Locator class
>>
>> WanMil
>>
>>
>> I have improved calculation of the country information. In case no
>> country information is available the most used country in a tile is
>> used
>> for all elements without country tag.
>>
>> The patch now patches the trunk and no longer the index branch.
>>
>> I have observed that lots of POIs are assigned the default country
>> although I they are assigned a different country. I have no clue where
>> this happens.
>> Maybe an index related problem? I was hoping that Steves commit today
>> would fix that problem but it didn't.
>>
>> WanMil
>>
>> I have done some development and want you to share my current
>> findings.
>>
>> 1. The MapElement copy constructor seems to have a bug. The attributes
>> map which contains the city, region and country information is not
>> copied. From my point of view this is an important thing that
>> should be
>> fixed in the trunk also.
>>
>> 2. In the patch the Locator is disabled as much as it was possible for
>> me up to now.
>>
>> 3. I am using now separate tags (mkgmap:city, mkgmap:region etc.).
>>
>> I recommend to run this patch with location-autofill=-1 or 0 to see
>> how
>> the patch works and not how the old Locator works.
>>
>> WanMil
>>
>>
>> After the index branch has been developed so far that we can use the
>> results in MapSource (thanks to Steve!!) there is a big need for a
>> mechanism to fill missing location information.
>>
>> There has been a lot of discussion about that. It stopped because it
>> was
>> quite theoretical (from my point of view). Therefore I have
>> started to
>> implement a prototype osm hook that tries to complete the missing
>> location information. I think the is_in tag is quite problematic
>> therefore I decided to use the addr tags.
>>
>> Attached is a *very* early implementation that makes use of the
>> boundary
>> polygons. It checks all nodes and ways/polygons within all boundary
>> polygons and fills the missing addr tags.
>>
>> I think this is a better foundation for the discussion how we want to
>> fill the missing location information.
>>
>> Before flaming me about terrible code please be aware that:
>> * the patch is not optimized
>> * it's version 1, so far away from handling all situations /
>> countries
>> * it's a very early prototype
>>
>> I would be happy if you try it and make some proposals how it
>> could be
>> improved.
>>
>> Some improvements I already know and thinking about:
>> * The new hook should run after the multipolygon finishing and not
>> before
>> * The is_in tag should be handled
>> * boundary information propagation =>  First go through all boundaries
>> and add the information from higher levels to lower ones, i.e. add
>> addr:district to all city boundaries within that district
>> * Use landuse=residential information in combination with the place
>> nodes
>> * In case a boundary level is missing (which mostly will be the case
>> for
>> countries admin_level=2), select the addr information from all nodes
>> and
>> ways within a boundary by using the most used addr-tag value. (in
>> germany the addr:country=DE should be the most used)
>>
>>
>> Have fun!
>> WanMil
>>
>>
>>
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>> --
>> Por favor, no me envíe documentos con extensiones .doc, .docx, .xls, .xlsx,
>> .ppt, .pptx, .mdb, mdbx
>> Instale OpenOffice desde http://es.openoffice.org/programa/index.html
>> OpenOffice es libre: se puede copiar, modificar y redistribuir libremente.
>> Gratis y totalmente legal.
>> OpenOffice está en continuo desarrollo y no tendrá que pagar por las nuevas
>> versiones.
>>
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>>
>
>
>