[mkgmap-dev] splitter and options --description / --geonames-file

From Randolph J. Herber army.bronze.star at gmail.com on Fri Apr 10 01:17:47 BST 2020

Hi, all!

Yes, it helps.

It was not what I had in mind.

Fragment of template.args showing the OLC or Google Plus Codes that were 
added.

These codes can be directly used to show the area in question: 
EG-Alexandria:8G3F5W00+ <https://plus.codes/8G3F5W00+>

Image of Alexandria, Egypt, area referenced.

and for an Egyptian area with no cities with a population as large as 15 
000: EgyptUC:7GJM0000+ <https://plus.codes/7GJM0000+>

Large area in southern Egypt with no cities as large as 15 thousand people.

This area is in southern Egypt.

The differences in AreaList.java, in Unix format are in the attached 
differences file.

The modified AreaList.java files in the attached java file.

Can you now understand what I want and why?

Randolph J. Herber


On 4/9/2020 9:48 AM, Gerd Petermann wrote:
> Hi all,
>
> sorry, I think some mails where not sent to the list. I've attached a patch that adds a number to the city name when --geonames-file is used and multiple tiles
> are close to that city.
>
> I fear I don't know in what situation these names are important. So, please try if this improves something.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Gerd Petermann <gpetermann_muenchen at hotmail.com>
> Gesendet: Mittwoch, 8. April 2020 16:40
> An: Development list for mkgmap; Ticker Berkin
> Betreff: Re: [mkgmap-dev] splitter and options --description    /       --geonames-file
>
> Hi Randolph,
>
> in what situation do you use these codes? I am happy with the mapids generated by splitter.
>
> Gerd
>
> ________________________________________
> Von: mkgmap-dev <mkgmap-dev-bounces at lists.mkgmap.org.uk> im Auftrag von Randolph J. Herber <army.bronze.star at gmail.com>
> Gesendet: Mittwoch, 8. April 2020 15:13
> An: Development list for mkgmap; Ticker Berkin
> Betreff: Re: [mkgmap-dev] splitter and options --description /  --geonames-file
>
> Hi, all,
>
> The Definition of Google Plus Codes, a.k.a. Opern Location Codes, can be
> found at https://en.wikipedia.org/wiki/Open_Location_Code
>
> The complete code also specifies, by its length, the size of the area
> being described. When using them in splitter, I have never seen a tile
> whose code was longer than 8 characters, an area 250m east to west and
> 125m north to south. I use the largest OLC centered on the tile in
> question that fits within the tile.
>
> Randolph J. Herber
>
> On 4/8/2020 4:59 AM, Ticker Berkin wrote:
>> Hi Gerd
>>
>> I'd say that if the splitter is going to generate descriptions per tile
>> (geonames) it should generate one for each tile and, if there isn't a
>> centre of a city within the tile and no default description, it should
>> use a modified form of the nearest city to make it unique from a
>> possible adjacent tile that might use the same city.
>>
>> Maybe something like "GB-~Basingstoke", but no solution is ideal and
>> there could still be duplicates.
>>
>> The splitter could keep track of which names were have been used and,
>> for the ones outside the tile, added an increasing suffix number, so
>> would have "GB-Basingstoke-2" etc
>>
>> Ticker
>>
>> On Wed, 2020-04-08 at 08:38 +0000, Gerd Petermann wrote:
>>> Hi all,
>>>
>>> this is a follow up of http://gis.19327.n8.nabble.com/documentation-i
>>> mprovement-tp5962609p5962747.html
>>>
>>> Since I don't use these options for my maps I am unsure what is best.
>>> The --geonames-file option tries to find a city within the calculated
>>> tile. If none is found it either writes the value
>>> given with --description or just a comment.
>>> I think it would be best to always write the nearest city.
>>>
>>> Gerd
>>> _______________________________________________
>>> mkgmap-dev mailing list
>>> mkgmap-dev at lists.mkgmap.org.uk
>>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>> _______________________________________________
>> mkgmap-dev mailing list
>> mkgmap-dev at lists.mkgmap.org.uk
>> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
>
> _______________________________________________
> mkgmap-dev mailing list
> mkgmap-dev at lists.mkgmap.org.uk
> http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20200409/98d25613/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: template.args.JPG
Type: image/jpeg
Size: 45126 bytes
Desc: not available
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20200409/98d25613/attachment-0003.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Alexandria.JPG
Type: image/jpeg
Size: 67769 bytes
Desc: not available
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20200409/98d25613/attachment-0004.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 7GJM.JPG
Type: image/jpeg
Size: 22314 bytes
Desc: not available
URL: <http://www.mkgmap.org.uk/pipermail/mkgmap-dev/attachments/20200409/98d25613/attachment-0005.jpe>
-------------- next part --------------
269a270,271
> 	
> 	private static CityFinder cityFinder = null;
272,273c274,275
< 		CityFinder cityFinder = null;
< 		if (geoNamesFile != null){
---
> 		synchronized (this) {
> 			if (geoNamesFile != null){
279a282
> 			}
281,287c284
< 		for (Area area : getAreas()) {
< 			area.setName(description);
< 			if (cityFinder == null)
< 				continue;
< 
< 			// Decide what to call the area
< 			Set<City> found = cityFinder.findCities(area);
---
> 		for (Area area : getAreas()) {	
289,291c286,295
< 			for (City city : found) {
< 				if (bestMatch == null || city.getPopulation() > bestMatch.getPopulation()) {
< 					bestMatch = city;
---
> 			if (cityFinder != null) {
> 
> 				// Decide what to call the area
> 				Set<City> found = cityFinder.findCities(area);
> 
> 				// Select largest population city in area/
> 				for (City city : found) {
> 					if (bestMatch == null || city.getPopulation() > bestMatch.getPopulation()) {
> 						bestMatch = city;
> 					}
294,295c298,302
< 			if (bestMatch != null)
< 				area.setName(bestMatch.getCountryCode() + '-' + bestMatch.getName());
---
> 			area.setName(
> 				(bestMatch == null
> 					? description
> 					: bestMatch.getCountryCode() + '-' + bestMatch.getName()) +
> 				":"+plusCode(area));
350c357,402
< 	}	
---
> 	}
> 	
> 	private static final String codes = "23456789CFGHJMPQRVWX";
> 	
> 	private static final double GarminFactor = 360.0/((double)(1<<24));
> 	
> 	private String plusCode(Area area) {
> 		double south = area.getMinLat()*GarminFactor;
> 		double west = area.getMinLong()*GarminFactor;
> 		double north = area.getMaxLat()*GarminFactor;
> 		double east = area.getMaxLong()*GarminFactor;
> 		double centerLatitude = 0.5 *(south + north)+90.;
> 		double centerLongitude = 0.5 * (west + east)+180.;
> 		double boxHeight = north - south;
> 		double boxWidth = east - west;
> 		double height = 400.;
> 		double width = 400.;
> 		StringBuilder code = new StringBuilder();
> 		boolean done = false;
> 		while(code.length() < 8  & !done) {
> 			int p;
> 			height /= 20.;
> 			width /= 20.;
> 			p = (int)Math.floor(centerLatitude/height);
> 			code.append(codes.charAt(p));
> 			centerLatitude -= p * height;
> 			p = (int)Math.floor(centerLongitude/width);
> 			code.append(codes.charAt(p));
> 			centerLongitude -= p * width;
> 			done = width <= boxWidth && height <= boxHeight;
> 		}
> 		while(code.length() < 8) code.append("00");
> 		code.append("+");
> 		while(!done) {
> 			int la, lo;
> 			height /= 5.;
> 			width /= 4.;
> 			la = (int)Math.floor(centerLatitude/height);
> 			centerLatitude -= la * height;
> 			lo = (int)Math.floor(centerLongitude/width);
> 			centerLongitude -= lo * width;
> 			code.append(codes.charAt(4*la+lo));
> 			done = width <= boxWidth && height <= boxHeight;
> 		}
> 		return code.toString();
> 	}
-------------- next part --------------
/*
 * Copyright (c) 2009, Steve Ratcliffe
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License version 3 as
 * published by the Free Software Foundation.
 *
 * This program is distributed in the hope that it will be useful, but
 * WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 * General Public License for more details.
 */
package uk.me.parabola.splitter;

import java.awt.Point;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintWriter;
import java.io.Reader;
import java.io.Writer;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Date;
import java.util.List;
import java.util.Locale;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.xmlpull.v1.XmlPullParserException;

import uk.me.parabola.splitter.geo.City;
import uk.me.parabola.splitter.geo.CityFinder;
import uk.me.parabola.splitter.geo.CityLoader;
import uk.me.parabola.splitter.geo.DefaultCityFinder;
import uk.me.parabola.splitter.kml.KmlParser;
import uk.me.parabola.splitter.kml.KmlWriter;
import uk.me.parabola.splitter.solver.PolygonDesc;

/**
 * A list of areas.  It can be read and written to a file.
 */
public class AreaList {
	private final List<Area> areas;
	private final String description;
	private String geoNamesFile;
	
	/**
	 * This constructor is called when you are going to be reading in the list from
	 * a file, rather than making it from an already constructed list.
	 */
	public AreaList(String description) {
		this(new ArrayList<Area>(), description);
	}

	public AreaList(List<Area> areas, String description) {
		this.description = description;
		this.areas = areas;
	}

	/**
	 * Write out a file containing the list of areas that we calculated.  This allows us to reuse the
	 * same areas on a subsequent run without having to re-calculate them.
	 *
	 * @param filename The filename to write to.
	 */
	public void write(String filename) {
		try (Writer w = new FileWriter(filename);
				PrintWriter pw = new PrintWriter(w);) {
			pw.println("# List of areas");
			pw.format("# Generated %s%n", new Date());
			pw.println("#");

			for (Area area : areas) {
				pw.format(Locale.ROOT, "%08d: %d,%d to %d,%d%n",
						area.getMapId(),
						area.getMinLat(), area.getMinLong(),
						area.getMaxLat(), area.getMaxLong());
				pw.format(Locale.ROOT, "#       : %f,%f to %f,%f%n",
						Utils.toDegrees(area.getMinLat()), Utils.toDegrees(area.getMinLong()),
						Utils.toDegrees(area.getMaxLat()), Utils.toDegrees(area.getMaxLong()));
				pw.println();
			}

		} catch (IOException e) {
			System.err.println("Could not write areas.list file, processing continues");
		}
	}

	public void read(String filename) throws IOException {
		String lower = filename.toLowerCase();
		if (lower.endsWith(".kml") || lower.endsWith(".kml.gz") || lower.endsWith(".kml.bz2")) {
			readKml(filename);
		} else {
			readList(filename);
		}
	}

	/**
	 * Read in an area definition file that we previously wrote.
	 * Obviously other tools could create the file too.
	 */
	private void readList(String filename) throws IOException {
		areas.clear();

		Pattern pattern = Pattern.compile("([0-9]{8})[ ]*:" +
		"[ ]*([\\p{XDigit}x-]+),([\\p{XDigit}x-]+)" +
		" to ([\\p{XDigit}x-]+),([\\p{XDigit}x-]+)");

		try (Reader r = new FileReader(filename);
				BufferedReader br = new BufferedReader(r)) {			
			String line;
			while ((line = br.readLine()) != null) {
				line = line.trim();
				if (line.isEmpty() || line.charAt(0) == '#')
					continue;

				try {
					Matcher matcher = pattern.matcher(line);
					matcher.find();
					String mapid = matcher.group(1);

					Area area = new Area(
							Integer.decode(matcher.group(2)),
							Integer.decode(matcher.group(3)),
							Integer.decode(matcher.group(4)),
							Integer.decode(matcher.group(5)));
					if (!area.verify())
						throw new IllegalArgumentException("Invalid area in file "+ filename+ ": " + line);
					area.setMapId(Integer.parseInt(mapid));
					areas.add(area);
				} catch (IllegalStateException e) {
					throw new IllegalArgumentException("Cannot parse line " + line);
				}
			}
		} catch (NumberFormatException e) {
			throw new IllegalArgumentException("Bad number in areas list file");
		}
	}

	private void readKml(String filename) throws IOException {
		try {
			KmlParser parser = new KmlParser();
			parser.setReader(Utils.openFile(filename, false));
			parser.parse();
			List<Area> newAreas = parser.getAreas();
			areas.clear();
			areas.addAll(newAreas);
		} catch (XmlPullParserException e) {
			throw new IOException("Unable to parse KML file " + filename, e);
		}
	}

	public List<Area> getAreas() {
		return Collections.unmodifiableList(areas);
	}

	public void dump() {
		System.out.println("Areas read from file");
		for (Area area : areas) {
			System.out.println(area.getMapId() + " " + area.toString());
		}
	}
	
	public void dumpHex() {
		System.out.println(areas.size() + " areas:");
		for (Area area : areas) {
			System.out.format("Area %08d: %d,%d to %d,%d covers %s", area.getMapId(), area.getMinLat(),
					area.getMinLong(), area.getMaxLat(), area.getMaxLong(), area.toHexString());

			if (area.getName() != null)
				System.out.print(' ' + area.getName());
			System.out.println();
		}
	}

	/**
	 * Write out a poly file containing the bounding polygon for the areas 
	 * that we calculated. 
	 *
	 * @param filename The poly filename to write to.
	 */
	public void writePoly(String filename) {
		java.awt.geom.Area polygonArea = new java.awt.geom.Area();
		for (Area area : areas) {
			polygonArea.add(new java.awt.geom.Area(Utils.area2Rectangle(area, 0)));
		}
		List<List<Point>> shapes = Utils.areaToShapes(polygonArea);
		// start with outer polygons
		Collections.reverse(shapes);
		
		try (PrintWriter pw = new PrintWriter(filename)) {
			pw.println("area");
			for (int i = 0; i < shapes.size(); i++) {
				List<Point> shape = shapes.get(i);
				if (Utils.clockwise(shape))
					pw.println(i + 1);
				else
					pw.println("!" + (i + 1));
				Point point = null;
				for (int j = 0; j < shape.size(); j++) {
					point = shape.get(j);
					if (j > 0 && j + 1 < shape.size()) {
						Point lastPoint = shape.get(j - 1);
						Point nextPoint = shape.get(j + 1);
						if ((point.x == nextPoint.x && point.x == lastPoint.x)
								|| (point.y == nextPoint.y && point.y == lastPoint.y))
							continue;
					}
					pw.format(Locale.ROOT, "  %f  %f%n", Utils.toDegrees(point.x), Utils.toDegrees(point.y));

				}
				pw.println("END");
			}
			pw.println("END");
		} catch (IOException e) {
			System.err.println("Could not write polygon file " + filename + ", processing continues");
		}
	}
	
	/**
	 * Write a file that can be given to mkgmap that contains the correct arguments
	 * for the split file pieces.  You are encouraged to edit the file and so it
	 * contains a template of all the arguments that you might want to use.
	 */
	public void writeArgsFile(String filename, String outputType, int startMapId) {
		try (PrintWriter w = new PrintWriter(new FileWriter(filename))){

			w.println("#");
			w.println("# This file can be given to mkgmap using the -c option");
			w.println("# Please edit it first to add a description of each map.");
			w.println("#");
			w.println();

			w.println("# You can set the family id for the map");
			w.println("# family-id: 980");
			w.println("# product-id: 1");

			w.println();
			w.println("# Following is a list of map tiles.  Add a suitable description");
			w.println("# for each one.");
			int mapId = startMapId;
			if (mapId % 100 == 0)
				mapId++;
			for (Area a : areas) {
				w.println();

				w.format("mapname: %08d%n", (startMapId <0) ? a.getMapId() : mapId++);
				if (a.getName() == null)
					w.println("# description: OSM Map");
				else
					w.println("description: " + (a.getName().length() > 50 ? a.getName().substring(0, 50) : a.getName()));
				String ext;
				if("pbf".equals(outputType))
					ext = ".osm.pbf";
				else if("o5m".equals(outputType))
					ext = ".o5m";
				else
					ext = ".osm.gz";
				w.format("input-file: %08d%s%n", a.getMapId(), ext);
			}
			w.println();
		} catch (IOException e) {
			throw new SplitFailedException("Could not write template.args file " + filename, e.getCause());
		}
	}
	
	private static CityFinder cityFinder = null;

	public void setAreaNames() {
		synchronized (this) {
			if (geoNamesFile != null){
			CityLoader cityLoader = new CityLoader(true);
			List<City> cities = cityLoader.load(geoNamesFile);
			if (cities == null)
				return;

			cityFinder = new DefaultCityFinder(cities);
			}
		}
		for (Area area : getAreas()) {	
			City bestMatch = null;
			if (cityFinder != null) {

				// Decide what to call the area
				Set<City> found = cityFinder.findCities(area);

				// Select largest population city in area/
				for (City city : found) {
					if (bestMatch == null || city.getPopulation() > bestMatch.getPopulation()) {
						bestMatch = city;
					}
				}
			}
			area.setName(
				(bestMatch == null
					? description
					: bestMatch.getCountryCode() + '-' + bestMatch.getName()) +
				":"+plusCode(area));
		}
	}

	/**
	 * 
	 * @param mapId
	 */
	public void setMapIds(int mapId) {
		for (Area area : getAreas()) {
			area.setMapId(mapId++);
		}
	}

	public void setGeoNamesFile(String geoNamesFile) {
		this.geoNamesFile = geoNamesFile;
	}

	public void setAreas(List<Area> calculateAreas) {
		areas.clear();
		areas.addAll(calculateAreas);
	}
	
	/**
	 * 
	 * @param fileOutputDir
	 * @param polygons
	 * @param kmlOutputFile
	 * @param outputType
	 */
	public void writeListFiles(File fileOutputDir, List<PolygonDesc> polygons, String kmlOutputFile,
			String outputType) {
		for (PolygonDesc pd : polygons) {
			List<uk.me.parabola.splitter.Area> areasPart = new ArrayList<>();
			for (uk.me.parabola.splitter.Area a : areas) {
				if (pd.getArea().intersects(a.getRect()))
					areasPart.add(a);
			}
			if (kmlOutputFile != null) {
				File out = new File(kmlOutputFile);
				String kmlOutputFilePart = pd.getName() + "-" + out.getName();
				if (out.getParent() != null)
					out = new File(out.getParent(), kmlOutputFilePart);
				else
					out = new File(kmlOutputFilePart);
				if (out.getParent() == null)
					out = new File(fileOutputDir, kmlOutputFilePart);
				KmlWriter.writeKml(out.getPath(), areasPart);
			}
			AreaList al = new AreaList(areasPart, null);
			al.setGeoNamesFile(geoNamesFile);
			al.writePoly(new File(fileOutputDir, pd.getName() + "-" + "areas.poly").getPath());
			al.writeArgsFile(new File(fileOutputDir, pd.getName() + "-" + "template.args").getPath(), outputType,
					pd.getMapId());
		}
	}
	
	private static final String codes = "23456789CFGHJMPQRVWX";
	
	private static final double GarminFactor = 360.0/((double)(1<<24));
	
	private String plusCode(Area area) {
		double south = area.getMinLat()*GarminFactor;
		double west = area.getMinLong()*GarminFactor;
		double north = area.getMaxLat()*GarminFactor;
		double east = area.getMaxLong()*GarminFactor;
		double centerLatitude = 0.5 *(south + north)+90.;
		double centerLongitude = 0.5 * (west + east)+180.;
		double boxHeight = north - south;
		double boxWidth = east - west;
		double height = 400.;
		double width = 400.;
		StringBuilder code = new StringBuilder();
		boolean done = false;
		while(code.length() < 8  & !done) {
			int p;
			height /= 20.;
			width /= 20.;
			p = (int)Math.floor(centerLatitude/height);
			code.append(codes.charAt(p));
			centerLatitude -= p * height;
			p = (int)Math.floor(centerLongitude/width);
			code.append(codes.charAt(p));
			centerLongitude -= p * width;
			done = width <= boxWidth && height <= boxHeight;
		}
		while(code.length() < 8) code.append("00");
		code.append("+");
		while(!done) {
			int la, lo;
			height /= 5.;
			width /= 4.;
			la = (int)Math.floor(centerLatitude/height);
			centerLatitude -= la * height;
			lo = (int)Math.floor(centerLongitude/width);
			centerLongitude -= lo * width;
			code.append(codes.charAt(4*la+lo));
			done = width <= boxWidth && height <= boxHeight;
		}
		return code.toString();
	}

}

---

• Previous message: [mkgmap-dev] splitter and options --description / --geonames-file
• Next message: [mkgmap-dev] splitter and options --description / --geonames-file
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

---

More information about the mkgmap-dev mailing list