I'm looking for a database of cities in the world with their names in all languages of the world. Where it is possible to download?
-
1Each city's name in every language of the world? That does not exist, there are too many languages, many of which have not even been documented.– gerritMar 2, 2016 at 12:18
-
2i understand, but i need even more languages: 4, 10, 20... Like a: New York City, Nueva York, Нью-Йорк, নিউ ইয়র্ক সিটি, Νέα Υόρκη, Tchiaq York Iniqpak– ParamedicMar 2, 2016 at 13:39
5 Answers
Wikidata has a category called "city" which will give you a list of world cities
https://www.wikidata.org/wiki/Q515
You can use SPARQL to query Wikidata
A sample SPARQL query will give you the city name in the target language
SELECT ?city ?cityLabel ?country ?countryLabel
WHERE
{
?city wdt:P31/wdt:P279* wd:Q515 . # find instances of subclasses of city
?city wdt:P17 ?country # Also find the country of the city
# choose language
SERVICE wikibase:label {
bd:serviceParam wikibase:language "fr" .
}
}
LIMIT 10 # remove this for full list
In this case, the query is returning the French names for all the cities, because of this part:
bd:serviceParam wikibase:language "fr" .
Each city in multiple langauges can be linked by its Q number, for example "Q2801"
Note 0: Someone who really knows SPARQL can probably get all languages in one query
Note 1: there is a download button with links for CSV, JSON, etc
Note 2: here is the list of wikipedia subdomains : https://en.wikipedia.org/wiki/List_of_Wikipedias
-
1wow, beautiful! Now i need think how i can parse all cities results :/ Mar 3, 2016 at 12:15
I guess the Geonames project is the closest answer to your question. You can download the data directly on the website or you can also find it here and download it/use the api.
Cheers
-
Cities on this database not have names in other languages of the world. Mar 2, 2016 at 12:37
-
There is an "Alternate Names" column in it where you've got the cities names in other languages (not every languages obviously but still a lot in some cases)– TerpoMar 2, 2016 at 12:39
-
1You right, but how to compare the name of the city and language? Impossible Mar 2, 2016 at 13:27
-
You may try to use download.geonames.org/export/dump/iso-languagecodes.txt plus download.geonames.org/export/dump/alternateNames.zip Not sure if possible though– TerpoMar 2, 2016 at 13:41
As a variation of above, if you want all the labels in all the languages, you can get it from Wikidata like so:
SELECT DISTINCT ?city ?country (lang(?label) AS ?language) ?label
WHERE
{
?city wdt:P31/wdt:P279* wd:Q515 . # find instances of subclasses of city
?city wdt:P17 ?country # Also find the country of the city
OPTIONAL { ?city rdfs:label ?label }
}
LIMIT 10 # remove this for full list
Just run that on https://query.wikidata.org
-
Great! Can you advise how best to parse all transfers in every city? A lot of results!! Mar 10, 2016 at 14:59
It's not structured data, but for a robust dataset like this there is some value to manually curated data. For that, Wikipedia usually provides pretty good information.
If you take the wiki page for a city in one language
https://en.wikipedia.org/wiki/Anchorage,_Alaska
You'll see that the sub-domain en indicates the page is the English language wikipedia page. If you look on the bottom left menu bar, in a section called "Languages", you'll see links to the same page in different languages.
https://el.wikipedia.org/wiki/Ανκορέιτζ
is the Greek page, where the el sub-domain indicates Greek (Elliniká).
The HTML source of each page has easy-to-find URLs of the corresponding langauges, for example, the source of the English page
<li class="interlanguage-link interwiki-el">
<a href="//el.wikipedia.org/wiki/%CE%91%CE%BD%CE%BA%CE%BF%CF%81%CE%AD%CE%B9%CF%84%CE%B6" title="Ανκορέιτζ – Greek" lang="el" hreflang="el">
Ελληνικά</a>
</li>
So the title="Ανκορέιτζ would give you the multi-language place name.
It will take some programming, but with a list of English language URLs to city or place names, you can make one request for each city and then scrape the corresponding multi-language names out of that HTML.
You can also extract lists of URLs for city names from Wikipedia, for example
https://en.wikipedia.org/wiki/List_of_cities_in_Switzerland
Additionally, all of the Wikimedia sites are available as a database dump, so you could download and parse locally, instead of scraping.
More details: https://opendata.stackexchange.com/a/79/1511
-
Thanks for think, but its not easy way. I not have list of all citynames in a world. And some names of city have a many really city in a world Mar 3, 2016 at 12:35
Yes, it is possible. I am sharing following options that you can use to fetch world cities, their names and languages dataset.
https://www.geonames.org/ - The most credible source to fetch this kind of data.
https://www.back4app.com/database/back4app/list-of-all-continents-countries-cities - It includes all formats in which you can import data related to world cities and their languages. It is also credible because data is imported from the United Nations and GeoNames.
https://data.worldbank.org/ - For the current and updated dataset, the best place is the official data of the World Bank.