Is there a free csv, xml or in some other format database of all (or least top 20-50 biggest) cities for each country in the world?
My open data project (I am a co-founder) has a free list of all the cities in the world, along with their area centroid (lat/lng), as a CSV file. It is compiled from the USGS/GNIS (US) and NGA/GNS (non-US) databases.
As an alternate source, the United Nations Statistical Division publishes an annual yearbook on world statistics. Table 8 has the population of cities > 100,000
We have a version of it converted to ur Linked CSV format/vocabulary:
- ISO 3166-1 alpha-2 country code (e.g., US => United States)
- National Geospatial Intelligence Agency (NGA) Geographic Name Server (GNS) Feature Code (e.g., P = Populated Place Type Feature)
- NGA/GNS Feature Designation Code (e.g. PPL = Populated Place (incorporated))
- Extended Feature Description (e.g., city, capital)
- Total Area in Square Kilometers
- ISO 639-1 language code for language that name field is in (e.g., lc = local language native to the country)
- Language Script for name fields (e.g., latin, arabic, chinese)
- Short Name (Gazetteer) for City
- Year of Population Statistics
- Total Population (e.g., within city proper)
- Urban Population (e.g., within agglomerated area of city)
- Total Male Population
- Total Female Population
Consider geonames, which probably has the largest collection of place names anywhere (excluding street names, which is the purview of openstreetmap.org):
From the directory above, you can download a list of "large" cities (or every single placename geonames knows about), and "readme.txt" in that directory explains further.
UN/LOCODE includes over 103,034 locations in 249 countries and installations in international waters. It is used by most major shipping companies, by freight forwarders and in the manufacturing industry around the world. It is also applied by national governments and in trade related activities, such as statistics where it is used by the European Union, by the UPU for certain postal services, etc.
The highest quality dataset that I've seen is Natural Earth. About 7,000 cities total so you would need to filter it down if you don't want that many