9

I am looking for a dataset describing (nearly) all languages, and where they are spoken.

Bonus if the areas are linked to Geonames or another mainstream geographical dataset. Polygons are also OK.

It does not need to be a 1-1 mapping, as many places have several languages.

Official languages is OK, but actual usage (more than 1% of the locality's population has it as its mother tongue) of non-official languages would be even greater.
I am looking for sub-country granularity.

6 Answers 6

6

The CIA Word Factbook has a field listing for languages spoken per country and percentage of population that speaks it.

https://www.cia.gov/library/publications/the-world-factbook/fields/2098.html

For finer grain, you can Google 'language spoken home dataset'. This will find census datasets related to the distribution of languages spoken. Below are some examples:

United States (Census) by state:
https://www.census.gov/hhes/socdemo/language/data/other/detailed-lang-tables.xls

City of Chicago, IL by neighborhood:
https://data.cityofchicago.org/Health-Human-Services/Census-Data-Languages-spoken-in-Chicago-2008-2012/a2fk-ec6q

City of Cambridge, MA by neighborhood:
https://data.cambridgema.gov/Neighborhood-Census-Data/2007-2011-Language-Spoken-at-Home-by-Neighborhood/sba5-5kgg

Madison County, NY
http://cnyvitals.org/madison/demographics/language-spoken-at-home/

State of Hawaii:
https://data.hawaii.gov/Culture-and-Recreation/English-ability-by-language-spoken-at-home/i9hq-hna6

Canada (from their Census) by Province, Electoral District:
http://data.gc.ca/data/en/dataset/81a2bd6e-622f-4f17-84c1-215216485992

Queensland, AU by statistical area:
https://data.qld.gov.au/dataset/language-spoken-home-sa4-qld/resource/9fa1fc3a-ab09-4c99-a60f-f95c59269492

Greater London by borough:
http://publicdata.eu/dataset/first-language-spoken-at-home-borough0e333

UNESCO Atlas of the World's Languages in Danger:
http://www.unesco.org/culture/languages-atlas/

2
4

Countries of the World Includes country language attribute for 249 countries - I believe either point or poly; has points for admin1 if you are looking subnational

Global Database of Events, Language, and Tone Uses the CAMEO taxonomy which I believe includes language but you'll have to check

6
  • thanks for promoting my own dataset (Countries of the World). Commented Oct 1, 2014 at 15:17
  • I saw you responded without it but decided to add it anyway because someone else might find it useful. Won't happen again.
    – KPayne
    Commented Oct 1, 2014 at 17:32
  • 1
    gdelt.utdallas.edu seems down. Did you mean gdeltproject.org maybe?
    – Nicolas Raoul
    Commented Oct 2, 2014 at 3:08
  • "Won't happen again" <- No, please do it again, promoting someone else's open data is a good thing :-)
    – Nicolas Raoul
    Commented Oct 2, 2014 at 3:09
  • COW is subnational for the Americas, with for each one "the official state language", that's a good start.
    – Nicolas Raoul
    Commented Oct 2, 2014 at 3:12
3

The go-to site for linguists who are looking for language data (including e.g. their families, iso codes, position etc.) is glottolog.org. They have all their data available for download in several formats. wals.info is also useful, but that one uses a more restricted set of languages that might contain less sub-national languages.

2

Some time ago I made a csv file with ISO 3 language codes, latitude, and longitude of more than 7000 languages. The data was extracted from the RDF graph downloaded at Glottolog and sanitized (some coordinates missed a decimal point).

Example entry:

gui;Eastern Bolivian Guarani;-61.7179;-21.0909;;

You can download the file iso3_language_codes_with_geo_coordinates.csv from this Github repository.

License: http://creativecommons.org/licenses/by-sa/3.0/

Update:

I just stumbled on the "World Atlas of Language Structures" on Kaggle, containing data on the linguistic structures in 2,679 languages, including geo coordinates for each language.

1
  • Nice! One single location per language, right? Giving a point for a language like Western Yiddish sounds tricky. What is the license?
    – Nicolas Raoul
    Commented Feb 29, 2016 at 1:54
1

Splitting this into 2 posts so I can post more than 2 links. I would look at

Languages of the world Last time I checked data was bound in a pdf but that may have changed. For each country, details of each language including the region (in prose).

Indigenous living languages from Worldmapper Primarily sourced from ethnologue. Indigeneous languages of each country as Excel file, no sub-country data.

2
  • First link: I can't find any PDF. Actually the data seems copyrighted and not open. Data as books are available for purchase.
    – Nicolas Raoul
    Commented Oct 2, 2014 at 3:05
  • Right again - looks like in addition to books you can purchase data as shapefiles (.dbf) tied to DCW worldgeodatasets.com/language
    – KPayne
    Commented Oct 2, 2014 at 11:44
1

You may want to look at the language data from Ethnologue at http://www.ethnologue.com

Two caveats:

  • The data are released under a "fair use" licence which may meet your requirements, but technically is not Open Data
  • You may agree or disagree with the judgements of the Ethnologue on the language/dialect division

Example in Papua New Guinea: Papua New Guinea

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.