8

The World Factbook [1][2] published by the Central Intelligence Agency (CIA) offers 267 free country 'n' territories profiles (incl. flags 'n' overview and locator maps) in the public domain (that is, no copyright(s), no rights reserved).

Unfortunately, all the "data" is published as web pages and only some stat series (e.g. area, population, GNP, etc.) can get downloaded as tab-delimited datasets - the FAQ entry reads:

Q: Is The World Factbook country data available in machine-readable format? All I can find is HTML, but I’m looking for simple tabular data. A: The Factbook Web site now features Country Comparison pages for selected Factbook entries. All of the Country Comparison pages can be downloaded as tab-delimited data files that can be opened in other applications such as spreadsheets and database.

Anyway, any projects out there that offer the World Factbook in alternative formats (e.g. structured datasets in JSON etc.) for download? Any insight appreciated.

Disclaimer: I'm the coder of the factbook Ruby gem that lets you turn World Factbook pages into structured data (e.g. JSON etc.) - see the factbook.ruby and factbook.json repos on GitHub.

  • So are you looking for a better way to web scrape the CIA World Factbook? It seems pretty definitive from the FAQ item you quoted that the formats are not available directly from the CIA's site. – Mark Silverberg Sep 14 '14 at 14:41
  • I'm looking for alternative projects that offer datasets for the World Factbook pages - of course - first you need to parse/read/scrape the original web page and turn it into something more structured, that is, easier to process for converting to different formats. Any links / comments about alternative web scrapers are welcome too. Cheers. – Gerald Bauer Sep 14 '14 at 15:23
  • I just checked out the Download [raw - tab delimited] Data option added to the country comparison fields. This is great. I don't have to scrap the WFB anymore! – Andrew - OpenGeoCode Sep 19 '14 at 17:28
7

You can use factbook for data. A simple google search can get you raw json or csv.

Officially you can use their tool : https://github.com/factbook/factbook

or download data directly from CIA website :

https://www.cia.gov/library/publications/download/

Previous answer :

The links are dead - looks like data has been removed without any notice.

Check this out. https://github.com/factbook/factbook.json it has all data in json - I recommend using nosql database for importing.

They also offer sql dump https://github.com/factbook/factbook.sql

CSV with facts. https://github.com/factbook/factbook.csv

  • It seems to get quite regular updates so far (Jul'14-Nov'15). Nice! – ojdo Dec 14 '15 at 12:16
  • I was searching data for myself..Found this question....So i thought to update when i found a good solution, incase someone else needed. – Madhurendra Sachan Dec 15 '15 at 10:04
  • 1
    The last link seems to be broken. – Philipp Mar 19 '17 at 13:33
  • I think it's an important distinction to make that the GitHub repository is not owned or maintained by the CIA and thus not kept up-to-date with the latest releases. – Alex W Oct 25 at 16:01
4

I wrote some software awhile back ago that parses many of the WFB web pages and converted the data into CSV files. You can find them here:

http://www.opengeocode.org/cude1.1/CIA/WFB/index.php

3

For factbook data which has been parsed into numbers / arrays etc (unlike any existing projects which store all values as strings) have a look at https://github.com/iancoleman/cia_world_factbook_api#data

There's some explanation of the project at https://iancoleman.github.io/exploring-the-cia-world-factbook/

Disclaimer: I am the maintainer of that project.

1

This guy scrapes weekly and has maybe the most recent version

https://github.com/iancoleman/cia_world_factbook_api

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.