Looking for open dataset containing data for disease and symptoms

Question

I am doing a data mining project on "health prediction system". So, Is there any open dataset containing data for disease and symptoms.

Answer 1

Maybe you can try extract it from PubMed.

Or maybe you can use/refer to this paper by Barabasi. Got some data on the supplement section. http://www.nature.com/ncomms/2014/140626/ncomms5212/abs/ncomms5212.html#figures

Answer 2

JATS articles are "open dataset" in that sense... The main source at PubMed Central is the bulk data at http://www.ncbi.nlm.nih.gov/pmc/tools/ftp/

But your approach must start by the "candidate article selection", because the universe is so big... "Containing data for disease and symptoms" is so big...

---

(aleatory) Examples of cientific paper with data about "disease and symptoms": http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3533064/ or PMC3381271

Procedure to get the full XML and supplementary raw data:

1. get the list of all articles that are in the FTP of PubMed Central

2. filter articles by ID (in this example is PMC3706994) or a list of articles by journal name, volume, etc.

3. download it

4. analyse the XML

So, in a UNIX/Linux terminal

   # step 1
   wget -c ftp://ftp.ncbi.nlm.nih.gov/pub/pmc/file_list.txt

   # step 2
   cat file_list.txt | grep PMC3381271
   # or cat file_list.txt | grep -E BMJ.Open.2012.+19.2.6.+1977 | more

   # step 3     
   wget -c "ftp://ftp.ncbi.nlm.nih.gov/pub/pmc/e1/ce/BMJ_Open_2012_Nov_19_2(6)_e001977.tar.gz"
   tar -vzxf tar -vzxf BMJ_Open_2012_Nov_19_2\(6\)_e001977.tar.gz

   # step 4
   more BMJ_Open_2012_Nov_19_2\(6\)_e001977/bmjopen-2012-001977.nxml

PS: tables are in XHTML, into the <table-wrap> JATS tags, other content and metadata in other JATS tags.

Answer 3

There is an international coding system that lists and codes an enormous range of diseases/symptoms called ICD10. This is how Wikipedia describes it:

ICD-10 is the 10th revision of the International Statistical Classification of Diseases and Related Health Problems (ICD), a medical classification list by the World Health Organization (WHO). It contains codes for diseases, signs and symptoms, abnormal findings, complaints, social circumstances, and external causes of injury or diseases. Work on ICD-10 began in 1983 and was completed in 1992.

The data behind it is available from the CDC here. The most useful tables are probably 2018-ICD10-code-order-descriptions if you just want the lists, but there are plenty of other files explaining how the system works which may also be useful.

Other countries used variants of the same basic scheme, and those are available from country-specific sites. The WHO also make much of the information available online.

Answer 4

This url leads to health data: http://catalog.data.gov/dataset?_organization_limit=0&organization=hhs-gov#topic=health_navigation