I have a huge list of symptoms (1 million rows+ ). I would like to reduce the data by grouping the symptoms. For example, lower abdominal pain and upper abdominal pain are in the "abdominal pain" category. Is there any free API or downloadable database available to refer to the hierarchy of diseases and symptoms for grouping?
The NIH has published the UMLS database, which consists of more than 7 million concepts, diseases and symptoms. It's a very wealthy resource. The license is pretty permissive if you are working in the United States. Check it out on the UMLS website. It's completely free and curated by the National Library of Medicine.
I am not exactly sure that this is what you are referring to, but the MEDDRA database costs money in most cases, but I believe there are some research licenses. It is a hierarchy of indications.
Hope that helps.
Have a look at the supplementary material of Learning disease relationships from clinical drug trials.
disease_mappings10.txt file in the supplementary contains the info you need.
For instance, if you do a search (e.g.,
grep "lower abdominal pain" disease_mappings10.txt), you will see
lower abdominal pain abdominal pain Approximate