I've worked on a simulation of understanding the rate at which new concepts are found, and I'm looking for a dataset to test this on. I looked at this paper and they tested some of their hypotheses by developing clusters out of single chemicals annotated from MEDLINE publications. What they did is non-trivial and frankly out of the scope of my understanding. But I need something similar. I want to look at journal papers in some year range, and look at how many new concepts were found in that time period and how they are associated with each other. Because if a new concept is found in an area that is relatively new, it often leads to multiple new concepts found in that same general area in a short period of time, thus leading to a new cluster. This can be seen in the following figure from the attached paper. enter image description here

As new chemicals related to DNA were found, slowly a cluster of DNA and Molecular Biology was formed. Thus, in the above figure, I'd look at the number of clusters and the number of nodes (chemicals) in each cluster at a year, say 1990, and then at say 1996. The number of new nodes (chemicals) found in that period would form my dataset, and I'd try to set my parameters such that my simulation finds all those nodes in the correspondingly correlated time period. Are there any datasets that allow me access to such information of concepts in the form of clusters? If not, is there a relatively easy way to form such a dataset of clusters?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.