6

Does anyone know where I can find the dataset of all possible research publications ? I would like to get only titles and abstracts for those papers.

  • one research field or every research field? – philshem Mar 10 '14 at 8:54
  • 1
    Do you really mean "all possible"? Even "all" would be difficult to answer without some limitations (e.g., in peer-reviewed journals). – igelkott Mar 10 '14 at 10:36
  • For now I want to deal with Computer Science research papers only. I know "all possible" is a bit too much, but as many as I can get will be great. – tazo Mar 10 '14 at 16:08
  • 1
    Try Crossref. Here's their web services labs.crossref.org – sckott Mar 11 '14 at 6:30
  • After the Aaron Swartz incident, JSTOR started allowing people access to their corpus for text mining -- I don't know if they'd necessarily allow this sort of access, though. – Joe Jun 19 '14 at 16:39
6

Restricting this question to "Computer Science" (as suggested in the comments) and reducing "all possible" to the ones which were actually published, one resource is The DBLP Computer Science Bibliography. Note that this source follows ODC-BY 1.0.

The DBLP has a search interface with an attempt to identify coauthor communities. They also offer bulk downloads in XML format.

Some other research subjects have good sources, such as PubMed for biology, medicine, etc.

5

First to be able to better answer: what is your definition of open scientific publications?

Second: there is for sure nothing comparable to Wikipedia or so in terms of cross-disciplinarity and amount of content. Most of this data is in the hands of big (publishing) companies. There is no central register for scientific literature. One of the best ways is to get bibliographic data and find the DOI (digital object identifier). With this, you can search in different databases for it.

But I know some big open repositories, which have a lot of open access:

Another good spot to look for Open Access publications is the Directory of Open Access Journals (DOAJ).

3

BASE (Bielefeld academic search engine) is one of the world's most voluminous search engines especially for academic open access web resources. It claims 80 million documents, and offers an API interface (with registration).

1

"all possible research publications" is way too broad of a search.

It's estimated that there are over 26,000 peer reviewed journals, and 21,000 'non-journals', not to mention all of the journals that aren't peer reviewed.

It's possble that some of those only published one issue, and went defunct ... but even then the number of articles is huge

Searching Google Scholar for the letter 'a' yields 8,460,000 results. The Astrophysics Data System is at least 9,250,000 records based on a poster from this week's LISA conference.

To start, I'd probably try to use the various disciplinary bibliographic systems -- ADS for astronomy & physics, and PubMed Central for biology & health.

For books, you might try OCLC, but I think their 'open license' applies to their members, not necessarily the general public. ... that also wouldn't necessarily have 'abstract' info, except for a few phrases or sentence you might in the front matter of more recent books books.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.