29

Sci-Hub is a paywall-bypassing website that uses "shared" user credentials to provide PDF or HTML scientific papers. The website itself doesn't store any papers. (There are interesting comments on your same question on another site.) But LibGen (via wayback machine) is said to archive each PDF retrieved by Sci-Hub. http://gen.lib.rus.ec has a downloads ...


8

The the arXiv repository contains almost 1 million of scientific articles, whose bibliographic information is accessible through a documented API, including links to other papers.


7

Maybe the rcrossref package for R is helpful. To find the number of citations, You can do things such as library(rcrossref) # returns list: # pap <- cr_works(dois = "10.1371/journal.pone.0042793") # pap$data will be 28-column data frame. cr_citation_count(doi = "10.1371/journal.pone.0042793") Result: doi ...


7

The National Phenology Network has data for flowering dates (and other phenological characteristics) for trees in the United States. Using their download tool, you can specify a season of interest (if you select "Other" under Year Options, you can narrow it down to specific dates) and a range of species (selecting all the broadleaf and conifer Functional ...


6

There is also PLOS Public Library of Science http://api.plos.org/solr/faq/#solr_api_examples PLOS is open access.


6

CiteSeerX and DBPL are two of the most commonly used data sets used in the academic literature.


6

Restricting this question to "Computer Science" (as suggested in the comments) and reducing "all possible" to the ones which were actually published, one resource is The DBLP Computer Science Bibliography. Note that this source follows ODC-BY 1.0. The DBLP has a search interface with an attempt to identify coauthor communities. They also offer bulk ...


6

Data Mendeley does not seem to grant me any license to reuse the content (in particular the academic papers uploaded by other users), so by default the data must be considered closed: We are not allowed to redistribute it. Even if an API allows us to retrieve it: You may not use our Services to [...] download, use or re-use any Academic Papers without ...


6

The first step is splitting the image into character arrays. To do that, check out the answers in this question: Separate image of text into component character images. In particular, the ImageMagick answer from 2015. (If you can determine how the input is given, then collect the characters as separate images.) To convert the image into a 2D array, you can ...


5

First to be able to better answer: what is your definition of open scientific publications? Second: there is for sure nothing comparable to Wikipedia or so in terms of cross-disciplinarity and amount of content. Most of this data is in the hands of big (publishing) companies. There is no central register for scientific literature. One of the best ways is ...


5

Open Data Barometer recently published this: http://www.opendatabarometer.org/report/analysis/impact.html also, the open data working group under the Open Government Partnership will be looking at impact: http://www.opengovpartnership.org/groups/opendata/


5

http://libguides.mit.edu/apis gives a very nice list of APIs for scholarly resources, here is snapshot of the page in case it disappears:


5

First, you should check for any license for using the site to see if you can use their data for any reason (commercial and non-commercial). Second, you should read the terms of use and find anything about scraping website. Many websites don't allow scraping. Third, if non of the above is a problem, you should use a programming language to crawl and scrap ...


5

The Reddit thread Can I download all libgen books? describes how to download the entire Library Genesis database (~10TB).


4

Pubmed has data about 23 million biology and medicine papers. Unfortunately I think you need to request access and given US politics at the moment it could take some time till there a human available to give you that access.


4

Yes there is- I was part of a small review with Code for America folks of the McKinsey report looking at global value of open data- it's now published here: Open data: Unlocking innovation and performance with liquid information


4

I would suggest that you scrape the data on SSRN by looking by field/location/date.


4

EDIT 2017 PubMed Central changed all adressess and some methods. Only tips: To bulk access you can use complex FTP procedures similar to the described below in the "OLD" section... Or a simple perl (or shell) script to loop over API. The basic GET API are showed at https://www.ncbi.nlm.nih.gov/pmc/tools/get-full-text/ To GET by shell you must to use ...


4

From the ACM terms of usage page To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Send written requests for republication to ACM Publications, Copyright & Permissions at the address above or fax +1 (212) 869-0481 or email permissions@acm.org. Thus, I believe ...


4

If you are affiliated with an academic institution that subscribes to IEEE/ACM material, talk to your library. They may be able to negotiate access on your behalf. Chances are fair it isn't the first such request they've heard.


4

Many of the bibliographic databases offer APIs, but they might not be 100% open. Typically, you can't get access to the ones that charge for access unless your institution has a subscription to the service. I know that link-only answers are bad, but the problem is that policies change over time, and the folks at MIT libraries has a rather long list of ...


4

Take a look at my answer here. Basically, it goes over a primary manufacturing technology that allows collecting data from various machine tools. However, the same MTConnect "standard" can be applied to anything that can be digitally monitored using sensors. Take a look at this "live" (but fake) XSLT stream from a 3-Axis mill.


4

Some nice data sets for practicing sentiment classification are: Sentiment 140 Tweets2011 Dataset by Sanders Another source This one on Github One from a Kaggle contest These are some open datasets which contain emotions like happy, sad, etc: Affective Sciences (Data in .sav data files)


4

You can request access to SFARI, which has some ASD patients with EEG data. SFARI Base You can also request access to ACE. ACE will analyze four key forms of data: • EEG Data • Neuroanatomical Data • Genetics Data • Behavior Data I saw a reference indicating about 50 ASD patients are in the Australian EEG Database. ...


4

Openstreetmap (OSM) is an excellent source of all kinds of vector data on human infrastructure, particularly in urban areas of the USA and Europe. You'll need to figure out which OSM tags for polygons refer to the types of green spaces you are interested in. There are many ways to access OSM data - wholesale country downloads, regions, by tag, etc. Have a ...


3

BASE (Bielefeld academic search engine) is one of the world's most voluminous search engines especially for academic open access web resources. It claims 80 million documents, and offers an API interface (with registration).


3

I looked up the IEEE case: Guests/Members are not permitted to do the following: create an archive of any portion of CSDL; use robots or intelligent agents to access, search and/or systematically download any portion of CSDL; So not looking good for your project. But, if your project has merit, each group may be open to something. I highly ...


3

To update Cristian's answer, with regard to PubMed, you don't need to be from the USA. There are a couple of ways to get the data. 1) PubMed, go to http://www.ncbi.nlm.nih.gov/pubmed/?term=%22all%22%5Bfilter%5D (which today hosts 25M articles - not OpenAccess) choose send to -> file -> abstract or xml. I downloaded the XML which is kind of larger - 298.298....


3

The Open University and the University of Southampton definitely have their hands in open data. For more concrete ideas you'd have to look at the appropriate department (e.g. Computer Science) and find a supervisor.


3

The Weka machine learning toolkit (in Java) introduced the text-based ARFF format (Attribute-Relation File Format) as a proposed standard format for machine learning datasets. Its website has a list of ARFF datasets. Some of them are discussed in the excellent Weka book that is now in its third edition: Witten, Frank, & Hall (2011): „Data mining. ...


Only top voted, non community-wiki answers of a minimum length are eligible