The Stack Overflow podcast is back! Listen to an interview with our new CEO.
24

Sci-Hub is a paywall-bypassing website that uses "shared" user credentials to provide PDF or HTML scientific papers. The website itself doesn't store any papers. (There are interesting comments on your same question on another site.) But LibGen (via wayback machine) is said to archive each PDF retrieved by Sci-Hub. http://gen.lib.rus.ec has a downloads ...


15

There are several different potential sources of information. I don't think any are completely comprehensive and few would count as strict "open data": apart from Open Access titles, licensing is likely to vary between publishers. Having said that you could look at some of the following sources: Springer API Nature Linked Data Platform PubMed API PLOS API ...


14

There are several studies on the economical impact of Open Data. The most recent I know of is a study done in 2011 on data held by all public bodies in the European Union called Review of Recent PSI Re-Use Studies Published [docx] (PSI stands for Public Sector Information), the study is also know as the Vickery study. One of its main findings is that the EU'...


13

The University of California, Irvine provides a dataset repository specifically for machine learning purposes. There are currently 239 datasets in the repository. These datasets come in many different formats and topics. The oldest datasets in the repository date back to the late 80s, and there are some datasets that are from 2013.


9

One of the latest published reviews / reports (a few days ago) is the Shakespeare review which contains a chapter on evidence page 20pp: "...This figure comprises direct economic benefits estimated at around £1.8bn, and a wider social value of PSI conservatively estimated in excess of £5bn..." There are various comments on this report online from Guardian ...


8

The the arXiv repository contains almost 1 million of scientific articles, whose bibliographic information is accessible through a documented API, including links to other papers.


7

Hilary Mason (a data scientist at bit.ly and speaker on the topic of Machine Learning) put together a list of "Research Quality" datasets in a bit.ly bundle specifically for this reason https://bitly.com/bundles/hmason/1 https://web.archive.org/web/20150320022752/https://bitly.com/bundles/hmason/1 It has some interesting datasets from a variety of ...


7

The National Phenology Network has data for flowering dates (and other phenological characteristics) for trees in the United States. Using their download tool, you can specify a season of interest (if you select "Other" under Year Options, you can narrow it down to specific dates) and a range of species (selecting all the broadleaf and conifer Functional ...


6

There is also PLOS Public Library of Science http://api.plos.org/solr/faq/#solr_api_examples PLOS is open access.


6

CiteSeerX and DBPL are two of the most commonly used data sets used in the academic literature.


6

I understand your question and it does relate to open data. It seems like you have a piece of open data: municipal service requests (i.e. "fix the pothole in front of my house!"). Your followup question is a good one: given the number of service requests, are these people just cranky, or are there actually more needed items to be fixed in a certain area? I ...


6

This wiki page from linkedgov.org, The economic impact of open data is: ...collecting case studies and references to the economic impact of open data. The World Bank Knowledge Repository also has a collection of economic impact studies in the "How would my country benefit from Open Data?" section.


6

Restricting this question to "Computer Science" (as suggested in the comments) and reducing "all possible" to the ones which were actually published, one resource is The DBLP Computer Science Bibliography. Note that this source follows ODC-BY 1.0. The DBLP has a search interface with an attempt to identify coauthor communities. They also offer bulk ...


6

Data Mendeley does not seem to grant me any license to reuse the content (in particular the academic papers uploaded by other users), so by default the data must be considered closed: We are not allowed to redistribute it. Even if an API allows us to retrieve it: You may not use our Services to [...] download, use or re-use any Academic Papers without ...


6

Maybe the rcrossref package for R is helpful. To find the number of citations, You can do things such as library(rcrossref) cr_search(doi = "10.1371/journal.pone.0042793", year="2012") cr_citation_count(doi = "10.1371/journal.pone.0042793") The API will only work for CrossRef DOIs. This means that the only works that can be searched for must have been ...


6

The first step is splitting the image into character arrays. To do that, check out the answers in this question: Separate image of text into component character images. In particular, the ImageMagick answer from 2015. (If you can determine how the input is given, then collect the characters as separate images.) To convert the image into a 2D array, you can ...


5

Open Data Barometer recently published this: http://www.opendatabarometer.org/report/analysis/impact.html also, the open data working group under the Open Government Partnership will be looking at impact: http://www.opengovpartnership.org/groups/opendata/


5

http://libguides.mit.edu/apis gives a very nice list of APIs for scholarly resources, here is snapshot of the page in case it disappears:


5

First to be able to better answer: what is your definition of open scientific publications? Second: there is for sure nothing comparable to Wikipedia or so in terms of cross-disciplinarity and amount of content. Most of this data is in the hands of big (publishing) companies. There is no central register for scientific literature. One of the best ways is ...


5

First, you should check for any license for using the site to see if you can use their data for any reason (commercial and non-commercial). Second, you should read the terms of use and find anything about scraping website. Many websites don't allow scraping. Third, if non of the above is a problem, you should use a programming language to crawl and scrap ...


4

There are, but each system has a different scope, and so only collect up articles that they're interested in. For instance: SAO/NASA Astrophysics Data System focuses on astronomy and related fields PubMed focuses on [biomedical]. Any institutional repository collects papers authored by their staff or faculty. If you're asking about standards for releasing ...


4

Yes there is- I was part of a small review with Code for America folks of the McKinsey report looking at global value of open data- it's now published here: Open data: Unlocking innovation and performance with liquid information


4

Pubmed has data about 23 million biology and medicine papers. Unfortunately I think you need to request access and given US politics at the moment it could take some time till there a human available to give you that access.


4

EDIT 2017 PubMed Central changed all adressess and some methods. Only tips: To bulk access you can use complex FTP procedures similar to the described below in the "OLD" section... Or a simple perl (or shell) script to loop over API. The basic GET API are showed at https://www.ncbi.nlm.nih.gov/pmc/tools/get-full-text/ To GET by shell you must to use ...


4

I would suggest that you scrape the data on SSRN by looking by field/location/date.


4

Many of the bibliographic databases offer APIs, but they might not be 100% open. Typically, you can't get access to the ones that charge for access unless your institution has a subscription to the service. I know that link-only answers are bad, but the problem is that policies change over time, and the folks at MIT libraries has a rather long list of ...


4

Without being a trained copyright expert, the crux of your endevour lies in your process of aggregating your text corpus: Over the years, I collected texts from many different internet sources (Blogs, Forums, Amazon, News portals, Project Gutenberg, etc.) by my own, which I aggregated and compiled into various data sets that could be very valuable for the ...


4

Take a look at my answer here. Basically, it goes over a primary manufacturing technology that allows collecting data from various machine tools. However, the same MTConnect "standard" can be applied to anything that can be digitally monitored using sensors. Take a look at this "live" (but fake) XSLT stream from a 3-Axis mill.


4

Some nice data sets for practicing sentiment classification are: Sentiment 140 Tweets2011 Dataset by Sanders Another source This one on Github One from a Kaggle contest EMOTIONAL SENSOR DATA SET 1.0.8 These are some open datasets which contain emotions like happy, sad, etc: Affective Sciences (Data in .sav data files)


4

The Reddit thread Can I download all libgen books? describes how to download the entire Library Genesis database (~10TB).


Only top voted, non community-wiki answers of a minimum length are eligible