5 votes
Accepted

Where can I get a sample dataset for A/B split testing?

Based on the answer for this question on Cross-validated there are two great packages in R that include useful datasets from completed A/B and multi-variate tests: Agricolae and Agridat
wacax's user avatar
  • 1,042
5 votes

ASCII Character Frequency Analysis

You can pretty easily create character (or word) frequency data based on a few lines of code. Here is python 2.7 code that collects counts of each character # -*- coding: utf-8 -*- import string ...
philshem's user avatar
  • 17.6k
5 votes

How to scrape rugby union data from espn.co.uk

You can use python, R or other code language as you prefer: Example for python: https://stackoverflow.com/questions/6325216/parse-html-table-to-python-list Example for R: https://stackoverflow.com/...
GGA's user avatar
  • 275
5 votes

R package for geographic regions

On the software side : an alternative way of getting your data in R would be to find the package that imports spatial datasets. For example rgdal (https://gis.stackexchange.com/questions/19064/how-to-...
gisnside's user avatar
  • 253
5 votes

Seasonal Sales Data for time Series Analysis

The US Census Bureau provides a good data source: U.S. Census Bureau's Monthly & Annual Retail Trade I work at Quandl and here are some databases you might want to check out: US Census Bureau ...
user37317's user avatar
  • 1,036
4 votes

How can I work with a 4GB csv file?

I recently had to parse the 6GB NPPES file and here is how I did it: $ wget http://download.cms.gov/nppes/NPPES_Data_Dissemination_July_2017.zip $ unzip NPPES_Data_Dissemination_July_2017.zip $ split ...
Andrei Sura's user avatar
4 votes

How can I work with a 4GB csv file?

I am a big fan of tad which crunches these kind of files easily.
Steven's user avatar
  • 141
4 votes

List of data sources at the state, county and zip / zcta level

FRED, from the Federal Reserve Bank of St. Louis offers various economic time-series data on various geographic levels. I personally use these on a county level, where you can find data on: population,...
Ariel Karlinsky's user avatar
4 votes
Accepted

Global Precipitation Measurement (GPM) data download in R

I created a function to download data IMERG HDF5 and convert them to raster GeoTIFF. You can see the code in https://github.com/AybarCL/GPM-data-management
Cesar Luis Aybar Camacho's user avatar
4 votes

Data on hurricanes?

Wunderground has a quite extensive hurricane archive (since 1851), unfortunately they are a commercial organisation, so you'll probably have to pay for API integration with their historic data. A ...
Swier's user avatar
  • 1,453
4 votes
Accepted

Data on hurricanes?

NOAA's Hurricane Center has current gis as well as archived hurricane data
albert's user avatar
  • 11.9k
4 votes
Accepted

Machine readable list of monetary currencies, including symbol

There are at least three github repos Currency-List - many formats - "List of all currencies with names and ISO 4217 codes in all languages and all data formats." JSON (gist) - (I'll use this one for ...
philshem's user avatar
  • 17.6k
4 votes

Historical Twitter Data

Unfortunately, the search index for searching tweets only goes back 7 days. Link to the documentation here, it can be found under 'until'. You might be able to collect tweets from individual users' ...
Oliver Frost's user avatar
4 votes

Popularity of programming libraries

Github is the de facto internet repository for open source code. They've released source code data from packages hosted on this site (press release, blog post with examples). The data is available as ...
philshem's user avatar
  • 17.6k
4 votes
Accepted

Get Historic Temperature Data by GPS and Timestamp Using R

I recommend trying rnoaa https://cran.rstudio.com/web/packages/rnoaa/ - It's an interface to many different data sources from NOAA. df <- data.frame( id = letters[1:6], time = c("2016-06-01 ...
sckott's user avatar
  • 484
4 votes
Accepted

Mapping PUMA Codes to TIGER/line shapefiles

There are 4 variables you are leaving out of the shapefile that was imported to R. STATEFP10 is the 2-digit FIPS code for the state. PUMACE10 is the 5-digit FIPS code for the 2010 PUMA within a state....
Kotebiya's user avatar
  • 2,989
3 votes

ASCII Character Frequency Analysis

I wrote some code to analyse the reuters21578 corpus (thanks @philshem for pointing me to it)! You can find the results here - ascii_freq.txt or ascii_freq.json. The code used to generate them can be ...
PiersyP's user avatar
  • 131
3 votes

Where is the real beginner's guide

When you connect to an API, you are simply creating long URLs that contain filters, flags and other parameters. So you can test your URL in the browser before using a programming language. In this ...
philshem's user avatar
  • 17.6k
3 votes

Historical Mobile App Rankings/Downloads and Prices

Check http://androidrank.org , it contains data on android applications and it is for free.
janq's user avatar
  • 31
3 votes

Good tools to parse repetitive unstructured data

Take a look at UTAH: https://github.com/sonalake/utah-parser It's a good tool for handling files like this.
Daniel Bray's user avatar
3 votes

List of data sources at the state, county and zip / zcta level

Here is another county level BLS dataset: Quarterly Census of Employment and Wages (QCEW). I work for this part of BLS. QCEW provides employment and wage data by industry at the county, MSA, State, ...
David Hiles's user avatar
3 votes
Accepted

Does this violate copyright of music/lyrics/sound?

As long as you don't keep the song anywhere on your servers, and have clear warnings "Do not transform songs that you are not allowed to transform", I believe you don't risk that much. You are just an ...
Nicolas Raoul's user avatar
  • 8,416
3 votes

How to link postcode to broader geographical area in the United Kingdom

There are a number of open datasets which link a postcode to a "broader geographical unit". The best dataset for the purpose depends on which "broader geographical unit" you are after. The below list ...
Dan Winchester's user avatar
3 votes
Accepted

Site that provides up to date list of programming languages/frameworks to open source repositories using it?

Open Hub has something almost exactly like this, although I don't think there are specific pointers that say "we use this code, like this, here". Firefox's profile profile shows some of the data ...
albert's user avatar
  • 11.9k
3 votes

Bounty Programs for Open Source Projects

I've compiled a short list of potential sources to answer the question: https://www.freelancer.com/ https://www.bountysource.com/ https://www.bounties.network/ https://internetbugbounty.org/ https:/...
sboysel's user avatar
  • 1,214
3 votes

Unzipping the openfda data in R

Just use for: # unzipping the files temp<-tempfile() for (f in files) { download.file(f,temp) unzip(temp) }
Stanislav Kralin's user avatar
2 votes

How can I work with a 4GB csv file?

You can try nitroproc. It's currently a beta version I think, but allows to sort/summarize/filter/etc. It's free.
bob's user avatar
  • 21
2 votes

How to scrape rugby union data from espn.co.uk

In python there are many libraries available for web scraping these are some of the important ones scrapy, Beautiful Soup etc. In php we have simple_html_dom.php which can parse data from websites. ...
Eka's user avatar
  • 121
2 votes

Historical Mobile App Rankings/Downloads and Prices

There is an unofficial Android Market API (Java) where you can slowly collect all the data you may be interested in Android Market API This library allow you to ask directly google's official ...
philshem's user avatar
  • 17.6k
2 votes
Accepted

Lobby meetings of European Commission - Integrity Watch

With a hint from a person from Transparency International and some further browsing through the source code this link eventually made it work: http://static.tttp.eu/doi/data/meeting.csv
moezelot's user avatar
  • 153

Only top scored, non community-wiki answers of a minimum length are eligible