7
votes
Accepted
How can I reliably host and share large data sets?
If you want quick, dirty, and free, you can always host downloadable datasets at the Internet Archive (archive.org). No pretty interface, but as much storage as you could ever want.
- 496
7
votes
How can I reliably host and share large data sets?
I'm sorry to hear you had a frustrating time uploading to datahub.io and that the site wasn't clear enough about the size limit. Another datahub-like platform that we have worked with in the past is ...
- 699
5
votes
How can I reliably host and share large data sets?
The quick, dirty, easy answer: compress the large data files into multiple .7z files, setting the maximum limit to 95mb per compressed file. I do this to get around GitHub's limitations, and want to ...
- 11.8k
4
votes
Accepted
Data sets for predicting home value
You're probably going to need to get information from many jurisdictions - at least in the US, there is not a good/granular sources for property tax, crime, and other information for all states in one ...
- 5,174
3
votes
Can anyone suggest a big dataset for the purpose of regression?
At least a million records you say? I know of a data set that could fit the bill.
The 2013 ACS 5-Year PUMS has a record of 7,300,520 households (which represents 6,671,272 housing units and 629,248 ...
- 2,959
3
votes
Big Data and machine learning to Predictive analytics
"The Unreasonable Effectiveness of Data" is an insightful article, written by by some Google researchers in 2010.
If you search for articles that have cited this, you'll find plenty of stuff to read....
- 1,202
3
votes
Real-time data APIs
Here you can find tons of APIs:
https://www.programmableweb.com/category/all/apis
You also have a filter for streaming APIs:
https://www.programmableweb.com/category/streaming/apis?category=20253
- 241
2
votes
Publicly available transaction data
Here are two sales data sets from Kaggle:
Rossmann Store Sales
Walmart Recruiting — Store Sales Forecasting
You can ask on the forums to get an answer for the dateset licenses from the competition ...
- 121
2
votes
Dataset of large graphs for classifiction
Try Stanford Large Network Dataset Collection
https://snap.stanford.edu/data/egonets-Facebook.html[
Directed graphs twitter
is listed undirected and directed graphs.
Hope it answer your question.
- 425
2
votes
How can I reliably host and share large data sets?
I was going to add this as a comment on Dan Fowler's answer but I had to create a new account to post official in my role as co-founder and CPO at data.world. I'm glad you found that data.world's ...
- 21
1
vote
Where to find an actual bigdata with Variety and Volume?
You can find large public datasets in sources including Google Open Data, Kaggle Datasets, Github and Data.gov. If you are specifically looking for large data coverage, the California Open Data Portal-...
1
vote
How can I flter data from a stream of very large text files?
Have you looked at AWS Athena yet? It doesn't require any additional storage (besides the existing S3 bucket) and you can query the data (all of it) whenever you want - for example if your query ...
- 123
1
vote
How can I reliably host and share large data sets?
Have you thought about Google Spreadsheet?
You can retrieve the content of any public Google Spreadsheet in your web app using JSON feeds. The sharing permissions of the Google Spreadsheet should be ...
1
vote
Publicly available transaction data
For free sales data, check out this database on Quandl:
US Census Bureau - Here are the datasets you'll see if you search for "sales" within this database:
https://www.quandl.com/data/USCENSUS-U-S-...
- 1,036
1
vote
Real-time data APIs
Use Unigraph's data streaming API, it will return you thousands of items per second. Here's how to set it up: https://github.com/unigraph/docs/wiki/Streaming
For now just run this sample query in ...
- 573
1
vote
Data sets for predicting home value
Zillow has a free API that will return their estimate of average home prices. You can get the data down to zipcode and neighborhood level.
http://www.zillow.com/howto/api/APIOverview.htm
- 8,637
1
vote
Data sets for predicting home value
I guess you are looking for data like this
https://archive.ics.uci.edu/ml/datasets/Housing
- 121
1
vote
open Big Data to solve cancer epidemiology challenge
If you're interested in brain imaging data, you could check out the Connectome Coordination Facility
I downloaded the original HCP 1200 for our institution a couple of years free of charge. That's 7T ...
- 211
1
vote
What's the difference between Open Data and Big Data
Big Data.
I think the point has been missed about "Big Data". It doesn't need to be large in volume.
There are historically (from 2001) three tenets of big data
high volume (e.g. Facebook, Google ...
- 1,104
Only top scored, non community-wiki answers of a minimum length are eligible