Questions tagged [big-data]
Big data is one or more datasets that together are so large or complex as to require non-traditional techniques to understand.
22
questions
0
votes
0
answers
17
views
Seeking mobility big data
I'm looking for motion/mobility big data for a Spatial Data Mining project.
Can anyone help me in providing such data (except Microsoft project)?
- 1
1
vote
0
answers
50
views
Could privacy friendly open source OS/software be made for most tech products?
This is related to open source, but somewhat a new topic here that I hope you will contribute to in the discussion.
Many are tired of getting their privacy invaded by Google and all companies who take ...
- 11
1
vote
0
answers
31
views
Data lake layers approach
I am working on an approach to creating a data lake using Cloudera.
My question is for an approach to be taken for the Curated/Gold layer.
RAW layer - All data from the source will be available in ...
- 111
0
votes
2
answers
39
views
Where to find an actual bigdata with Variety and Volume?
I am exploring the field of BigData analytics. I am basically interested in dealing with two V's: Volume and Variety. I am not able to find a decent data which has a Volume enough to be called a big ...
1
vote
0
answers
25
views
Frequency of professions in cities data
I would like to know, is there any database out there, that will list, given a country and city, a list of professions followed by number of people working in that profession, to any degree of detail. ...
1
vote
0
answers
23
views
Any open dataset for Geopolitical queries and answers?
I'm searching for queries/questions and answers that can be run and tested on top of the ICEWS and GDELT dataset, so that I could detect whether the query inference engine returns all the expected ...
- 121
1
vote
1
answer
48
views
How can I flter data from a stream of very large text files?
I have a stream of data being delivered to my Amazon S3 bucket in the form of tab-delimited text. I would like to scan as it comes in to keep a very small subset of the rows from that match certain ...
- 111
13
votes
6
answers
5k
views
How can I reliably host and share large data sets?
Relevant to this answer, I discovered that datahub.io has a 100MB limit on file uploads. I was disappointed to learn this, and it tooks many server errors to realize I should read the FAQ, where I ...
- 17.2k
3
votes
2
answers
695
views
Publicly available transaction data
I need publicly available transaction data to use for my statistical analysis project.
Preferably, it should relate to either:
Flight data (arrival, departure, number of seats, locations, etc.)
...
- 31
6
votes
2
answers
753
views
Real-time data APIs
I've recently started building a home project in Java for which I need lots of data (around 100 events per second). The problem, of course, is that I can't seem to find any publicly available APIs I ...
- 61
1
vote
0
answers
23
views
Hadoop open source design and implementation
I have a big project to analyse the client’s history, records in order to predict and manage the resources more efficiently. This has to deal with millions of data sets flew from various resources. My ...
- 41
2
votes
0
answers
45
views
Open data sources similar to the Workplace and Employee Survey (WES)
I've read some interesting analysis papers based on the Workplace and Employee Survey (WES), however it seems the data for this survey is not open to the public: http://www.rdc-cdr.ca/datasets/wes-...
- 81
3
votes
3
answers
100
views
Big Data and machine learning to Predictive analytics
This is the first time I post a question here. I am a newbie researcher and want to write a report about Big Data influence on machine learning to improve predictive analytics. I looked through Google ...
- 41
7
votes
3
answers
166
views
Data sets for predicting home value
I'm working on a project for predicting the cost of houses. I'm looking for data sets enumerating the number of schools, hospitals, parks and any other condition that may affect the price of houses (e....
5
votes
1
answer
394
views
Dataset of large graphs for classifiction
I want to evaluate a graph kernel designed for large graphs (> 10^6 nodes). Hence, I'm looking for suitable graph data sets, i.e., a set of (huge) graphs and corresponding classes.
Any ideas?
- 51
3
votes
1
answer
117
views
Can anyone suggest a big dataset for the purpose of regression?
I am working on a regression model.I need a big dataset for this purpose.
I have already worked with the famous Airlines dataset(http://stat-computing.org/dataexpo/2009/) and the million song dataset (...
7
votes
1
answer
118
views
open Big Data to solve cancer epidemiology challenge
I am a doctor and I am coordinator of a project that will call for innovative projects around challenges in the field of Big Data applied to Cancer Epidemiology. The specifity of the projects is to ...
- 231
3
votes
5
answers
1k
views
Looking for a large data set of French data
As I am a French student in IT I have to work on a Big-Data project using French data (the country, not the language).
What I'm looking for is a data set of something interesting like weather or ...
- 39
7
votes
2
answers
872
views
Practice Hadoop?
Can someone guide me in how to learn and practice Hadoop? I want to work on big data in the future and I want to work further in the field. Some datasets, practice tutorials, and guidelines are what I ...
- 173
1
vote
4
answers
2k
views
What's the difference between Open Data and Big Data
Open Data can be Big Data, and Big Data can be Open Data -- but they are not synonymous. What is the major difference between the two?
- 345
5
votes
1
answer
157
views
Have genetic algorithms been applied to Open Data?
I have become very interested in genetic algorithms or GAs (which are not necessarily related in any way to genetics or genomics; see link). This algorithmic method of searching is very well-suited to ...
- 2,068
19
votes
4
answers
981
views
What are the practical limits of releasing open data via bit torrent?
I know that there are political issues (my place of work bans all peer-to-peer software without an explicit waiver), but how well does bit torrent scale for distributing large data collections where ...
- 4,485