7 votes

How can I reliably host and share large data sets?

I'm sorry to hear you had a frustrating time uploading to datahub.io and that the site wasn't clear enough about the size limit. Another datahub-like platform that we have worked with in the past is ...
Dan Fowler's user avatar
7 votes
Accepted

How can I reliably host and share large data sets?

If you want quick, dirty, and free, you can always host downloadable datasets at the Internet Archive (archive.org). No pretty interface, but as much storage as you could ever want.
Asparagirl's user avatar
6 votes

What methods are used to prepare sensitive data for public access?

The first step as you said is to define which types of entities need to be removed in the public version of the data set. One example of such definition in the United States is given in the Health ...
Franck Dernoncourt's user avatar
5 votes

What are the most common ways that users find out about new data sets?

I've also found DataBullet.in and Enigma Public Data Explorer to be helpful. Also, a recent social network called data.world has tons of interesting stuff.
Carl V. Lewis's user avatar
5 votes

What are the most common issues with data cleaning (e.g. outliers, duplicates)? Who has data sets that need to be prepared for analysis?

There can be numerous instances where data needs to be cleansed and standardised. To name a few -: 1) Mandatory fields having Null values - This is the greatest source of data loss Remedy - Going ...
Pramit's user avatar
  • 233
5 votes

How can I reliably host and share large data sets?

The quick, dirty, easy answer: compress the large data files into multiple .7z files, setting the maximum limit to 95mb per compressed file. I do this to get around GitHub's limitations, and want to ...
albert's user avatar
  • 11.9k
4 votes

Publishing (structured?) data about products: information about products, barcodes, photos (incl. packaging)

As you can see in Wikimedia Commons, packages usually involve some copyrighted content and therefore its image can't be published under a free license. It can only be published with permission of ...
Pere's user avatar
  • 381
4 votes

How do I get my data indexed by Google Dataset Search?

You may want to stop and ask why you should want to be indexed there. Dataset Search seems like a poorly planned effort to try and corner dataset search. Why the push isn't for open linked datasets/...
3 votes
Accepted

Implications of "For official use only"

The warning says nothing about using the data. Unauthorized activities are prohibited. Don't set up your own website or store your vacation images on their servers even if there is unfettered access ...
pndfam05's user avatar
  • 552
3 votes

A database of open databases?

I didn't see these mentioned. Rockstar Data Sources on Github. List of free and open journals. Mostly scientific journals.
Bulrush's user avatar
  • 667
3 votes
Accepted

How to embed licenses within SVG?

This is not RDFa, but rather embedded RDF/XML, which is allowed by the SVG Tiny 1.2 Specification. There are the following errors: <cc:license> instead of <cc:License> in lines 26, ...
Stanislav Kralin's user avatar
3 votes

Reliable point spatial data of cities of New Zealand

You could set up an account with Land Information New Zealand--LINZ Data Service Once you log in, hit browse data. The select Gazetteer option. That should show you three options. Select NZ Place ...
Aaron M's user avatar
  • 223
3 votes

Data set for Mcdonalds

Don't know if this will be of interest to you, but some of the datasets related to McDonald's on Quandl include the following: News Sentiment History for McDondald's corp: https://www.quandl.com/...
user37317's user avatar
  • 1,036
3 votes

What are the most common ways that users find out about new data sets?

Jeremy Singer-Vine, BuzzFeed's data editor, runs a pretty good mailing list of "interesting" data sets: Data Is Plural. It's not an exhaustive collection, but it covers a variety of topics.
pml's user avatar
  • 181
3 votes

What are the most common ways that users find out about new data sets?

As an addendum, even though it may not sound like the most obvious source, NICAR (National Institute for Computer-Assisted Reporting) maintains a very active listserv of data journalists corresponding ...
Carl V. Lewis's user avatar
3 votes

What are the most common ways that users find out about new data sets?

Data.world has some interesting open datasets, and allows you to look at recent additions.
wirefire's user avatar
  • 351
3 votes

Is there (or should there be) a standard way to categorize procurement data at the municipal level?

You hit on a particularly relevant pain-point when it comes to procurement data. The purchasing thresholds differ wildly across the map, and there isn't any universal standard for classification. I ...
Carl V. Lewis's user avatar
3 votes

What are the most useful formats in which to release geospatial data?

Geospatial Data Standardization No more KML, KMZ, SHP files! I will make the somewhat bold and possibly controversial statement that geospatial data should always be available in two single ...
Carl V. Lewis's user avatar
3 votes

Financial text data/corpora

For copyright issues; these datasets are no longer available. Check out those datasets here: 8.5M titles from Reuters (2007-2016) https://github.com/philipperemy/Reuters-full-data-set 450k + 100k ...
Philippe Remy's user avatar
3 votes

A database of open databases?

Google Dataset Search https://toolbox.google.com/datasetsearch A recent tool in beta. Blogpost FAQ Guidelines for developers (get your data indexed)
Andrey Portnoy's user avatar
3 votes

Is there a dataset having multiple files and multiple data formats (JSON+CSV+XML) for research purpose etc.?

Open Hampton Roads' datahub.io has a number of datasets with multiple file formats. At one point I would convert them to as many as possible, so you have a lot of options, including GIS data. Here's ...
albert's user avatar
  • 11.9k
3 votes
Accepted

Are there any public agencies which are currently streaming real-time video data?

Yes, there are (U.S.) public agencies that are currently (2017-03-07) streaming real-time video data. I'm not aware of a list of them (would LOVE it, if one such exists), but here's one: Live Video ...
albert's user avatar
  • 11.9k
3 votes
Accepted

Are version control storing solutions a good option to store Medical Imaging open data?

I'd rather suggest trying the open science framework (https://osf.io), as the above is one of their claimed aims.
jf1's user avatar
  • 146
2 votes

What's the best way to host map tiles?

I actually no longer recommend CartoDB (which has since been renamed to CARTO), although I was once a power user and helped build the platform from its open-source days. For maptile hosting, I'd go ...
Carl V. Lewis's user avatar
2 votes

Data set for Mcdonalds

Statista has a number of datasets about McDonald's, you can cherry pick through here: Statistics and facts on McDonald's
albert's user avatar
  • 11.9k
2 votes

Wind region Data for New Zealand

As NZ's weather service www.metservice.com has this data, their website says they provice this kind of data to windfarms: about.metservice.com/weather-for-business Their open data can be found here: ...
Swier's user avatar
  • 1,453
2 votes

Twitter - Labelling system for sentiments and other things

Yes, please share this information with the world. Sentiment analysis is always desired, I see plenty of questions regarding sa, as well as seeking twitter datasets on here regularly. Don't worry ...
albert's user avatar
  • 11.9k
2 votes

what are the best ways to build / scrape an open-sourced data set of a retailer type (e.g. tobacco retailers?)

There are a number of government sources that you could turn to for a live list of a specific type of retailer, in lieu of scraping Google or Yelp. For example, the FDA publishes a dataset of all ...
eveahe's user avatar
  • 389

Only top scored, non community-wiki answers of a minimum length are eligible