Data sets for teaching

Question

I'll be giving a (short) introductory course for Data Science within a couple of months.

One of my ideas was, instead of giving lectures on theoretic aspects of it (which of course are important), make people actually get their hands dirty with data and get the feel for what's like to work with data on a daily basis.

But for that I think I'd need a single large and interesting (open) data set for which we could go about all the phases of a data science project. Something like:

1. Define the problem
2. Get the data
3. Explore it
4. Prepare it
5. Develop our model(s)
6. Reiterate as necessary
7. Deploy

With the same data set. Or maybe with two related data sets.

So, any ideas of what that data could be?

Answer 1

It's hard to know your exact audience, but there is a great blog post with (by now) more than 100 interesting data sets - 100+ Interesting Data Sets for Statistics

"Data science" is pretty far-ranging, so you could do many different types of analysis:

• IT-related data mining - for example, Firefox browser data (alternative download site)

• Image processing, recognition, mining - for example, 10,000 annotated images of cats

• Language processing - for example, Google N-grams (downloadable raw data)

• Text mining - for example, Enron Email dataset

Answer 2

I would suggest using crime data per city. These datasets give daily as well as historical data, as well as location and time of day. When combined with other data (weather, census, employment, types of businesses, housing, etc) many correlations can be analyzed with different models being applied.

Below is a link to a blog that I wrote on this a while back ago:

http://www.opengeocode.org/articles/crime%20analysis.txt

Answer 3

Your ideal dataset is local, relevant to their discipline (if they have one), and interesting in its own right. In the past I have used Melbourne's "urban forest" dataset of 70,000 trees.

If you have hardcore computer scientists, you'll want a big and challenging dataset, maybe something transport related, like movements of taxis, or public transport, or bike share schemes etc.