6

Data:

I am looking for a well-studied data set for regression problems, where the data could be split - per example - into a set of features and a continuous variable that depends on those features.

The subject matter is not important, but license compatibility is.

Context:

I would like to add better support for regression problems to a public machine learning library, which is published using the two-clause BSD license.

For tests of its classification routines, the library uses a subset of the MNIST handwritten digits.

My current pull-request for the library uses synthetic data from a non-linear function. I just made up a function, added some random noise, and created a unit test around that. However, I think it would be much nicer to use something that demonstrates a practical use for the library, and even better if it is a data set that has some history of being studied to demonstrate effectiveness of machine learning techniques.

Region:

Not important, the data could describe any phenomena in any region.

License:

This is a key issue for me. The library code is published using the two-clause BSD license

It needs to be OK to add a version of the data to the library, and publish it with the library. It is not my library - I will be submitting a pull request to the original author - so I am not at liberty to adjust the license type or add any constraints to the library in order to incorporate the data. However, it should be fine to attribute the data source, or a related academic publication.

Format:

Tabular numeric data, one row per example.

Authority:

Not important, provided the rights to use the data are clear.

Requirements:

The primary goal is to run a unit test in under a minute, and to demonstrate a test error better than un-trained guessing in that time. That rules out "big data" sources where a regression problem might only be tractable when using gigabytes of data. Something that has only a few hundred or few thousand examples, and where the regression problem is tractable would be best.

A nice-to-have: If the data set also has published error values obtained using different techniques.

Non-answers:

My searches so far have only turned up large data sets, great to play with and learn techniques, but not really unit-test material. I am also blocked by finding logistic regression (a classification technique with an annoying name) whenever I try to search for e.g. "like MNIST digits, but a regression problem"

Sources like https://archive.ics.uci.edu/ml/datasets/Auto+MPG look promising, but are not directly licensed for this use (I have checked this by emailing the data repos contact address). I am considering contacting data donors from that site individually to check whether my intended use is OK, but of course it would be easier if there was a standard open data set already.

  • I'd like to try to answer this, but I'm finding the wording to the question a little confusing. Do you mind to reformulate the question (see here for a useful template) – philshem Apr 4 '15 at 13:04
  • 1
    @philshem: I have tried to match the suggested template. – Neil Slater Apr 5 '15 at 21:40
2

The "elemapi" data set may be promising (or maybe it will help refine the question of what you're looking for). I don't see an explicit license, but it's used widely and is from a public source, so hopefully there is a friendly license somewhere.

UCLA-affiliated course use:

In this chapter, and in subsequent chapters, we will be using a data file that was created by randomly sampling 400 elementary schools from the California Department of Education's API 2000 dataset. This data file contains a measure of school academic performance as well as other attributes of the elementary schools, such as, class size, enrollment, poverty, etc.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.