2

I'm studying data clustering algorithms.

However, I can only find little labeled real data suitable for clustering. Probably the most popular one is the iris data set, since it contains some well defined clusters that agree with the classes. Most of the data sets here are synthetic and tiny, or unlabeled.

Please don't point me to the UCI machine learning repository. Much of the data sets that are categorized as "clustering" there don't cluster well, and don't have labels suitable for clustering evaluation either.

I'm looking for data sets with the following characteristics:

  • preferably multivariate-numerical, since many algorithms only support this (text can be vectorized, but usually is not labeled into clusters. Graph clustering is very different, and I'm not looking into this right now)
  • large enough - 10,000 instances is small, 100,000 would be better. Beyond this, many algorithms will fail (many scale O(n²) or O(n³), unfortunately).
  • labeled for clustering. Many classification data sets are not good, because classes themselves contain multiple clusters, or multiple classes may be the same cluster (you can observe this on the iris data set, too - give an unlabeled data set to a human, and he will say there are two clusters instead of three).

Since many of the UCI data sets do not appear to work well for clustering, I'm figuring I should put out a call here at OpenData to find fresh data sets.

I am also interested in:

  • studies of clustering quality that use real data.
  • toolkits for data clustering (such as ELKI) that have many and fast algorithm implementations. Since I'm interested in advanced methods, I don't care about k-means-only, nor about tools that can't cluster at least 100,000 instances.
Improve this question

1 Answer 1

Reset to default
3

In the ReplicationWiki (that I founded) you can find many datasets used for empirical studies, so far most of them from economics. Quite a few of them are labelled to use some sort of clustering. You find these with the link for methods on the main page. It seems to me the way this term is used in economics is different from what you are thinking about. You should however be able to find datasets of the size you are looking for. In this overview you can search. Here you get only the cases for which "clustering" was already used. The datasets are not (yet) sufficiently well described in the wiki so that you can easily get what you want but maybe you can deduce from the topics and then take a look at the actual datasets on the journal websites. (There are a few additional ones that are accessible elsewhere online, for this switch the search value to "accessible elsewhere online"). Let me know if this helps you at all and if so if I can be of further help. And please join the wiki!

Improve this answer
4
  • Thank you. But I do not understand how to navigate/search that page. How can I filter to get only data for clustering and by size/data type? I don't want tiny data sets.
    – Has QUIT--Anony-Mousse
    Feb 7, 2016 at 15:48
  • I linked to two searches. You can add further property fields. There is one for data type and one for methods, you get them into the search with ?Article data type and ?Article method and estimation on the display part of the search, I'll post you a link here. Size is not yet included but you will sometimes be able to guess from the other information.
    – Jan Höffler
    Feb 7, 2016 at 15:51
  • I don't see an option to add such a property field.
    – Has QUIT--Anony-Mousse
    Feb 7, 2016 at 15:54
  • Display ?Article data type like here. Search for clustering like here.
    – Jan Höffler
    Feb 7, 2016 at 16:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.