I would like to test on real numerical data a clustering algorithm which works well in the following setting:
- large number of points (let's say at least 10^5, but can be way more);
- medium dimension, at least a few hundreds, up to a few thousands;
- a number of clusters k not too large (ideally ~10-20, maybe up to 100 but no more)
Does anyone know a dataset, or a use-case for which this could be interesting?
For now I tried to use 10 classes with 40000 pictures each from the Places2 dataset, and extracted a VLAD descriptor for each image (dimension 8192, further reduced to 1024 by PCA), but I don't really get well-separated clusters (adujsted rand index between the result of a standard kmeans and ground truth = 0.12). I guess I can have a better separation of the clusters using for instance spectral clustering, but this would reduce the dimension of the features to d=k as well, and then it would not be really interesting anymore.