Hierarchical clustering of 1 million objects

Question

Can anyone point me to a hierarchical clustering tool (preferable in python) that can cluster ~1 Million objects? I have tried hcluster and also Orange.

hcluster had trouble with 18k objects. Orange was able to cluster 18k objects in seconds, but failed with 100k objects (saturated memory and eventually crashed).

I am running on a 64bit Xeon CPU (2.53GHz) and 8GB of RAM + 3GB swap on Ubuntu 11.10.

Answer 1

Consider switching the algorithm instead of using Hierarchical clustering and try using DBSCAN or OPTICS .
Hierarchical Clustering involves space and time complexities due to hierarchical layering and clustering of the data.
One of the drawbacks of Hierarchical Clustering is that it is nor suitable for large datasets.
One way to go ahead with huge data set and with same algorithm is to to divide the data into clusters and then construct hierarchical  trees. Again the working will depend upon the levels of the tree, size and shape of the tree.
System software and other things are also taken into account here.