Clustering evaluation with Silhouette extremely slow

38 次查看（过去 30 天）
Phu Lai
Phu Lai 2017-1-9
评论： Stephen john 2022-5-24
I am evaluating my kmeans clustering solutions using the built-in evalclusters function with Silhouette criterion:
eva = evalclusters(data,idx,'Silhouette');
Data size is 434874x4, tested on my laptop (core i7, 8GB RAM). I have been waiting for more than an hour but it still has not completed. Is there any way to boost the speed of Silhouette evaluation in Matlab?
Thanks a lot.
  2 个评论
Fernando Isorna Retamino
Fernando Isorna Retamino 2017-2-11
I have the same problem. My data size is 1726944x7 and I have been waiting for more than 2 weeks with a similar computer.
eva = evalclusters(data,'kmeans','Silhouette','KList',[1:10]);

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

John D'Errico
John D'Errico 2017-2-11
编辑：John D'Errico 2017-2-11
Your computer is not infinitely large or infinitely fast, unless of course, you are on TV or in the movies. Then anything is computable, merely by typing a few characters, and it will happen immediately, because who wants to wait for a few hours if they are watching a movie?
In real life, big problems take time. No matter how fast is your chosen algorithm, you can always throw more data at it than the algorithm is able to handle efficiently, no matter what is the algorithm. In real life, I don't know of anybody who does not want their code to execute more quickly.
There are a few simple choices to make here:
1. Learn patience, get some coffee and read a good book while you wait. Preferably one about MATLAB, or perhaps one about numerical methods in computing.
2. Find a seriously faster computer to use. This is never as cheap as you want.
3. Choose a different, more efficient algorithm. Different tools have different properties, but faster tools need not be always as good as others.
4. Learn enough about the methods that you can implement the algorithm EFFICIENTLY in a lower level language. Note that merely compiling your code from MATLAB is rarely going to give much of a speedup. That question gets asked on a weekly basis here.
5. Learn enough about parallel computing that you can take advantage of tools like the parallel computing TB. There are many issues here, and not all codes can be sped up.

更多回答（1 个）

neuromechanist
neuromechanist 2019-5-30
A problem with evalclusters is that it essentially runs multiple k-means (or any other clustering algortihm) one by one, not in parallel. I think Adding parallel cpability to the function increases its speed dramatically, given that parallel computing toolbox is available.
evalclusters is written in object form, and I don't know how to change for loops in object oriented setups to parfor for now. Will update my response if I figure it out.
  3 个评论
显示 1更早的评论
neuromechanist
neuromechanist 2022-5-23
Hi Stephen,
No, Unfortunately I did not find an answer to speed up the evalclusters function.
Sorry to disappoint.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Cluster Visualization and Evaluation 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by