Error using pdist2mex Error in kmeans>distfun
1 次查看(过去 30 天)
显示 更早的评论
Hi, To represent our data (3233477*256) with Bag of visual word (BOW) which use KMeans clustering to extract visual words when we choose K=5000 this problem show:
Error using pdist2mex
Requested 3233477x5000 (120.5GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a
long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.
Error in kmeans>distfun (line 747)
D = pdist2mex(X,C,'sqe',[],[],[]);
Error in kmeans/loopBody (line 445)
D = distfun(X, C, distance, 0, rep, reps);
Error in internal.stats.parallel.smartForReduce (line 136)
reduce = loopbody(iter, S);
Error in kmeans (line 335)
ClusterBest = internal.stats.parallel.smartForReduce(...
Error in BOWHistogram (line 12)
[idx,c,sumd,D2] = kmeans(double(Tab_Feature_Data),NumClust);
What can I do to fix the error? Please advise me
2 个评论
Rik
2018-3-18
Can you split the array into smaller parts? Unless you get a 120GB contiguous block of memory, you can't use this method. I'm not familiar enough with what you want to do to suggest a real solution.
采纳的回答
Bernhard Suhm
2018-3-25
You could try converting your large input data into a tall array (maybe as simple as t = tall(double(Tab_Feature_Data)), and then pass that tall array to kmeans. Though watch there are limitations which options of kmeans are available with tall arrays, see https://www.mathworks.com/help/stats/tall-array-support-usage-notes-and-limitations.html
4 个评论
Bernhard Suhm
2018-3-26
Your laptop will block unless you leverage multiple cores, but that also requires the Parallel Computing Toolbox. For large datasets like yours, you will need more powerful hardware or be patient with execution. The doc page I pointed you does point out that the 'tall' version of kmeans has only the first 3 output variables.
更多回答(1 个)
carpcarp carpcarp
2021-4-2
我也遇到过这个问题,你可以看看你是不是命名了一个kmeans的m文件,如果有,并且是自己添加的,先把它删除(或者先发到qq等可以保存代码的应用后删除),然后你再去跑你的程序就可以成功了。
2 个评论
carpcarp carpcarp
2021-4-2
You can see whether you have named a .m file called kmeans, like this:![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/570309/image.png)
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/570309/image.png)
if so, please delete it. Then run your program again.
Or if you don't know where is kmeans.m, you can download an application called everything.
My English level is limited, so I can only describe it simply.(←from translator)
Walter Roberson
2021-4-2
(User points out that there can be problems if you accidentally have your own kmeans.m instead of using MATLAB's)
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!