array exceeds maximum array size using dbscan function

3 次查看（过去 30 天）

显示更早的评论

SatyaPrakash Gupta 2020-4-8

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/516260-array-exceeds-maximum-array-size-using-dbscan-function

编辑： SatyaPrakash Gupta 2020-4-8

error_pdist2.png

Hi,

i have a huge data from the radar sensor most likely 2-3 millions of data i.e xy coordinates.

i am using dbscan where pdist2 function is called, due to huge data it says array exceeds maximum size , you can find the same as attahced, can you please do help me out how can i proceed further ?

Please note : i do not want to cut/separate the data.

3 个评论
显示 1更早的评论隐藏 1更早的评论

SatyaPrakash Gupta 2020-4-8

ImageSpots.mat

There is no mistake as such , i have already dried it.

you can also try by running pdist2 for the attached data and provide me the feedback.

Can you please do suggest how shall i run pairwise distance calculation for the attached data ?

Mehmed Saad 2020-4-8

在 MATLAB Online 中打开

Yup, not working sorry for that

what is the Distance metric?

'euclidean','squaredeuclidean','mahalanobis, ....

For euclidean, just simply apply

sqrt(sum(abs(ImageSpots.x-ImageSpots.y).^2))

请先登录，再进行评论。

请先登录，再回答此问题。

回答（1 个）

Ameer Hamza 2020-4-8

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/516260-array-exceeds-maximum-array-size-using-dbscan-function#answer_424731

pdist2 can indeed exhaust the memory. If the inputs are vectors of size m and n, then it returns a matrix of size m*n. If m and n are in millions, then it will fail.

SatyaPrakash, you tagged the question with R2018b. But Mathworks introduced dbscan from R2019a. Are you using the function from an external source? In that case, it is best to contact the developer to ask for the workaround. I checked the definition of dbscan in R2020a, and it does not include any call to pdist2. I think that the MATLAB's built-in function will be able to handle such large arrays. You might try this own some latest release.