How can I calculate very large matrix and store the values?
1 次查看(过去 30 天)
显示 更早的评论
Suppose I have 1 million-by-1 million matrix.
I will read later about creating very large matrix at here http://blogs.mathworks.com/loren/2011/10/14/new-mat-file-functionality-in-r2011b/
Now here's the problem.
I have 1 million points (with their Euclidean coordinate). I want to calculate distances between them, for example distance between point 1 and 1, 1 and 2, 1 and 3, and so on; distance between point 2 and 1, 2 and 2, 2 and 3, and so on; distance between point 3 and 1, 3 and 2, 3 and 3, 3 and 4 , and so on.
I want to store those calculations in that very large matrix.
Is mapreduce, or any other methods, can do the calculations and store them? If yes, how can I do that?
Any help will be greatly appreciate.
Thank you very much.
0 个评论
回答(1 个)
Hugo
2015-4-10
I guess one option would be to reduce the size of your data using clustering such as K-means or Gaussian mixtures, and replace all points in a cluster with the centre of that cluster. You will end up with a reduced distance matrix and a map between cluster centres and data points. Of course, doing so will incur in an error, which can be decreased by increasing the number of clusters, at the cost of increasing the computational expenses.
0 个评论
另请参阅
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!