Data access in sparse matrix
4 次查看(过去 30 天)
显示 更早的评论
I am trying to speed up a section of code which uses a very large (300,000 x 8 x 8) matrix whose elements are derived from other variables via a non-trivial function. Currently I have pre-calculated all the terms and saved the matrix, but even loading this matrix into memory takes a long time.
In any individual run of my code it's likely that I will only need to access a small subset of the values (probably 5000 of the 300,000 matrices?), but those terms will need to me accessed many times (1000's), so I was considering changing to using a sparse matrix and calculating the terms as required, if not previously calculated.
To decide if this is a worthwhile change, can someone tell me how the sparse memory access works in MATLAB. Specifically, if I already have 4999 of the matrix slices calculated, will the following be very slow operations: a) search to see if a given matrix has already been computed and placed in the sparse matrix b) place a new matrix in the sparse matrix.
b) is the one I don't really understand - I suspect I can solve any a) problems with a large boolean vector or similar.
If it makes a difference this code has to run on R2010b.
1 个评论
James Tursa
2012-10-10
编辑:James Tursa
2012-10-10
FYI, MATLAB does not directly support 3D sparse arrays. To get that behavior you have to use something like Matt J's ndSparse class:
Also, it is typically more efficient to store the 2D slices in the first two indexes rather that the last two. E.g., have the array 8 x 8 x 300,000.
采纳的回答
Doug Hull
2012-10-10
The truth is, a test is worth a thousand expert opinions.
Have you run this through the profiler to confirm that it is the matrix access that is killing your speed? Break down any slow lines of code into smaller pieces so you can see which operation in a line of code is hurting you as profiler works on a line by line basis.
更多回答(1 个)
Matt J
2012-10-10
编辑:Matt J
2012-10-10
I can't imagine that accessing the matrix is slowing you down appreciably on any decent machine. It might help a bit if you reorganize your array to be 8x8x300000, because the first dimensions of an array is always the slowest to access. I'm also surprised that loading it takes a long time, though I'm not sure what "long" is for you.
As for sparse matrices, updating them on the fly is always a lot slower than for full matrices, but it doesn't sound like it matters. Only a small percentage of your operations sound like they're actually devoted to generating and filling the 'slices'. Most of your computation involves pulling them out again, which should be very fast no matter what you use.
0 个评论
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Data Type Identification 的更多信息
产品
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!