3D sparse matrix
显示 更早的评论
I have 1500 images almost each have 5000 rows and 7000 columns. so I need to load one by one. make a large matrix. The matrix size will be 5000x7000x1500. Image are Greyscale Image.
A portion of code is below. In the images variable preallocaiton will be done. To save the memory I need to do the 3D sparse matrix.
Any of your help will be appreicated. Thank you so much.
filename = sprintf(basefilename,start_idx);
full_path = fullfile(image_dir, filename);
f1 = imread(full_path);
nframes = end_idx-start_idx+1;
% preallocating the matrix
sz0 = [size(f1,1:2) nframes];
images = zeros(sz0,class(f1));
fidx = int8(start_idx:end_idx);
tic
parfor k = 1:length(fidx) % these are output frame indexes
% reading one by one frame operation.........
images(:,:,k) = tmp_img(:,:,1);
end
采纳的回答
Walter Roberson
2023-11-20
0 个投票
MATLAB sparse matrices are strictly vectors or 2D, with no capacity for 3D.
The File Exchange Contribution https://www.mathworks.com/matlabcentral/fileexchange/29832-n-dimensional-sparse-arrays can emulate multi-dimensional sparse matrices.
Note that performance of sparse matrices can be poor if you add more entries than there are available "unused slots".
A basic sparse matrix has one 8-byte pointer per column . If the column is empty, the pointer is NULL (binary 0). If the column is not empty, then the pointer points to a list of row indices and corresponding values.
Making assignments into a sparse matrix can require copying the entire structure of the existing sparse matrix into a new structure that has more space, doing the assignment in the new structure, and throwing away the old structure.
However, when you create a sparse matrix using spalloc you can designate the number of unused slots to allocate. Each unused slot takes 16 bytes. When you assign into a sparse matrix that has an unused slot, then the sparse matrix can record the row and column and corresponding value without having to copy the matrix.
As you plan to eventually write in entire layers of image data, if you do not pre-allocate lots of memory, you would end up with a notable amount of copying the sparse matrix to make room for the new data. And if you do pre-allocate then you need 16 bytes per pre-allocated entry, which is going to add up enough that it is going to be worse than having a full matrix that is twice as large.
Sparse matrices that are densely occupied are not efficient.
1 个评论
Walter Roberson
2023-11-20
parfor k = 1:length(fidx) % these are output frame indexes
% reading one by one frame operation.........
images(:,:,k) = tmp_img(:,:,1);
end
have you considered
parfor k = 1:length(fidx) % these are output frame indexes
% reading one by one frame operation.........
images_cell{k} = tmp_img(:,:,1);
end
If later you have operations that need to work across multiple images, then you can cat(3,images_cell{:}) to create a grand array. Or extract portions of each cell and cat(3) those portions together if you only need a segment of the cuboid.
更多回答（0 个）
类别
在 帮助中心 和 File Exchange 中查找有关 Sparse Matrices 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
