ImageDataStore and tall array, How to use to save Labels and 4D Matrices in for loop?

5 次查看（过去 30 天）
M
M 2023-11-7
评论： Matt J 2023-11-13
I generate 4D matrices of images (Double) and their respective labels (Double) with huge sizes in each iteration,
for example in one iteration I get images with size 120*120*1*6000 where 6000 the number of images, 1 is the number of channels, and their respective lables 1*6000
How to use ImageDataStore and tall array to store Labels and 4D Matrices in for loop in each iteration to use it then in further analysis in machine learning?
Also, Is there any other efficient solution to deal with huge data?
Note that the number of images and their labels vary in each iteration
  4 个评论
显示 2更早的评论
M
M 2023-11-13
编辑：M 2023-11-13
@Walter Robersonany hint how to pass in each iteration the 4D double matrices and double lables to imagedatastore and tall arrays please?
Because in the documentation there is no double format supported!
SupportedOutputFormats: ["png" "jpg" "jpeg" "tif" "tiff"]

请先登录，再进行评论。

请先登录，再回答此问题。

回答（3 个）

Jorrit Montijn
Jorrit Montijn 2023-11-7
I'm not sure what your workflow exactly looks like, but if you transform your images from double-precision floating numbers (64 bits per pixel) to uint8 (8 bits per pixel) you'll only need 1/8th of the memory capacity.
In most cases this won't even cause a loss of data, as images are often encoded in 8-bit depth anyway. In each iteration, you can then transform the (set of) image(s) you're working on back to double-precision floats so you don't have to change your workflow.
Transform the whole stack:
Ims_uint8 = cast(Ims_double*255,'uint8');
Then in each iteration:
Ims_double = cast(Ims_uint8,'double')/255;
Matt J
Matt J 2023-11-7
imageDatastores are provided for situations like that.
  8 个评论
显示 6更早的评论
M
M 2023-11-13
编辑：M 2023-11-13
@Matt J still I am not getting the process, what do you mean by Each image should be kept in its own file, which the data store would read in one-by-one as you loop
do you mean that I have to save all the files first then put them in the imds? What is the benefits of doing that??
If your images must be doubles (though I wonder why),
to not change the work flow
you can put them in individual .mat files.
how to do that? by them in the desk??
I cant find any simple example, the process is complex!
Matt J
Matt J 2023-11-13
do you mean that I have to save all the files first then put them in the imds? What is the benefits of doing that??
Yes. That way, the images are stored on disk, rather than consuming RAM.

请先登录，再进行评论。

Image Analyst
Image Analyst 2023-11-7
Cell arrays are very inefficient compared to regular numerical arrays. They use a lot more memory. You could even use single instead of double to preallocate using half the memory.
Not sure what you're doing but you may be doing what a lot of beginners do and that is to read ALL the input images into one huge array in a loop, then after the loop process each slice of the huge array as an individual image. This is usually NOT the way to process a sequence of images. You usually (if it's a series of 2-D images you want to analyze and not a volumetric image that you're reading in slice-by-slice) want to have a loop where you read in one image at a time and then analyze it immediately in that same iteration. See the FAQ:
https://matlab.fandom.com/wiki/FAQ#How_can_I_process_a_sequence_of_files?
Another option is an imageDatastore as @Matt J already mentioned.
If you must read all the images into memory at once time, then try to use a single array rather than a cell array.

类别

在 Help Center 和 File Exchange 中查找有关 Image Processing Toolbox 的更多信息

标签

产品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by