How to save .mat files on the disk after each iteration?

20 次查看（过去 30 天）
M
M 2023-11-14
编辑： Stephen23 2023-11-15
I am facing difficulties how to save the data as struct after each loop.
I have huge image matrix and its respective labels matrix ,I want to save them after each iteration in a folder, and then delete them after each iteration from workspace, any suggestion?
Please note that I want to save the data in the disk and clear them after each iteration and save

请先登录，再回答此问题。

采纳的回答

Dyuman Joshi
Dyuman Joshi 2023-11-14
The values will be over-written in each iteration, so you don't need to clear them.
for k = 1:number_of_iterations
%get images and labels for this iteration
%save them as MAT files, the variable names should be provided as a string or a char vector
%As you have mentioned that there is a large data for each iteration
%use the (MAT File) version 7.3
save("images"+k+".mat", "images", "-v7.3")
save("labales"+k+".mat", "labels", "-v7.3")
end
  23 个评论
显示 21更早的评论
Stephen23
Stephen23 2023-11-15
编辑：Stephen23 2023-11-15
P = 'absolute or relative path to the parent directory';
S = dir(fullfile(P,'*Data','*.mat'));
If you want the folder and filenames sorted into alphanumeric order then either 1) add sufficient lrading zeros to the folder/filenames or 2) sort them yourself, e.g. by downloading NATSORTFILES:
S = natsortfiles(S); % DOWNLOAD https://www.mathworks.com/matlabcentral/fileexchange/47434-natural-order-filename-sort
S contains all filenames together with their folders and other file meta-data. You could loop over all files:
for k = 1:numel(S)
F = fullfile(S(k).folder,S(k).name);
D = load(F);
.. do whatever with D (except concatenate them into a 64 GB array)
end
Of course you could also filter S for specific file/folder names. Or modify the DIR match expression to only match "labels" or "TRO" at the start of the filenames... whatever.
"I cant find clear method to deal with this data properly , I want to use this data in SVD and NN but before that I want to do some computations as I mentioned in the previous thread"
If this is the same data discussed in your other threads then you (probably) cannot fit all of that data into memory. So there is likely no point in trying to concatenate all of those arrays together. But you could supply those filenames to DATASTORE (of some kind) and use tall arrays to process it.... if you have sufficient time to spare (perhaps days... or weeks... or more... who knows).
Before you decide to attempt SVD on a 64GB array you should probably do an internet search on SVD of large arrays.
I suspect that you do not really appreciate that computers do not have infinite memory and infinite speed.
Stephen23
Stephen23 2023-11-15
编辑：Stephen23 2023-11-15
"I cant do that since the name inside each .mat file is the same "
How exactly do the variable names prevent you from processing your data?
Having exactly the same variable names in every MAT file makes your data easier to work with. It also makes your code simpler, more efficient, and more robust:
https://www.mathworks.com/matlabcentral/answers/2046902-how-to-save-mat-files-on-the-disk-after-each-iteration#comment_2961112
What exactly is stopping you from LOADing into an output variable and accessing its fields?
Show us the exact code that you tried.
"I want to pick up from each folder 15 files of images (4D double) and their respective files of labels and combine the images and combine the labels"
In this thread you wanted to combine around 400 "images" into a 64GB array:
https://www.mathworks.com/matlabcentral/answers/2047157-concatenation-of-big-matrices-failed
Now you only want to combine "15 files of images".
Which is correct? When your explanations keep changing then it makes it hard to understand what you want.

请先登录，再进行评论。

更多回答（1 个）

Catalytic
Catalytic 2023-11-14
编辑：Catalytic 2023-11-14
Often, you would put all images with the same label in the same folder -
Folders={'Label1','Label2','Label3'};
for j=1:numel(Folders)
mkdir(Folders{j});
for i=1:10
image=rand(10);
save( fullfile( Folders{j}, "Image_"+num2str(i,'%3d') ) , 'image')
end
end
  2 个评论
M
M 2023-11-14
编辑：M 2023-11-14
@Catalytic I don't want to put all the images of the same label together for now
I just want to save the two .mat files as it is with their iterations name in the same of the code directory to not overwrite.
As well I got the following error after trying your code:
Error using save
Argument must be a text scalar.
Thanks
Catalytic
Catalytic 2023-11-14
We are not here to refine code for you down to the last detail. The example clearly shows how you can create file names in a loop to use with the save() command. Adapt it as you see fit.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Datastore 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by