What is the fastest/best way to create several thousand (text) files?

7 次查看（过去 30 天）
AAA
AAA 2023-9-12
评论： AAA 2023-9-15
What is the fastest/best way to create several thousand (text) files?
The files should have the same name base and be numbered accordingly.
  6 个评论
显示 4更早的评论
John D'Errico
John D'Errico 2023-9-12
Not at all surprising. Odds are this just says your hard disk is fragmented. That is a natural thing for any drive that has been in use for possibly multiple years. So each time, your OS takes a little more time to allocate the necessary disk space. This is not a MATLAB issue by the way. It is your OS that is doing the work, and taking the time.
I might suggest you do it by working on an empty drive on the side. Now you would probably find it is less hungry for time.
AAA
AAA 2023-9-15
编辑：AAA 2023-9-15
Thank you everyone for your answers!
I analyzed a lot of different implementation possibilities and the main reason for the slow down when implementing it with a for-loop and "copyfile" was that the destination directory where the files get copied to, was in my search path. Seems like it is not an issue of the OS, instead the "copyfile" function triggers matlab to scan all the data in the search path which drastically increases the execution time over time.
When removing the destination directory from the search path the execution time does NOT increase and is drastically decreased.
When using "cellfun(@(FID) copyfile(sourcefile, FID), destnames);" instead of the for-loop the execution time can be further optimized and is a good way to go. (https://de.mathworks.com/matlabcentral/answers/14962-making-multiple-copies-of-a-file#answer_20334)
The fastest implementation for my application however is the method suggested by Kunal Kandhari.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Kunal Kandhari
Kunal Kandhari 2023-9-12
Using "fopen" & "fprintf" is one of the fastest way of creating several thousand text files
below is the sample code for it:
baseFileName = 'textfile'; % The base name for your files
numFiles = 100000; % The number of files you want to create
for i = 1:numFiles
% Generate the full file name with a numerical suffix
fileName = sprintf('%s_%04d.txt', baseFileName, i);
% Open the file for writing
fileID = fopen(fileName, 'w');
% Write some content to the file (you can customize this part)
fprintf(fileID, 'This is file number %d\n', i);
% Close the file
fclose(fileID);
end
The speed of file creation depends on various factors, including the file system's performance, your hardware, and how efficiently you write the files.
The MATLAB code above is a straightforward way to accomplish the task, and it's not inherently slow.
You can read more about the "fprintf" and "fopen" from the following documentation:
  1 个评论
AAA
AAA 2023-9-15
Thank you everyone for your answers!
I analyzed a lot of different implementation possibilities and the main reason for the slow down when implementing it with a for-loop and "copyfile" was that the destination directory where the files get copied to, was in my search path. Seems like it is not an issue of the OS, instead the "copyfile" function triggers matlab to scan all the data in the search path which drastically increases the execution time over time.
When removing the destination directory from the search path the execution time does NOT increase and is drastically decreased.
When using "cellfun(@(FID) copyfile(sourcefile, FID), destnames);" instead of the for-loop the execution time can be further optimized and is a good way to go. (https://de.mathworks.com/matlabcentral/answers/14962-making-multiple-copies-of-a-file#answer_20334)
The fastest implementation for my application however is the method suggested by Kunal Kandhari.

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Startup and Shutdown 的更多信息

标签

产品


版本

R2020b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by