Writing files to .csv

2 次查看（过去 30 天）

Angshuman Podder 2020-4-8

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/516486-writing-files-to-csv

评论： Angshuman Podder 2020-4-10

Hi, I am curious to know about the execution speed of a task I want to perform. I have a lot of data (variables) to store that is generated at every time step. Currently, I am dumping the output to a .csv file at each timestep. Something like:

%demo
fid1='C:\Users\Desktop\data\demo.csv';
v1  = fopen(fid1,'w');
for i =1:10000
    %say these are the variables that I need to store
    x2  = rand(5,2); 
    x3  = rand(6,3);
    
    fprintf(v1,'%d \t', i);
    fprintf(v1,'%10.3e \t', i,x2(:,1)',x3(:,1)');
    fprintf(v1,'%d \t', i);
    fprintf(v1,'%10.3e \t', i,x2(:,1)',x3(:,1)');
    fprintf(v1,'\n');
end
fclose(v1);

Is this the fastest way? Or should I allow matlab to store all the values for all the variables and then after the end of process runtime, I dump them to a .csv? Are either of the process going to impact the execution speed of the actual model code which generates the data?

5 个评论
显示 3更早的评论隐藏 3更早的评论

Walter Roberson 2020-4-10

Suppose you are logging data from an hour long experiment. Suppose that if you lost 5 minutes of data, that you could still get reasonable results. But suppose losing the entire hour would be expensive. In such a case, every (say) 4 minutes you could take the data you had accumulated in the last 4 minutes and try to write it out, and if the write failed you would still have most of a minute to raise an alarm so that the people running the experiment could take appropriate action. Likewise if you were writing every 4 minutes, then if the power failed or the system rebooted for some reason, you would have only lost at most 4-and-a-hair worth of data.

Collecting all of the data first and then writing it all out at once is the most efficient from the point of view of time and system resources used, but that kind of efficiency is not always the most important factor.

At the same time, if you do not absolutely need every value to be logged every time, then it is not advisable to log every result immediately, because that is costly in system resources.

Typically the most efficient hard disk useage is if you write out multiples of 8 kilobytes at a time, with the operating system overhead being least if you write out a number of those at a time, such as a megabyte.

Angshuman Podder 2020-4-10