Write data disc periodically: how to organize this?

6 次查看（过去 30 天）
Amy
Amy 2017-3-1
编辑： Amy 2017-3-3
Introduction
I would like to run simulations for an unknown number of time increments until a certain condition is reached. Typically, I would have several matrices of e.g. 50.000x50 big and some matrices of e.g. 50.000x17x50. After each time increment, the results are written to these big matlab files for storage.
Question
I would like to store the matlab files in a more efficient way, write the results to disk periodically. What is the best way to organize this?
  • Should I have a 50.000*50 matrix in xxx.mat, and save to entries (3001:4000,1:50) after every 1000 increments for example?
  • Or should I save the matrix into smaller parts of 1000x50 big? If so, what kind of structure/cell/array is good to use for that? And what name giving is handy in that case?
Thank you very much in advance!
Amy

请先登录，再回答此问题。

回答（1 个）

Walter Roberson
Walter Roberson 2017-3-1
Perhaps you should consider using matFile
  2 个评论
Amy
Amy 2017-3-2
Thank you! I hadn't heard of that before. It looks promising.
Amy
Amy 2017-3-3
编辑：Amy 2017-3-3
I implemented matfile and I run a profiling test for about 40 minutes. It seems that this method works quite slow, the way I implemented it.
In this time I ran my script for 398 increments, and after each increment I wrote data to the matfile for 80 different variables. This resulted in 47.6 MB of data.
The profiler took 424 seconds (7 minutes) for Matfile, see picture.
What can I do to make it faster? Is matfile the solution for me?
My implementation:
m=matfile('newtest1.mat','Writable',true);
m.t(10000,1)=0;
m.saveddv(10000,20)=0;
etc. for all 80 variables
And:
m.t(i,1)=t_now;
m.saveddv(i,20)=Vsave;
etc.
where i is the number of the increment.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Workspace Variables and MAT-Files 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by