fprintf for ~million row string matrix

1 次查看（过去 30 天）
Erik Johannes Loo
Erik Johannes Loo 2022-7-26
编辑： Jon 2022-7-26
Hi,
I'm trying to write data to csv files, but using fprintf (or the way I am using it) seems to be extremely slow
The data is formatted in columns containing fields such as: {ID (string), date (string), numeric data, flag (string), ...}.
I get the data as a cell array, with each cell having a column of data.
What I've done is cast everything into a string and take the transpose of the data (it seems MATLAB does things in a column-major order).
Also note I've added a bunch of commas without data inbetween. That's the expected format of the data, so that has to stay.
fid = fopen(fullfile(output, replace(file.name, '.dat', '.csv')), 'w');
fprintf(fid, [repmat('%s,', 1, nCols), repmat(',', 1, 86-nCols), '\n'], T);
fclose(fid);
It looks as if saving a single file takes several minutes (haven't managed to save even one file yet - and I need to do this for 365 files).
Is there a way to speed up this process?
Kind Regards,
Erik

请先登录，再回答此问题。

回答（1 个）

Jon
Jon 2022-7-26
You could try writecell and see if that is more efficient.
  2 个评论
Erik Johannes Loo
Erik Johannes Loo 2022-7-26
Thanks. I don't have writecell on my version (R2018b) but I tried the code on a different computer (with R2020b) and my original approach (fprintf) worked much faster without modification.
I'm assuming fprintf was improved from R2018b to R2020b which has led to dramatic performance improvements.
Jon
Jon 2022-7-26
编辑：Jon 2022-7-26
Or maybe the computer with R2020b was just faster? Also I have found that MATLAB is sometimes very slow writing to network drives. If possible write it to a file on your local hard drive.

请先登录，再进行评论。

类别

在 Help Center 和 File Exchange 中查找有关 Whos 的更多信息

标签

产品


版本

R2018b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by