Use of textscan instead of load in a for loop

1 次查看（过去 30 天）
Isma_gp
Isma_gp 2018-9-6
回答： Walter Roberson 2018-9-7
Hi,
I'm at the moment loading data using the following code:
data = cell(1,n_sim);
for m_a = 1:n_sim
for m_b = 1:n_seed
data{1,m_a}{m_b} = load(fileList{1,m_a}{m_b}, 'A1:end');
end
end
This takes a long time since there are a lot of files and they are quite big. The resulting data cell is a 1xn_sim cell, and each cell has 1xn_seed cells inside. Each of those i.e. data{1,1} contains a 11000x25 double.
I would like to use textscan to get the same format on my results. The files that I'm reading have 25 columns and 11000 rows. Can I get some help with the scripting of texscan for this case?
Thanks
  6 个评论
显示 4更早的评论
Walter Roberson
Walter Roberson 2018-9-7
编辑：Walter Roberson 2018-9-7
It appears that in practice loading from a text file ignores any parameters that do not start with '-' . It would be better to remove the 'A1:end' from the load() in order to reduce confusion. In particular, all of the file will be read in, not just column 1.

请先登录，再进行评论。

请先登录，再回答此问题。

采纳的回答

Walter Roberson
Walter Roberson 2018-9-7
ncol = 25;
fmt = repmat('%f', 1, ncol);
data = cell(1, n_sim);
for m_a = 1:n_sim
data_mb = cell(1, n_seed);
for m_b = 1:n_seed
fid = fopen('fileList{1,m_a}{m_b}', 'rt');
data_mb(m_b) = textscan(fid, fmt, 'CollectOutput', 1); %notice () not {} on output
fclose(fid);
end
data{1,m_a} = data_mb;
end

更多回答（1 个）

Alexander Jensen
Alexander Jensen 2018-9-6
I imagine that the reason why it takes a while is due to the lack of preallocation of memory, I would however do like this (I know it is not using textscan, but why would you? :D )
dims = [11000 25];
data = nan(n_sim, n_seed, dims(1), dims(2));
for m_a = 1:n_sim
for m_b = 1:n_seed
data(m_a,m_b,:,:) = load(fileList{1,m_a}{m_b}, 'A1:end');
end
end

类别

在 Help Center 和 File Exchange 中查找有关 Large Files and Big Data 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by