Speed up this fread loop

3 次查看（过去 30 天）
T
T 2017-4-1
编辑： Carl 2017-4-4
I need your help! I am going to read a lot of big files with fread. Any ideas on how to speed this up? Also if there is a way to estimate N from the length of the file before allocating B1 to B8? Every advice is greatly appreciated
fid = fopen(Filename,'r');
B1=zeros(m, N,'single');
B2=zeros(m, N,'single');
B3=zeros(m, N,'single');
B4=zeros(m, N,'single');
B5=zeros(m, N,'single');
B6=zeros(m, N,'single');
B7=zeros(m, N,'single');
B8=zeros(m, N,'single');
fseek(fid,headerlength, 'bof');
fseek(fid, offset, 0);
for k = 1 : N
try
tag(k)= fread(fid,1,'int64');
catch
disp('End of file')
break
end
Idontneedthis = fread(fid,2,'uint64');
Idontneedthis= fread(fid,1,'int');
Idontneedthis = fread(fid,1,'int');
try
A = fread(fid,m*8,'short','ieee-be');
A = reshape(A,m,8);
A=single(A);
catch
disp('End of file')
break
end
B1(:,k) = A(:,1);
B2(:,k) = A(:,2);
B3(:,k) = A(:,3);
B4(:,k) = A(:,4);
B5(:,k) = A(:,5);
B6(:,k) = A(:,6);
B7(:,k) = A(:,7);
B8(:,k) = A(:,8);
end
fclose(fid);
Thanks a lot!!!

请先登录，再回答此问题。

回答（2 个）

Carl
Carl 2017-4-4
编辑：Carl 2017-4-4
Hi T,
  1. If you are creating the files yourself, you can store the value of N at the beginning of the file.
  2. You can replace the assignments to 'Idontneedthis' with a call to fseek.
  3. You can also look into using the Parallel Computing Toolbox for doing the read in parallel. Here is an example of doing so with the 'parfor' loop: http://www.mathworks.com/matlabcentral/newsreader/view_thread/307594#835402
Jan
Jan 2017-4-4
编辑：Jan 2017-4-4
I simply added the bytes per block and divided the file size by this considering the "headerlength" and the "offset". This seems to be trivial maths.
m = 16; % Guessed
FileInfo = dir(Filename);
N = (FileInfo.bytes - headerlength - offset) / (3 * 8 + 4 + 4 + m * 8 * 2);
fid = fopen(Filename, 'r', 'ieee-be');
if fid == -1 % I've seen too many code failing silently...
error('Cannot open file: %s', Filename);
end
B = zeros(m, 8, N, 'single');
tag = zeros(1, N, 'uint64');
fseek(fid, headerlength + offset, 'bof');
for k = 1:N
tag(k) = fread(fid, 1, 'int64');
fread(fid, 3, 'uint64'); % Skip 24 bytes
B(:, :, k) = fread(fid, [m, 8], 'int16=>single');
end
fclose(fid);
Do not create 8 different variables with an index in the name. Prefer a 3D array instead. Please test this - due to the missing of test data I cannot run this.

类别

在 Help Center 和 File Exchange 中查找有关 Multifunction Radar 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by