Large Binary Data Files: Can I Animate while Simultaneously Loading the Next Data Block Into Memory from Disk?

2011 5 20
1 个回答
回答已采纳
2 次查看（30 天）

0 个投票

Hi There,
I'm making tools for visualizing very large data sets. A typical routine will often look something like:
function AnimateFile(fidIn,BytesPerStep,NumSteps,StepsPerBlock,GraphicsHandles)
NumBlocks = ceil(NumSteps/StepsPerBlock);
for BlockNum = 1:NumBlocks
DataBlock = fread(fidIn,[BytesPerStep StepsPerBlock],'*uint8');
for StepNum = 1:size(DataBlock,2)
MakePrettyPictures(DataBlock(:,StepNum),GraphicsHandles)
end
end
But of course this means that the animation runs in fits and starts as it jumps back and forth between rendering in the inner loop and waiting for the fread() in the outer loop to complete. Is there some way of making the fread() command run "in the background" to load up the data for the next block while the current block is being animated?
-Jan

请先登录，再回答此问题。

 采纳的回答

Oleg Komarov
Oleg Komarov 2011-5-20

0 个投票

EDIT
  1. Read in first block
  2. Start timer which executes the TimerFcn after the previous execution has ended (so no hard coded delays). The TimerFcn now doesn't share the same workspace so it should be enough to pass DataBlock as an argument to make one single copy
  3. In the meantime the import should proceed overwriting the DataBlock with the condition that TimerFcn is working on the previous
Let me know if it works.
function AnimateFile(fidIn,BytesPerStep,NumSteps,StepsPerBlock,GraphicsHandles)
NumBlocks = ceil(NumSteps/StepsPerBlock);
% Read in first block
DataBlock = fread(fidIn,[BytesPerStep StepsPerBlock],'*uint8');
% Create timer object which executes mpp every .5 seconds with a delay of
% .5 second to wait for the first chunk of data to be loaded
t = timer('ExecutionMode' , 'fixedSpacing',...
'Period' , 0 ,...
'BusyMode' , 'queue' ,...
'TimerFcn' ,{@mpp,DataBlock,GraphicsHandles});
start(t);
for b = 2:NumBlocks
% Cannot load the third block if still executing the first
% Example: if b = 3 and TaskExecuted = 1 then it can proceed otherways wait
while b - get(t,'TasksExecuted') > 2 && strcmp(get(t,'Running'),'on')
pause(0.1)
end
% Overwrite
DataBlock = fread(fidIn,[BytesPerStep StepsPerBlock],'*uint8');
end
end
% Sub function
function mpp(varargin)
for StepNum = 1:size(varargin{3},2)
MakePrettyPictures(varargin{3}(:,StepNum),varargin{4})
end
end

4 个评论
显示 2更早的评论 隐藏 2更早的评论

Jan Rubak
Jan Rubak 2011-5-20
Hi Oleg,
Thanks for your response. I haven't used timers before, so I'll have to have a closer look at that option. My main constraint is that I'm dealing with very large data files, which is why I have to load them in one or two blocks at a time. Is there some way to delay the read from file operation for each data block until the animation of the previous data block commences (and to then delete each data block from memory once its animation has been carried out)?
Also, I'd prefer not to use hard-coded delays like 0.5, as the data load time will vary with BytesPerBlock and with the hardware. Thoughts?
Cheers,
Jan
Jan Rubak
Jan Rubak 2011-7-28
Hi Oleg,
Thank you for this. I got pulled off onto a different task, so I haven't had a chance to test this yet. I will report back after I do.
Cheers,
Jan

请先登录，再进行评论。

更多回答（0 个）

类别

在 帮助中心 和 File Exchange 中查找有关 Simulation 的更多信息

产品

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by