Writing Video Frames in Parallel

8 次查看（过去 30 天）

Jacob Mevorach 2017-5-29

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/342330-writing-video-frames-in-parallel

编辑： Jacob Mevorach 2017-5-30

So I came up with a way to write video frames in parallel so that I could take advantage of all the cores in my computer only I keep getting a weird bug where I can only write videos up to 52.1 MB. Does anyone have any idea what might be going on? Below is my code I've been using to write video frames in parallel.

spmd(1,4)
open(videoWriter);
CountIndex = [0];
for i = labindex:numlabs:numFiles 
  if labindex == 1
  frame1 = imread(fullfile(workingDirectory, stackName, imageFiles(i).name));
  end
  if labindex == 2
  frame2 = imread(fullfile(workingDirectory, stackName, imageFiles(i).name));
  end
  if labindex == 3
  frame3 = imread(fullfile(workingDirectory, stackName, imageFiles(i).name));
  end
  if labindex == 4
  frame4 = imread(fullfile(workingDirectory, stackName, imageFiles(i).name));
  end
    while ~any(abs(CountIndex-(labindex-1))<1e-10)
    end
    if labindex == 1
    writeVideo(videoWriter,frame1);
    end
    if labindex == 2
    writeVideo(videoWriter,frame2);
    end
    if labindex == 3
    writeVideo(videoWriter,frame3);
    end
    if labindex == 4
    writeVideo(videoWriter,frame4);
    end
    CountIndex = [CountIndex labindex];
    if length(CountIndex) == length(numlabs)+1
        CountIndex = [0];
    end
end
end
end

6 个评论
显示 4更早的评论隐藏 4更早的评论

Walter Roberson 2017-5-30

在 MATLAB Online 中打开

You should not need that extra labSend / labReceive: the labReceive specifies which worker it is waiting to hear from and it cycles through the workers, so if (say) worker 3 has the data ready before worker #2, then the data from worker 3 will sit in the queue until the labReceive(3) is executed. If worker 3's data is not ready yet then labReceive(3) would wait for it.

The one thing I do not know, though, is what happens if you cue multiple items, whether a single labReceive will pick them all up.

Here is another way to avoid the labSend / labReceive that you added:

spmd
  if labindex == numlabs
    open(videoWriter);
    for i = 1 : numFiles
      frame = labReceive('any', i);
      writeVideo(videoWriter, frame);
    end
  else      
    for i = labindex:numlabs-1:numFiles
      frame = imread( imageFiles(i).name );
      labSend(frame, numlabs, i);
    end
  end
end

But there is also a completely different approach that does not use spmd: use parfeval()

p = parpool();
f(1:numFiles) = parallel.FevalFuture;
for i = 1  : numFiles
  f(i) = parfeval(p, @imread, 1, imageFiles(i).name );
end
open(videoWriter);
for i = 1 : numFiles
  frame = fetchOutputs(f(i));
  writeVideo(frame);
end
delete(f);

Jacob Mevorach 2017-5-30

编辑：Jacob Mevorach 2017-5-30

I put in the extra labreceive/labsend because labsend can potentially return before its corresponding labreceive completes its operation (see here: https://www.mathworks.com/help/distcomp/labsend.html). This has to do with the buffer size of the underlying MPI library's buffering behavior (see section 5.1.3 of this paper for a good explanation: http://paperity.org/p/17803450/matlab-r-a-language-for-parallel-computing). This would mean that if one was getting ready to send but two had already finished sometimes two would return early which would cause it to send frame n+3 rather than the frame n we wanted it to send which would in turn result in frame number n being dropped.

The SPMD example with the tags you sent seems like another solution to this problem.

请先登录，再进行评论。

请先登录，再回答此问题。