Perform Image Acquisition and Parallel Image Processing

Extract Device Information and Set Up Video Output

Clear previous image acquisition objects and extract information about the video device currently connected to the machine.

objects = imaqfind;
delete(objects);
imaqreset;
deviceInfo = imaqhwinfo('winvideo')

deviceInfo = struct with fields:
       AdaptorDllName: 'adaptor.dll'
    AdaptorDllVersion: '6.1 (R2019b)'
          AdaptorName: 'winvideo'
            DeviceIDs: {[1]}
           DeviceInfo: [1×1 struct]

Check if a folder for the output video already exists in the current directory. If no folder for output video exists, create one.

if ~isfolder('OutputFolder')
    mkdir OutputFolder
end 

To write video data to an AVI file in the output folder, create a VideoWriter object.

videoOut = VideoWriter('OutputFolder/myVideo.avi');

Set Up Parallel Environment

To enable the offloading of postprocessing to the workers, first start a parallel pool.

p = parpool('Processes');

Starting parallel pool (parpool) using the 'Processes' profile ...
Connected to parallel pool with 6 workers.

Create a parallel.pool.Constant object to create a denoising network only once in the workers and use it to filter the noise out from the frames.

C = parallel.pool.Constant(@() denoisingNetwork('dncnn'));

To send the postprocessed frames back from the workers and write them in order, use an OrderedDataQueue. Set a callback to write the frames to disk by using afterEach.

Q = OrderedDataQueue;
afterEach(Q,@(frame) writeVideo(videoOut,frame));

The OrderedDataQueue object is defined in a supporting file to this example. If you want to use it in your own code, copy and place it with the rest of your files.

Set Up Video Input Object

Create a video input object. Set the object to perform acquisition in the client frame by frame.

videoIn = videoinput('winvideo',1,'YUY2_800x600')

Summary of Video Input Object Using 'Microsoft® LifeCam Cinema(TM)'.

   Acquisition Source(s):  input1 is available.

  Acquisition Parameters:  'input1' is the current selected source.
                           10 frames per trigger using the selected source.
                           'YUY2_800x600' video data to be logged upon START.
                           Grabbing first of every 1 frame(s).
                           Log data to 'memory' on trigger.

      Trigger Parameters:  1 'immediate' trigger(s) on START.

                  Status:  Waiting for START.
                           0 frames acquired since starting.
                           0 frames available for GETDATA.

videoIn.ReturnedColorSpace = 'RGB';
videoIn.FramesPerTrigger = Inf;
videoIn.FramesAcquiredFcnCount = 1;

Set the video writing frame rate to the same rate as for video reading, and open the video output object.

src = videoIn.Source;
videoOut.FrameRate = str2double(src.FrameRate);
open(videoOut);

To start postprocessing operations after each frame is acquired, define a FramesAcquiredFcn callback for the video input object and start the acquisition.

videoIn.FramesAcquiredFcn = {@postProcessAndWrite,C,Q};
start(videoIn);

Create a preview window. You can stop the video as soon as the preview is manually closed by using waitfor on the figure handle hPreviewFig. For this example, stop video acquisition after 2 seconds.

hPreviewImg = preview(videoIn);
hPreviewFig = ancestor(hPreviewImg,'figure');
pause(2);
stop(videoIn);

The postprocessing function stores a future variable in the UserData property of the video object. This variable represents a future execution of the video write operations. To close the video writer after all the data is written to the output file, use afterAll on this future variable.

postProcessFutures = videoIn.UserData;
closeVideoFuture = afterAll(postProcessFutures,@() close(videoOut),0);

The postprocessing operation in this example is can take a few minutes. On a Windows 10, Intel® Xeon® W-2133 3.60 GHz CPU, with 6 cores, postprocessing took 4 minutes.

You can use a waitbar to track the postprocessing progress. To update the waitbar after each postprocessing operation finishes, use afterEach. To close the waitbar after all operations finish, use afterAll. For more information, see Update User Interface Asynchronously Using afterEach and afterAll.

h = waitbar(0,'Postprocessing...');
updateWaitbarFuture = afterEach(postProcessFutures, ...
    @(~) waitbar(sum(strcmp('finished',{postProcessFutures.State}))/numel(postProcessFutures),h), 1);
afterAll(closeVideoFuture, @() close(h),0);

Block execution in the client session until the writing finishes by waiting for the future variable.

wait(closeVideoFuture);

Delete the video input object when finished.

delete(videoIn);

Visualize Results

After the video file has been created, you can visualize the results.

Use a VideoReader object to read the video file.

vidObj = VideoReader('OutputFolder/MyVideo.avi');

Read some frames by using the readFrame function.

images = cell(1,5);
times = .4:.4:2;
for ii = 1:numel(times)
    vidObj.CurrentTime = times(ii);
    images{ii} = readFrame(vidObj);
end

To visualize the frames, use the montage function.

montage(images,'Size',[1 5])

Define Helper Functions

Define the main postprocessing routine, which is executed after each frame acquisition. This function postProcessAndWrite fetches the data from the video input object and calls parfeval to start the frame denoising in a parallel worker.

function postProcessAndWrite(videoIn,~,C,Q)
    [frame,~,metadata] = getdata(videoIn,1); 
    postProcessFuture = parfeval(@postProcess,0,frame,C,Q,metadata.FrameNumber);
    videoIn.UserData = [videoIn.UserData postProcessFuture];
end

Define the postprocessing function to be executed in the worker. For this example, to simplify computation, convert each frame to gray, and then denoise it by using the denoiseImage function. The function postProcess takes the frame and the denoising network object stored in the Value field of the parallel.pool.Constant object as inputs. For more information on denoising images with a denoising neural network, see Get Pretrained Image Denoising Network (Image Processing Toolbox).

function postProcess(frame,C,Q,frameNumber)
    grayFrame = im2double(rgb2gray(frame));
    denoisedGrayFrame = denoiseImage(grayFrame,C.Value);
    denoisedGrayFrame = im2uint8(denoisedGrayFrame);
    send(Q,frameNumber,denoisedGrayFrame)
end