Apply Canny Edge Detection to video and exporting it to another video file.

Question

Ahmed Mahmoud 2020-3-3

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/508782-apply-canny-edge-detection-to-video-and-exporting-it-to-another-video-file

评论： Image Analyst 2023-2-15

I have done the canny edge detection part, but I don't see to figure out how to read a video frame by frame and applying the canny edge detection to it.

I am trying to then combine the canny edge detected frames into a new video.

I have only extracted the frames and they are named "frame1.jpg" up to "frame197.jpg".

5 个评论
显示 3更早的评论隐藏 3更早的评论

Nisha Amin 2023-2-15

it is executing sir, but sir can u pls help me out fuzzy edge detection in same method instead of canny for video frames

Image Analyst 2023-2-15

@Nisha Amin I do not have the Fuzzy Toolbox and know very little about fuzzy so I can't help you. Try starting your own question or calling Mathworks Tech Support.

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Image Analyst 2020-6-6

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/508782-apply-canny-edge-detection-to-video-and-exporting-it-to-another-video-file#answer_447092

在 MATLAB Online 中打开

OK Ahmed and mersede, looks like you had trouble adapting my demos, so here is a brand new full demo for you both. The output movie has the color image in both upper quadrants, the gray scale image in the lower left quadrant, and the Canny edge image in the lower right quadrant. Adapt as needed. If it works, please Vote for this answer, and Ahmed can also "Accept this answer".

% Demo to extract frames from a movie and get the Canny edge image from the frames.
% Then rebuilds a new movie and (optionally) save individual output frames to separate image files.
% Illustrates the use of the VideoReader and VideoWriter classes.
% A Mathworks demo (different than mine) is located here http://www.mathworks.com/help/matlab/examples/convert-between-image-sequences-and-video.html
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
imtool close all;  % Close all imtool figures.
clear;  % Erase all existing variables.
workspace;  % Make sure the workspace panel is showing.
fontSize = 22;
% Open the rhino.avi demo movie that ships with MATLAB.
% First get the folder that it lives in.
folder = fileparts(which('rhinos.avi')); % Determine where demo folder is (works with all versions).
% Pick one of the two demo movies shipped with the Image Processing Toolbox.
% Comment out the other one.
movieFullFileName = fullfile(folder, 'rhinos.avi');
% movieFullFileName = fullfile(folder, 'traffic.avi');
% Check to see that it exists.
if ~exist(movieFullFileName, 'file')
	strErrorMessage = sprintf('File not found:\n%s\nYou can choose a new one, or cancel', movieFullFileName);
	response = questdlg(strErrorMessage, 'File not found', 'OK - choose a new movie.', 'Cancel', 'OK - choose a new movie.');
	if strcmpi(response, 'OK - choose a new movie.')
		[baseFileNameNoExt, folderName, FilterIndex] = uigetfile('*.avi');
		if ~isequal(baseFileNameNoExt, 0)
			movieFullFileName = fullfile(folderName, baseFileNameNoExt);
		else
			return;
		end
	else
		return;
	end
end
try
	%-----------------------------------------------------------------------------------------------------------------------------
	% Set up a VideoReader object.
	videoReaderObject = VideoReader(movieFullFileName)
	% Determine how many frames there are.
	numberOfFrames = videoReaderObject.NumFrames;
	vidHeight = videoReaderObject.Height;
	vidWidth = videoReaderObject.Width;
	
	%-----------------------------------------------------------------------------------------------------------------------------
	% Create a VideoWriter object to write the video out to a new, different file.
	writerObj = VideoWriter('NewRhinos.avi');
	% Make the output movie frame rate match the input movie frame rate.  
	% Must do this before calling open(writerObj) if you want to change the frame rate, otherwise you'll get 30 frames per second as the default.
	writerObj.FrameRate = videoReaderObject.FrameRate;
	open(writerObj);
	% Preallocate recalledMovie, which will be an array of structures.
	% First get a cell array with all the frames.
	allTheFrames = cell(numberOfFrames, 1);
	% Make the height and width double since we want to have two color images, the gray scale image, and the Canny edge image.
	allTheFrames(:) = {zeros(2 * vidHeight, 2 * vidWidth, 3, 'uint8')};
	% Next get a cell array with all the colormaps.
	allTheColorMaps = cell(numberOfFrames,1);
	allTheColorMaps(:) = {zeros(256, 3)};
	% Now combine these to make the array of structures.
	outputMovieStructure = struct('cdata', allTheFrames, 'colormap', allTheColorMaps)
	%-----------------------------------------------------------------------------------------------------------------------------
	% Set up the figure window.
	numberOfFramesWritten = 0;
	% Prepare a figure to show the images in the upper half of the screen.
	hFig = figure('Name', 'Video Demo by Image Analyst', 'NumberTitle', 'Off');
	% 	screenSize = get(0, 'ScreenSize');
	% Enlarge figure to full screen.
% 	set(gcf, 'Units', 'Normalized', 'OuterPosition', [0, 0.04, 1, 0.96]); % Old style
	hFig.WindowState = 'maximized'; % New way of maximizing.
	
	% Ask user if they want to write the individual frames out to disk.
	promptMessage = sprintf('Do you want to save the individual output frames out to individual disk files?');
	button = questdlg(promptMessage, 'Save individual frames?', 'Yes', 'No', 'Yes');
	if contains(button, 'Yes')
		writeToDisk = true;
		
		% Extract out the various parts of the filename.
		[folder, baseFileNameNoExt, extension] = fileparts(movieFullFileName);
		% Make up a special new output subfolder for all the separate
		% movie frames that we're going to extract and save to disk.
		% (Don't worry - windows can handle forward slashes in the folder name.)
		folder = pwd;   % Make it a subfolder of the folder where this m-file lives.
		outputFolder = sprintf('%s/Movie Frames from %s', folder, baseFileNameNoExt);
		% Create the folder if it doesn't exist already.
		if ~exist(outputFolder, 'dir')
			mkdir(outputFolder);
		end
	else
		writeToDisk = false;
	end
	
	%-----------------------------------------------------------------------------------------------------------------------------
	% Loop through the movie, computing the edge image and making the output movie.
	for frame = 1 : numberOfFrames
		% Extract the frame from the movie structure.
		thisFrame = read(videoReaderObject, frame);
		
		% Display it
		hImage = subplot(2, 2, 1);
		image(thisFrame);
		caption = sprintf('Frame %4d of %d.', frame, numberOfFrames);
		title(hImage, caption, 'FontSize', fontSize);
		axis('on', 'image'); % Show tick marks and get aspect ratio correct.
		drawnow; % Force it to refresh the window.
		
		% Convert to gray scale because Canny needs that.
		grayImage = rgb2gray(thisFrame);		
		% Display the gray scale image.
		subplot(2, 2, 2);
		imshow(grayImage);
		title('Gray Scale Image', 'FontSize', fontSize);
		axis('on', 'image'); % Show tick marks and get aspect ratio correct.
		
		% Update user with the progress.  Display in the command window.
		if writeToDisk
			progressIndication = sprintf('Wrote frame %4d of %d.', frame, numberOfFrames);
		else
			progressIndication = sprintf('Processed frame %4d of %d.', frame, numberOfFrames);
		end
		disp(progressIndication);
		% Increment frame count (should eventually = numberOfFrames
		% unless an error happens).
		numberOfFramesWritten = numberOfFramesWritten + 1;
		
		% Now let's do the Canny
		edgeImage = edge(grayImage, 'Canny');
		edgeImage = uint8(255 * edgeImage); % Scale 0-255
		% Make color for output movie.
		edgeImage = cat(3, edgeImage, edgeImage, edgeImage);
		% Display the Canny edge image.
		subplot(2, 2, 3);
		imshow(edgeImage);
		title('Canny Edge Image', 'FontSize', fontSize);
		axis('on', 'image'); % Show tick marks and get aspect ratio correct.
		% Convert the image into a "movie frame" structure.
		% Stitch together the frames.
		outputFrame = [thisFrame, thisFrame; cat(3, grayImage, grayImage, grayImage), edgeImage];
		outputMovieStructure(frame) = im2frame(outputFrame);
		% Write this frame out to a new video file on disk.
		writeVideo(writerObj, outputFrame);
		
		% Optional: Write the individual output movie frames to disk, if requested.
		if writeToDisk
			% Construct an output image file name.
			outputBaseFileName = sprintf('Frame %4.4d.png', frame);
			outputFullFileName = fullfile(outputFolder, outputBaseFileName);
			
			% Write it out to disk.
			imwrite(outputFrame, outputFullFileName, 'png');
		end
		
	end
	caption = sprintf('Finished all %d frames.', numberOfFrames);
	title(hImage, caption, 'FontSize', fontSize);
	close(writerObj);
	
	% Alert user that we're done.
	if writeToDisk
		finishedMessage = sprintf('Done!  It wrote %d frames to folder\n"%s"', numberOfFramesWritten, outputFolder);
	else
		finishedMessage = sprintf('Done!  It processed %d frames of\n"%s"', numberOfFramesWritten, movieFullFileName);
	end
	disp(finishedMessage); % Write to command window.
	uiwait(msgbox(finishedMessage)); % Also pop up a message box.
	
	% Enlarge figure to full screen.  Seems to stay full screen if using movie().
	hFig2 = figure;
	set(hFig2, 'Name', 'Video Demo by Image Analyst', 'NumberTitle', 'Off');
	imshow(outputFrame);
	fontSize = 14;
	title('Playing output video with the MATLAB movie() function', 'FontSize', fontSize);
	% Play the movie in the axes.  Unfortunately, there doesn't seem to be a way to show it full screen.
	movie(outputMovieStructure);   % recalledMovie is the movie structure variable, not the filename.
	close(hFig2); % Close down this movie window.
	% Note: if you want to display graphics or text in the overlay
	% as the movie plays back then you need to do it like I did at first
	% (where you extract and imshow a frame at a time.)
	
	% Ask user if they want to play the movie in an external player, like Windows Media Player.
	promptMessage = sprintf('Do you want to play the movie in an external (non-MATLAB) player?');
	button = questdlg(promptMessage, 'Play Movie?', 'Yes', 'No', 'Yes');
	if contains(button, 'Yes') && contains(computer, 'WIN', 'IgnoreCase', true) && exist('NewRhinos.avi', 'file')
		winopen('NewRhinos.avi');
	end
	
	fprintf('Done with this demo!\n');
	msgbox('Done with this demo!');
	
catch ME
	% Some error happened if you get here.
	strErrorMessage = sprintf('Error extracting movie frames from:\n\n%s\n\nError: %s\n\n)', movieFullFileName, ME.message);
	uiwait(msgbox(strErrorMessage));
end

8 个评论
显示 6更早的评论隐藏 6更早的评论

mersede maksabi 2020-6-6

Hi Image Analyst,

Your demo worked briliantly, thank you very much,

In fact I wanted to do the edge detection while video is playing, which thanks to you I could achieve that.

What I really want to do with my video, is to detect two lines (please see the attached photo.pdf) in each video frame using houghtransform! I did manage to tune houghpeak and hough lines parameter to detect these two lines. Albeit, for the blue line I had to tilt my image to make the approach work ( as it is horizontal with respect to image origin!)

The problem is that, my video has a dynamic background, lighting changes, background object repeatedly changes and etc and I dont think using hough transform with set tuning parameters will work due to different properties that each frame has (both the detected line also moving across the video frames) and sometimes after thresholding they barely determine a straight line!

I've been searching alot, and found that GPU computing might help as I want the detected lines to be visible at each frame while video is playing. I an not sure if it is going to be possible, as apprantly GPU computing also has some limitation and I might only to be able to process more than 7 fps, which is not good at all due to the application of the system I am working on!

Also not much of the image processing toolbox function are usable for GPU computing. However, I found Radon Transform, that also enbales line detection if the slope of the line with respect to x axis is known! Still not sure how to enable detetcion of the lines of ineterst at each single video frame using Radon transform!

I appreciate if you can advise me the best feasible way to achieve this task, or if you can guide me to relevant documantions that I can read and extend my knowldge about the field areas that I need to make this work!

I truly apprecaite any sort of help in advance

Image Analyst 2023-1-9

@Nurul Farhana Mohd Fadzli, yes you have to track the centroid or some point(s) on the box. Save the points into an array so that you have the position for each time (frame of the movie). Then you can scan along that array fitting a certain number of elements to a line to get the slope or angle as a function of frame time.

Nurul Farhana Mohd Fadzli 2023-1-9

@Image Analyst, Sir i only know how to do that in an image but not in video. Can you help me? Should I include this in your example demo?

请先登录，再进行评论。

Answer 2

Image Analyst 2020-3-9

1
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/508782-apply-canny-edge-detection-to-video-and-exporting-it-to-another-video-file#answer_419300

See my attached movie demos.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

Answer 3

Image Analyst 2020-6-7

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/508782-apply-canny-edge-detection-to-video-and-exporting-it-to-another-video-file#answer_447620

在 MATLAB Online 中打开

What I would do, assuming you have the Computer Vision Toolbox is to use fitPolynomialRansac(). Since the camera is fixed, you know roughly where things will be. So I'd scan down the top portion of the image getting the location of points in the middle of the frame where the line is.

[cableRows, cableColumns] = find(edgeImage(row1:row2, col1:col2));

Now pass those into fitPolynomialRansac to get the equation of the line coming down from the top.

Next I would scan columns that are known to be where that top rail is, roughly the middle quarter of the image, but only scan in the tight row band where you know the top rail will always be.

[railRows, railColumns] = find(edgeImage(row1:row2, col1:col2));

So now you have points for the top rail plus a few from the cable. So put that into fitPolynomialRansac() to find the line for the rail. Now you have two line equations and you can set they equal to each other and find out where the intersection point is.

7 个评论
显示 5更早的评论隐藏 5更早的评论

mersede maksabi 2020-6-8

编辑：mersede maksabi 2020-6-8

Hi Image Analyst,

Thank you very much for the reply,

I have been trying to see if can make the recommended approach work. However, I am not sure if I am doing this correctly!

I did convert my image to bw and then I applied canny with 0.5 threshold to have my edge image. Then I choose the following coordination for scanning:

for the cable my rows are 130:170 and my columns are 193:451

for the rail my rows are 170:181 and my columns are 193:451

when I used fine, the retuned values as cableRows and CableColumns are 173x1 double and for the rail, they are 377x1 double (I attached the returned matrices )

When I put this values to fitPolynomialRansac() and plot it, I doesnt provide me with a good fit and I have lots of outliners, and I cannot undrestand the plot!

What are the returned values in columns and rows after using find for certain boundries of the image ? are these pixel intensity and how these are calculated ? what should I do if I want to apply this on each single frame of my video?

I am truly sorry for asking you lots of question, I've been working on this task for 2 weeks now and been switching to different approaches to make this work, but sadly didn't make any progress on it yet.

mersede maksabi 2020-6-11

编辑：mersede maksabi 2020-6-11

在 MATLAB Online 中打开

Hello Imgae Analyst

I managed to detect the line of interest as I am sure detection of a horizontal line and almost a vertical moving lines across the entire video frame through hough transform is not possible!

The code that I am using can detect the cable and I am thinking to use the column number of point2 (where the wire interating with the rail)to show the lateral movement of the cable across the rail!

the video is 3 s and I have 80 frames, at each frame I can see the detection of the line, however the store hLines coordination are less than the frame number. I need to show the location at each single frame in order to plot the graph!

So out of 80 frames I only have 52 returned coordination!

i was thinking maybe this is to do with the fps rate, so I reduced it to 10 and again out of nearly 32 frames it returned only 25 coordination,

Is it possible that there is no detection at some of the frames and bc it shows the detection while the video is playing, the detection of the previous frame also appreas in the next frame with no detection? !!

is there any way that I can define another condition to say if there is no hLines detected in the current frame return zero to fill the array component in away that I can identify the frame with no detection from the returned coordination and plot my graph !

I am using the following code, and the attached is my.mat file which are coordination of the detected line in 52 frame.

I appreciate your help in advance

	store = []; 
	% Loop through the movie, computing the edge image and making the output movie.
	for frame = 1 : numberOfFrames
		% Extract the frame from the movie structure.
		thisFrame = read(videoReaderObject, frame);
		
		drawnow; % Force it to refresh the window.
		% Convert to gray scale because Canny needs that.
		grayImage = rgb2gray(thisFrame);
        BW = im2bw(grayImage);
		
		% Update user with the progress.  Display in the command window.
		if writeToDisk
			progressIndication = sprintf('Wrote frame %4d of %d.', frame, numberOfFrames);
		else
			progressIndication = sprintf('Processed frame %4d of %d.', frame, numberOfFrames);
		end
		disp(progressIndication);
		% Increment frame count (should eventually = numberOfFrames
		% unless an error happens).
		numberOfFramesWritten = numberOfFramesWritten + 1;
		
		% Now let's do the Canny
		edgeImage = edge(BW, 'Canny',0.9);
		
%[H,T,R]= hough(edgeImage); 
%identify peaks in Hough Transfrom 
 start_angle = -45;
end_angle = 45;
theta_resolution = 0.5;
[H,T,R]= hough(edgeImage,'Theta', start_angle:theta_resolution:end_angle);
hPeaks = houghpeaks(H,2,'threshold',ceil(0.5*max(H(:))));
% hPeaks = houghpeaks(H,2,'NhoodSize',[151 151],'threshold',ceil(0.3*max(H(:))));
  hLines = houghlines(BW,T,R,hPeaks,'FillGap',15,'MinLength',330);
  store = [store,hLines];
     [linePos,markerPos] = getVizPosArray(hLines);    %getViz Utility Function 
     %for lidx = 1:length(hLines)
   %linePos = [linePos;...
             %  hLines(lidx).point1 hLines(lidx).point2];
      % store =[store;linePos];   
    % Get linePos in [x1 y1;x2 y2...] format. 
   %markerPos = [markerPos;...
              %[hLines(lidx).point1 markerSize];...
              %[hLines(lidx).point2 markerSize]];
       %  storepoint= [storepoint;markerPos];     
    % Get markerPos in [x1 markerSize;y1 markerSize...] format. 
       if(~isempty(hLines));
        lineFrame = insertShape(thisFrame,'Line',linePos,...
      'color','blue','LineWidth',5);
  outFrame = insertObjectAnnotation(lineFrame,...
      'circle',markerPos,'','Color','yellow','LineWidth',5);
  
       else
        outFrame =thisFrame; 
 
       end
 
       

Image Analyst 2020-6-11

You're not being annoying. It's just that this could end up being a major project as we encounter more unusual cases, and I usually work on Answers questions for just a few minutes, not hours, days, or weeks. I'm dealing with that now on my main project. What looked rather straightforward has now turned rather complicated as we find more images it doesn't work with. So I fix it to handle those and sometimes it breaks with the original images that it used to work with. It's like "Whack-a-mole".

There's a reason why they say in software development "The first 90% of the project takes the first 90% of the time, and the last 10% of the project takes the second 90% of the time."

Or "When estimating the time to complete a project, double your initial number, then go to the next higher units."

mersede maksabi 2020-6-12

Thank you for the advise, I appreciate it! I did spend considerable time extracting another parameter of my system using cascadeclassifer, but my supervisor believes it shouldn't be the case and I have to develop my own code! I have been searching alot and reading alot to find suitbale approach but considering the fact that my camera is moving and each time I have new features, has made the foreground detection impossible! I spent considerable time to tune my cascade classifier and supply samples to work with the approach! I am so lost! Anyway, thank you and I wish you luck with yours.

请先登录，再进行评论。

Apply Canny Edge Detection to video and exporting it to another video file.

5 个评论
显示 3更早的评论隐藏 3更早的评论

采纳的回答

8 个评论
显示 6更早的评论隐藏 6更早的评论

更多回答（2 个）

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

7 个评论
显示 5更早的评论隐藏 5更早的评论

另请参阅

类别

标签

产品

Community Treasure Hunt

Apply Canny Edge Detection to video and exporting it to another video file.

5 个评论 显示 3更早的评论隐藏 3更早的评论

采纳的回答

8 个评论 显示 6更早的评论隐藏 6更早的评论

更多回答（2 个）

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

7 个评论 显示 5更早的评论隐藏 5更早的评论

另请参阅

类别

标签

产品

Community Treasure Hunt

5 个评论
显示 3更早的评论隐藏 3更早的评论

8 个评论
显示 6更早的评论隐藏 6更早的评论

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

7 个评论
显示 5更早的评论隐藏 5更早的评论