Create a 'fake' video from an image and previously calculated optical flow vectors

2 次查看（过去 30 天）
NICOLE LONGHI
NICOLE LONGHI 2023-10-23
评论： NICOLE LONGHI 2023-10-23
I calculated Horn & Schunk's Optical Flow on a video, thus obtaining a Vx and Vy matrix containing the velocity vectors of the pixels in the video.
Now I would like to do the reverse procedure, i.e., start from frame 1 of the video, apply the Optical Flow obtained frame by frame, and create a 'fake' video (which should look like the real video), in which the pixels move according to their value of Vx and Vy of each frame.
I tried to create a code and the pixels move but I always see frame1, i.e., not all the pixels move. Could someone help me with this? Thank you
here is the code:
video = VideoReader('autoincrocio.mp4'); %choose a video
outputVideo_rec = VideoWriter('autoincrocio_INV.mp4', 'MPEG-4'); %name of the finale fake video
% Set the video properties, making sure they are within the allowed limits
outputVideo_rec.Quality = 100; % Imposta la qualità desiderata (può variare da 0 a 100)
outputVideo_rec.FrameRate = 10; %frame al secondo
open(outputVideo_rec);
%read frame 1, to get dimensions
frame1 = read(video,1);
[rows, cols, z] = size(frame1);
%loop for the n frames that I have
for p=1:30
flow=flow_save{p}; %I saved the optical flow values for each frame. Flow_save is a 1x30 row vector where at each position is the optica flow of the 30 frames
Vx = flow.Vx;
Vy = flow.Vy;
%calculation displacements
frameRate = videoReader.FrameRate;
time_between_frames = 1/frameRate;
displacementX = Vx * time_between_frames;
displacementY = Vy * time_between_frames;
% Amplifies displacements - this is because the values of displacementX and displacementY are too small, so I thought of amplifying them
scale = 100; % You can adjust this value to get a more noticeable movement
displacementX = displacementX * scale;
displacementY = displacementY * scale;
frame_sim=frame1;
% Applies shifts and generates subsequent frames
for i = 1:rows
for j = 1:cols
new_x = round(i + displacementX(i, j));
new_y = round(j + displacementY(i, j));
% Checks that the new coordinates are within the image boundaries.
if new_x >= 1 && new_x <= righe && new_y >= 1 && new_y <= colonne
frame_sim(new_x, new_y,:)=frame1(i, j,:);
end
end
end
writeVideo(outputVideo_rec, frame_sim);
end
% close video output
close(outputVideo_rec);
disp('Displacement video successfully generated.');

请先登录，再回答此问题。

采纳的回答

Walter Roberson
Walter Roberson 2023-10-23
编辑：Walter Roberson 2023-10-23
time_between_frames = 1/frameRate;
displacementX = Vx * time_between_frames;
displacementY = Vy * time_between_frames;
You are calculating one frame's-worth of displacement. That implies that you are moving relative to the previous frame.
frame_sim=frame1;
But here you are resetting to the first frame. If you are doing relative motion to the previous frame, then frame_sim should be initialized to frame1 before the loop, but otherwise motion should be allowed to accumulate. If you are doing motion since the first frame then you need to account for the fact that you are now on frame #p not on frame #1, so you have had either (p)/frameRate motion or (p-1)/frameRate motion.
  1 个评论
NICOLE LONGHI
NICOLE LONGHI 2023-10-23
Thank you very much! I modified the code like this:
...
%loop for the n frames that I have
for p=1:30
frame = read(video,p);
flow=flow_save{p}; %I saved the optical flow values for each frame. Flow_save is a 1x30 row vector where at each position is the optica flow of the 30 frames
Vx = flow.Vx;
Vy = flow.Vy;
%calculation displacements
time_between_frames = 1; the time between frames is 1 because now I calculate the time between one frame and the next frame so it is 1
displacementX = Vx * time_between_frames;
displacementY = Vy * time_between_frames;
frame_sim=frame;
...
In this way, at each iteration I read a new frame.
The resulting video is actually better with these changes, although I still see the equal background of frame 1, and above that the movement of later frames (basically I see a movement of an object above the still object image).

请先登录，再进行评论。

更多回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Optics 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by