Video preview has growing delay when used in uifigure, but not in normal figure

11 次查看（过去 30 天）
Raphael Kriegl
Raphael Kriegl 2020-8-20
评论： Luisa Cruz 2020-9-25
Hey there,
I am building an app which includes axes to display a preview of my camera (gentl adaptor). I created the axes with uiaxes.
My problem is, that the video is really laggy and the delay grows the longer the preview is running. When I stop the preview, it keeps running until it displays the last recorded frame. The memory usage of matlab is also steadily increasing while it is running.
If I try the example of a custom preview GUI everything runs smoothly. But that one does not create a axis object at all.
I also tried it this way:
figure
ax = axes
im = image(ax, zeros(1542, 2064,3 ));
axis(ax, 'image')
vid = videoinput('gentl')
preview(vid, im)
And this also works fine.
I then tried replacing the uiaxes object in my app with an axes object (same parent), but then it does not show.
Is there a way to not use the uiaxes, because I think that thats the issue?
I also noticed, that the uiaxes do not use as much GPU as the axes, if thats useful.
This is my code (was created with app designer and then ported to normal matlab code in an attempt to fix the lag):
classdef Preview_Window < handle
% Properties that correspond to app components
properties (Access = public)
UIFigure matlab.ui.Figure
Menu matlab.ui.container.Menu
Menu2 matlab.ui.container.Menu
GridLayout matlab.ui.container.GridLayout
LeftPanel matlab.ui.container.Panel
GridLayout2 matlab.ui.container.GridLayout
ResetCameraInterfaceButton matlab.ui.control.Button
PreviewPanel matlab.ui.container.Panel
GridLayout4 matlab.ui.container.GridLayout
StartButton matlab.ui.control.Button
StopButton matlab.ui.control.Button
RightPanel matlab.ui.container.Panel
UIAxes matlab.ui.control.UIAxes
Timestamp matlab.ui.control.Label
end
% Properties that correspond to apps with auto-reflow
properties (Access = private)
onePanelWidth = 576;
end
properties (Access = private)
feed; % camera feed
src; %getsource(feed)
canvas; %image handle to show the preview on
end
methods (Access = private)
function preview_fcn(app, ~, event, himage)
% callback of preview()
% display some additional information
% Get timestamp for frame.
tstampstr = event.Timestamp;
framerate = event.FrameRate;
%status = event.Status;
resolution = event.Resolution;
% Set the value of the text label.
app.Timestamp.Text = strcat(tstampstr, " | " , num2str(framerate), " | ", resolution);
%app.Status.Text = strcat( "| ", status, " | :", resolution);
% Display image data.
himage.CData = event.Data;
end
function reset_camera(app)
% setup camera and interface
app.ResetCameraInterfaceButton.Enable = false;
% preset axes with black picture
app.canvas = image(app.UIAxes, zeros(1542, 2064, 3) );
colormap(app.UIAxes, gray);
axis(app.UIAxes, 'image');
%reset camera interface and try to open camera feed
imaqreset
try
app.feed = videoinput("gentl", 1 , 'Mono8');
catch
warndlg("Can't open Camera feed. Fix and reset interface!", "Warning!")
return
end
%Settings for camera
triggerconfig(app.feed, 'manual');
app.feed.FramesPerTrigger = 1;
app.src = getselectedsource(app.feed);
app.src.ReverseY = 'True';
app.src.ExposureTime = 60000;
% Setup preview container
frame = getsnapshot(app.feed);
vidRes = app.feed.VideoResolution;
nBands = app.feed.NumberOfBands;
app.canvas = image(app.UIAxes, zeros(vidRes(2), vidRes(1), nBands));
app.UIAxes.XLim = [0, double(vidRes(1))];
app.UIAxes.YLim = [0, double(vidRes(2))];
pbaspect(app.UIAxes, [vidRes(1),vidRes(2),1]);
app.canvas.CData = frame;
% activate custom preview callback
setappdata(app.canvas, 'UpdatePreviewWindowFcn', @app.preview_fcn);
% enable buttons
app.StartButton.Enable = true;
app.ResetCameraInterfaceButton.Enable = true;
end
end
% Callbacks that handle component events
methods (Access = private)
% Code that executes after component creation
function startupFcn(app)
app.UIAxes.Title.String = '';
app.UIAxes.XLabel.String = '';
app.UIAxes.YLabel.String = '';
app.UIAxes.Visible = 'off';
app.UIAxes.XTick = [];
app.UIAxes.YTick = [];
disableDefaultInteractivity(app.UIAxes)
app.UIAxes.PlotBoxAspectRatioMode = 'manual';
app.reset_camera;
end
% Changes arrangement of the app based on UIFigure width
function updateAppLayout(app, event)
currentFigureWidth = app.UIFigure.Position(3);
if(currentFigureWidth <= app.onePanelWidth)
% Change to a 2x1 grid
app.GridLayout.RowHeight = {480, 480};
app.GridLayout.ColumnWidth = {'1x'};
app.RightPanel.Layout.Row = 2;
app.RightPanel.Layout.Column = 1;
else
% Change to a 1x2 grid
app.GridLayout.RowHeight = {'1x'};
app.GridLayout.ColumnWidth = {220, '1x'};
app.RightPanel.Layout.Row = 1;
app.RightPanel.Layout.Column = 2;
end
end
% Button pushed function: StartButton
function StartButtonPushed(app, event)
app.StopButton.Enable = true;
app.StartButton.Enable = false;
%app.src.ExposureAuto = 'Continuous';
preview(app.feed, app.canvas);
end
% Close request function: UIFigure
function UIFigureCloseRequest(app, event)
try
stoppreview(app.feed);
catch
end
try
delete(app.feed);
catch
end
delete(app)
end
% Button pushed function: StopButton
function StopButtonPushed(app, event)
app.StopButton.Enable = false;
app.StartButton.Enable = true;
stoppreview(app.feed);
end
% Button pushed function: ResetCameraInterfaceButton
function ResetCameraInterfaceButtonPushed(app, event)
app.reset_camera;
end
end
% Component initialization
methods (Access = private)
% Create UIFigure and components
function createComponents(app)
% Create UIFigure and hide until all components are created
app.UIFigure = uifigure('Visible', 'off');
app.UIFigure.AutoResizeChildren = 'off';
app.UIFigure.Position = [100 100 640 480];
app.UIFigure.Name = 'MATLAB App';
app.UIFigure.CloseRequestFcn = @(obj, event)app.UIFigureCloseRequest;
app.UIFigure.SizeChangedFcn = @(obj, event)app.updateAppLayout;
% Create Menu
app.Menu = uimenu(app.UIFigure);
app.Menu.Text = 'Menu';
% Create Menu2
app.Menu2 = uimenu(app.UIFigure);
app.Menu2.Text = 'Menu2';
% Create GridLayout
app.GridLayout = uigridlayout(app.UIFigure);
app.GridLayout.ColumnWidth = {220, '1x'};
app.GridLayout.RowHeight = {'1x'};
app.GridLayout.ColumnSpacing = 0;
app.GridLayout.RowSpacing = 0;
app.GridLayout.Padding = [0 0 0 0];
app.GridLayout.Scrollable = 'on';
% Create LeftPanel
app.LeftPanel = uipanel(app.GridLayout);
app.LeftPanel.Layout.Row = 1;
app.LeftPanel.Layout.Column = 1;
% Create GridLayout2
app.GridLayout2 = uigridlayout(app.LeftPanel);
app.GridLayout2.RowHeight = {'1x', '1x', '1x', '1x', '1x', '1x', '1x', '1x'};
app.GridLayout2.RowSpacing = 25;
% Create ResetCameraInterfaceButton
app.ResetCameraInterfaceButton = uibutton(app.GridLayout2, 'push');
app.ResetCameraInterfaceButton.ButtonPushedFcn = @(obj, event)app.ResetCameraInterfaceButtonPushed;
app.ResetCameraInterfaceButton.Enable = 'off';
app.ResetCameraInterfaceButton.Layout.Row = 6;
app.ResetCameraInterfaceButton.Layout.Column = 1;
app.ResetCameraInterfaceButton.Text = {'Reset Camera'; 'Interface'};
% Create PreviewPanel
app.PreviewPanel = uipanel(app.GridLayout2);
app.PreviewPanel.Title = 'Preview';
app.PreviewPanel.Layout.Row = [1 3];
app.PreviewPanel.Layout.Column = 1;
% Create GridLayout4
app.GridLayout4 = uigridlayout(app.PreviewPanel);
app.GridLayout4.ColumnWidth = {'1x'};
% Create StartButton
app.StartButton = uibutton(app.GridLayout4, 'push');
app.StartButton.ButtonPushedFcn = @(obj, event)app.StartButtonPushed;
app.StartButton.Enable = 'off';
app.StartButton.Layout.Row = 1;
app.StartButton.Layout.Column = 1;
app.StartButton.Text = 'Start';
% Create StopButton
app.StopButton = uibutton(app.GridLayout4, 'push');
app.StopButton.ButtonPushedFcn = @(obj, event)app.StopButtonPushed;
app.StopButton.Enable = 'off';
app.StopButton.Layout.Row = 2;
app.StopButton.Layout.Column = 1;
app.StopButton.Text = 'Stop';
% Create RightPanel
app.RightPanel = uipanel(app.GridLayout);
app.RightPanel.Layout.Row = 1;
app.RightPanel.Layout.Column = 2;
% Create UIAxes
app.UIAxes = uiaxes(app.RightPanel);
title(app.UIAxes, 'Title')
xlabel(app.UIAxes, 'X')
ylabel(app.UIAxes, 'Y')
app.UIAxes.PlotBoxAspectRatio = [1 1.02770083102493 1];
app.UIAxes.Box = 'on';
app.UIAxes.Position = [6 28 408 425];
% Create Timestamp
app.Timestamp = uilabel(app.RightPanel);
app.Timestamp.Position = [6 1 408 22];
app.Timestamp.Text = '';
% Show the figure after all components are created
app.UIFigure.Visible = 'on';
end
end
% App creation and deletion
methods (Access = public)
% Construct app
function app = Preview_Window
% Create UIFigure and components
createComponents(app)
% Execute the startup function
app.startupFcn
if nargout == 0
clear app
end
end
% Code that executes before app deletion
function delete(app)
% Delete UIFigure when app is deleted
delete(app.UIFigure)
end
end
end
Thanks in advance for any help!

请先登录，再回答此问题。

采纳的回答

Raphael Kriegl
Raphael Kriegl 2020-8-25
编辑：Raphael Kriegl 2020-9-4
I got an answer from the MATLAB support team:
After collaborating with my colleagues I figured that this is a known limitation with the "uiaxes" performance in terms of rendering the images. Currently, "uiaxes"/"uifigure" (javascript based) does not support high frame rates like the traditional MATLAB figure window (java based) and the frames passed to the "uiaxes" are queued for rendering, this is causing the memory increase they noticed. For the same reason, it will cause a lag which worsens with larger images and high acquisition frame rates if the acquisition is running for a long duration .
Until the performance bottleneck in the "uiaxes" is addressed, there is little we can do to help you, if you goal is to have the live preview running in the app.
Having said that, for the time being, I would suggest you to open the preview in traditional MATLAB figure window outside of the app and i believe you have already tried this.
Another thing to experiment is to turn the imaqmex's "slowpreview" flag which currently sets a max limit of 10 frames for previewing. However, with the "uiaxes" rendering performance even 10 FPS might be high. Following is the code to enable slow preview.
>> imaqreset
>> imaqmex('feature', '-slowpreview', 'true')
>> vid = videoinput(...)Please note that this is an undocumented feature and it could change and/or be removed in the future.
Additionally, lowering the resolution and/or ROI can help as this decreases the load on the "uifigure".

更多回答（1 个）

Evan Bates
Evan Bates 2020-9-4
Raphael, thank you for posting that response from Matworks. That is a huge help. I just ran into this issue as well. I’m glad the problem is not on my end. They should add functionality in the preview function to set the playback framerate.
I found a workaround to this issue. I made a UpdatePreivewWindowFcn and inside the callback function I added a pause(1/60). The preview function will dump frames if the UpdatePreivewWindowFcn has not finished executing, therefore slowing the video to the desired framerate (frame rate is set by the time in the pause).
Hope this helps

类别

在 Help Center 和 File Exchange 中查找有关 Image Preview and Device Configuration 的更多信息

标签

产品


版本

R2020a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by