Change lidar labeler app's core function to read into continuously updating .pcd file

Question

kevin harianto 2022-3-28

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1682614-change-lidar-labeler-app-s-core-function-to-read-into-continuously-updating-pcd-file

I am looking into the possibilities in which to read into a .pcd file that is being continuously updated using the machine learning algorithm that will allow the lidar labeller app to do so. This applications end result is too read into the area where the lidar sensor would store the .pcd files so that it would import them and "concatonate" the next point clouds. This is in addition with the help from the automated ground truth labeling eaxmple.

classdef LidarSemanticSegmentation < lidar.labeler.AutomationAlgorithm
    % LidarSemanticSegmentation Automation algorithm performs semantic 
    % segmentation in the point cloud.
    %   LidarSemanticSegmentation is an automation algorithm for segmenting
    %   a point cloud using SqueezeSegV2 semantic segmentation network
    %   which is trained on Pandaset data set.
    %
    %   See also lidarLabeler, groundTruthLabeler 
    %   lidar.labeler.AutomationAlgorithm.
    
    %   Copyright 2021 The MathWorks, Inc.
    % ----------------------------------------------------------------------
    % Step 1: Define the required properties describing the algorithm. This
    % includes Name, Description, and UserDirections.
    properties(Constant)
        % Name Algorithm Name
        %   Character vector specifying the name of the algorithm.
        Name = 'Lidar Semantic Segmentation';
        % Description Algorithm Description
        %   Character vector specifying the short description of the algorithm.
        Description = 'Segment the point cloud using SqueezeSegV2 network.';
        % UserDirections Algorithm Usage Directions
        %   Cell array of character vectors specifying directions for
        %   algorithm users to follow to use the algorithm.
        UserDirections = {['ROI Label Definition Selection: select one of ' ...
            'the ROI definitions to be labeled'], ...
            'Run: Press RUN to run the automation algorithm. ', ...
            ['Review and Modify: Review automated labels over the interval ', ...
            'using playback controls. Modify/delete/add ROIs that were not ' ...
            'satisfactorily automated at this stage. If the results are ' ...
            'satisfactory, click Accept to accept the automated labels.'], ...
            ['Accept/Cancel: If the results of automation are satisfactory, ' ...
            'click Accept to accept all automated labels and return to ' ...
            'manual labeling. If the results of automation are not ' ...
            'satisfactory, click Cancel to return to manual labeling ' ...
            'without saving the automated labels.']};
    end
    % ---------------------------------------------------------------------
    % Step 2: Define properties you want to use during the algorithm
    % execution.
    properties
        % AllCategories
        % AllCategories holds the default 'unlabelled', 'Vegetation',
        % 'Ground', 'Road', 'RoadMarkings', 'SideWalk', 'Car', 'Truck', 
        % 'OtherVehicle', 'Pedestrian', 'RoadBarriers', 'Signs',
        % 'Buildings' categorical types.
        AllCategories = {'unlabelled'};
        % PretrainedNetwork
        %   PretrainedNetwork saves the pretrained SqueezeSegV2 network.
        PretrainedNetwork
    end
    %----------------------------------------------------------------------
    % Note: this method needs to be included for lidarLabeler app to
    % recognize it as using pointcloud
    methods (Static)
        % This method is static to allow the apps to call it and check the
        % signal type before instantiation. When users refresh the
        % algorithm list, we can quickly check and discard algorithms for
        % any signal that is not support in a given app.
        function isValid = checkSignalType(signalType)
            isValid = (signalType == vision.labeler.loading.SignalType.PointCloud);
        end
    end
    %----------------------------------------------------------------------
    % Step 3: Define methods used for setting up the algorithm.
    methods
        function isValid = checkLabelDefinition(algObj, labelDef)
            % Only Voxel ROI label definitions are valid for the Lidar
            %  semantic segmentation algorithm.
            isValid = labelDef.Type == lidarLabelType.Voxel;
            if isValid
                algObj.AllCategories{end+1} = labelDef.Name;
            end
        end
        function isReady = checkSetup(algObj)
            % Is there one selected ROI Label definition to automate.
            isReady = ~isempty(algObj.SelectedLabelDefinitions);
        end
    end
    %----------------------------------------------------------------------
    % Step 4: Specify algorithm execution. This controls what happens when
    %         the user presses RUN. Algorithm execution proceeds by first
    %         executing initialize on the first frame, followed by run on
    %         every frame, and terminate on the last frame.
    methods
        function initialize(algObj,~)
            % Load the pretrained SqueezeSegV2 semantic segmentation network.
            outputFolder = fullfile(tempdir, 'Pandaset');
            pretrainedSqueezeSeg = load(fullfile(outputFolder,'trainedSqueezeSegV2PandasetNet.mat'));
            % Store the network in the 'PretrainedNetwork' property of this object.
            algObj.PretrainedNetwork = pretrainedSqueezeSeg.net;
        end
        function autoLabels = run(algObj, pointCloud)
            % Setup categorical matrix with categories including 
            % 'Vegetation', 'Ground', 'Road', 'RoadMarkings', 'SideWalk', 
            % 'Car', 'Truck', 'OtherVehicle', 'Pedestrian', 'RoadBarriers', 
            % and 'Signs'.
            autoLabels = categorical(zeros(size(pointCloud.Location,1), size(pointCloud.Location,2)), ...
                0:12,algObj.AllCategories);
            % Convert the input point cloud to five channel image.
            I = helperPointCloudToImage(pointCloud);
            % Predict the segmentation result.
            predictedResult = semanticseg(I, algObj.PretrainedNetwork);
            autoLabels(:) = predictedResult;
        end
    end
end
function image = helperPointCloudToImage(ptcloud)
% helperPointCloudToImage converts the point cloud to 5 channel image
image = ptcloud.Location;
image(:,4) = ptcloud.Intensity;
rangeData = iComputeRangeData(image(:,1),image(:,2),image(:,3));
image(:,5) = rangeData;
index = isnan(image);
image(index) = 0;
end
function cmap = helperPandasetColorMap
    cmap = [[30 30 30];  % Unlabeled
        [0 255 0];       % Vegetation
        [255 150 255];   % Ground
        [237 117 32];    % Road
        [255 0 0];       % Road Markings
        [90 30 150];     % Sidewalk
        [255 255 30];    % Car
        [245 150 100];   % Truck
        [150 60 30];     % Other Vehicle
        [255 255 0];     % Pedestrian
        [0 200 255];     % Road Barriers
        [170 100 150];   % Signs
        [255 0 255]];    % Building
    cmap = cmap./255;
end
function rangeData = iComputeRangeData(xChannel,yChannel,zChannel)
rangeData = sqrt(xChannel.*xChannel+yChannel.*yChannel+zChannel.*zChannel);
end