I can't synchronize lidar and camera together for fusion

Question

Ali sameh 2023-12-16

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2061317-i-can-t-synchronize-lidar-and-camera-together-for-fusion

回答： Sanjana 2025-1-21

This my code. When I move camera detects that am moving then 5 seconds later I start moving in the lidar.

% lidarFig = subplot(1, 2, 2); % Create subplot for lidar data
countinueStreaming = true;
while countinueStreaming
    % Record the current timestamp before capturing the image
    
    % Capture the image from the Jetson camera hardware.
    I = snapshot(cam);
    
    % Record the timestamp after capturing the image
    
    % Call to detect method
    [bboxes, scores, labels] = detect(detector, I);
    personIndices = find(labels == 'person');
    personBboxes = bboxes(personIndices, :);
    
    % Convert categorical labels to cell array of character vectors
    labels = cellstr(labels);
    
    % Annotate detections in the camera image.
    img = insertObjectAnnotation(I, 'rectangle', bboxes, labels);
    
    % Display camera image
    imshow(img);
    hold on;
    
    % Read data from Velodyne Lidar
    pc = read(lidar);
    % Remove ground plane
    groundPtsIndex = segmentGroundFromLidarData(pc, 'ElevationAngleDelta', 0, ...
        'InitialElevationAngle', 0);
    nonGroundPts = select(pc, ~groundPtsIndex);
    imPts = projectLidarPointsOnImage(pc, intrinsics, LidarToCam);
    scatter(imPts(:, 1), imPts(:, 2), 10, 'g', 'filled');
    % Display lidar data
    % pcshow(nonGroundPts.Location, 'Parent', lidarFig);
    
    if ~isempty(personIndices)
        for i = 1:numel(personIndices)
            index = personIndices(i);
            personBbox = bboxes(index, :);
            
            % Process lidar data for each detected person
            [lidarBbox, ~, boxUsed] = bboxCameraToLidar(personBbox, nonGroundPts, intrinsics, ...
                camToLidar, 'ClusterThreshold', 2, 'MaxDetectionRange', [1, 10]);
            [distance, nearestRect, idx] = helperComputeDistance(personBbox, nonGroundPts, lidarBbox, ...
                intrinsics, LidarToCam);
            
            % Print distance information
            fprintf('DistOfPerson %d: %.2f meters\n', i, distance);
            % Update image with bounding boxes
            % im = updateImage(display, im, nearestRect, distance);
            % updateLidarBbox(display, lidarBbox)
        end
    else
        % Process lidar data for other objects
        [lidarBbox, ~, boxUsed] = bboxCameraToLidar(bboxes, nonGroundPts, intrinsics, ...
            camToLidar, 'ClusterThreshold', 2, 'MaxDetectionRange', [1, 10]);
        [distance, nearestRect, idx] = helperComputeDistance(bboxes, nonGroundPts, lidarBbox, ...
            intrinsics, LidarToCam);
        
        % Print distance information
        fprintf('DistOfObj: %.2f meters\n', distance);
        % Update image with bounding boxes
        % im = updateImage(display, im, nearestRect, distance);
        % updateLidarBbox(display, lidarBbox)
    end    
    hold off;
end

2 个评论
显示无隐藏无

Image Analyst 2023-12-16

I don't understand the statement. Can you type it into Google translator

https://translate.google.com/?sl=auto&tl=en&op=translate

in your native language and then paste the translation here.

I don't even see a question. There is no sentence that ends with a question mark.

Ali sameh 2023-12-17

Look my camera and lidar isn't synchronised. There is a lidar delay of 5 seconds from the camera my question is how can I synchronise lidar and camera together?

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Sanjana 2025-1-21

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2061317-i-can-t-synchronize-lidar-and-camera-together-for-fusion#answer_1557825

在 MATLAB Online 中打开

Hi,

To Synchronize Camera Sensor Data and Lidar Sensor Data, you can use the synchronize function of scenariobuilder sensorData objects.

Kindly, refer to the following code to synchronize LidarData with CameraData,

% Step - 1: Create a scenariobuilder.LidarData object
% timestamps - timestamps of lidarData
% pointClouds - Lidar point cloud data
lidarData = scenariobuilder.LidarData(timestamps,pointClouds)
% Step - 2: Create a scenariobuilder.CameraData object
% timestamps - timestamps of cameraData
% frames - frames of images of cameraData
cameraData = scenariobuilder.CameraData(frames,timestamps);
% Step - 3: Synchronize LidarData with CameraData as reference using the synchronize
% function of LidarData
synchronizedLidarData = synchronize(lidarData,cameraData);