Camera and LiDAR calibration can indeed be challenging due to the differences in the nature of the sensors and their data. Based on your description, here are some steps and considerations that might help you troubleshoot the issues:
Synchronization of Data:
- Ensure that the camera and LiDAR data are synchronized. This means that for every frame captured by the camera, there should be a corresponding point cloud captured by the LiDAR at the same time. If the frame rates are different, you may need to interpolate or match timestamps to align the data.
Data Capture:
- When capturing data, make sure the checkerboard pattern is well-illuminated and visible in both the camera and LiDAR data.
- The checkerboard should be placed at various orientations and distances from the sensors to capture a diverse set of calibration data.
Preprocessing of LiDAR Data:
- When converting from pcap to pcd, ensure that you are extracting the relevant frames. You may need to write a script to parse the pcap file and extract the frames that match the camera timestamps.
- If you are having trouble with the ROI in the LiDAR data, you might need to preprocess the point cloud to remove outliers and noise that could be affecting the detection of the checkerboard.
ROI Adjustment:
- If the ROI tool is not working as expected, you may need to manually inspect the point clouds and identify the coordinates of the checkerboard.
- You can then crop the point cloud around these coordinates to focus the calibration process on the checkerboard area.
Calibration Process:
- Please refer to the following documentation for learning more about lidar and camera calibration in MATLAB https://www.mathworks.com/help/lidar/ug/lidar-and-camera-calibration.html
Capture Data at Specific Angles:
- If you are using a Velodyne VLP-16 LiDAR, capturing data at specific angles might require you to filter the point cloud data based on the angle information. The VLP-16 provides azimuth angle information for each point, which you can use to select points within a certain angular range.
I hope this helps
