Point cloud not accurate - Stereo Vision

Question

Luca 2023-9-17

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2022287-point-cloud-not-accurate-stereo-vision

编辑： Aishwarya 2023-10-6

I'm trying to do a distance estimation of an object with stereo vision.

I've done the calibration of the cameras using Matlab's Toolbox, and then I took a photo of a sheet where I printed a template in A3 format.

Then I tried to do a depth estimation using this code:

%% load
load params1309.mat
left=imread("406mm/left.png");
right=imread("406mm/right.png");
figure
imshowpair(left,right,"montage")
%% rectification
[leftRect,rightRect,reprojectionMatrix]=rectifyStereoImages(left,right,stereoParams1309);
figure
A=stereoAnaglyph(leftRect,rightRect);
imshow(A)
%% disparity map
disparityRange = [0 20*16];
disparityMap = disparityBM(leftRect,rightRect,'DisparityRange',disparityRange,'UniquenessThreshold',10,'BlockSize',9,'ContrastThreshold',0.5);
figure
imshow(disparityMap,disparityRange)
title('Disparity Map')
colormap parula
colorbar
filtered=medfilt2(disparityMap,[5 5]);
figure
imshow(filtered)
title('Disparity Map filtered')
colormap parula
colorbar
%% reconstruction
xyzPoints=reconstructScene(filtered,reprojectionMatrix);
%% xyzPoints=xyzPoints;
X=xyzPoints(:,:,1);
Y=xyzPoints(:,:,2);
Z=xyzPoints(:,:,3);
ptcloud=pointCloud(xyzPoints);
player=pcplayer([-500 500],[-500 500],[0 2000],'VerticalAxis','y');
view(player,ptcloud)
%% mask
mask=repmat(Z>0 & Z<500,[1,1]);
leftRect(~mask)=0;
figure
imshow(leftRect)
%% average
figure();imshow(xyzPoints(:,:,3)); colormap jet
distance=xyzPoints(:,:,3);
distance(distance>420)=[];
isfinite=mean(distance(isfinite(distance)))
average=mean(distance,'all',"omitnan")

Those are left and right photos:

And this is what the point cloud looks like:

The object was at 406mm from the left camera, but as you can see in the point cloud there were some correct points at 400mm circa and other wrong points far away from them (at like 1000mm), and I don't understand why. It looks like it couldn't understand that it was a 2D sheet.

That's why I tried to remove the points > 500mm with a mask before trying to calculate the average, and the result was this one:

Do you know what's wrong? Why does it split the points in two different regions at two different distances?

And if you don't know how to improve the point cloud, do you maybe know how can I get rid of all the points i don0t want in order to make the average more accurate (using only the points around 400mm)?

Thank you!

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Aishwarya 2023-10-6

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2022287-point-cloud-not-accurate-stereo-vision#answer_1327234

编辑：Aishwarya 2023-10-6

Hi Luca,

As per my understanding you are getting an unexpected result in the point cloud generated through stereo vision depth estimation. After reviewing the information provided, here are some suggestions to improve the results:

1. Scene with repetitive pattern:

The presence of uniform noise in the background of the point cloud could be due to difficulties in finding correspondences between the two images.
This may be caused by the repetitive pattern and colour of the checkerboard scene in the images.
To address this, I suggest testing the algorithm on images with more texture and variable depths in the scene. This can provide better features for the stereo vision algorithm to establish accurate correspondences.

2. Improvement in camera calibration:

Follow the workflow outlined in the document provided below to enhance calibration accuracy: https://www.mathworks.com/help/vision/ug/using-the-stereo-camera-calibrator-app.html
Ensure that the calibration data is captured under the same conditions as the rest of the experiment to maintain consistency.

3. Mismatched stereo pair: Verify that both cameras have the same intrinsic and extrinsic parameters to ensure accurate depth estimation.

4. Tuning disparity range and other parameters:

Adjust the disparity range and other relevant parameters to optimize depth estimation.
If the scene has limited texture, consider increasing the "BlockSize" parameter. A larger "BlockSize" can reduce noise in the disparity map, but it may come at a higher computational cost.
For detailed instructions on tuning the disparity range using the "disparityBM" function, refer to the "Choosing Range of Disparity" subsection in the document provided: https://www.mathworks.com/help/vision/ref/disparitybm.html

5. Try different disparity mapping method:

There are different disparity mapping methods like Block Matching (BM) and Semi-Global Matching (SGM).
Semi-global matching is more computationally expensive than block matching, but it can produce high-quality results. It is recommended to try both algorithms and compare their results for the specific application.
For instance, I have implemented the "disparityBM" algorithm using the parameters specified in your code and applied a "medfilt2" filter, as indicated in the code. Subsequently, I compared these results with those obtained using the "disparitySGM" method.