How can I detect the dark black socket from the dark or very reflective bacground?

Question

L 2022-8-2

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1772715-how-can-i-detect-the-dark-black-socket-from-the-dark-or-very-reflective-bacground

评论： L 2022-8-10

I need to detect the CCS2 connector of electric cars. It is very dark, with dark and poor textures.

I have some results with intensity transformations using imadjust and gamma correction when I am deparating the connector from the background. Later, I am using morphology.

Still, when I place the camera little far from the connector, 40cm-50cm the image spoils, and the detection fails, specially at daylight. The detection from 20-25cm is 100%.

I am using ZED2i depth camera and diffuse light sources, but the depth image is very poor due to the socket texutes, and now first I need to detect the connector position from the color image.

I would like to use image processing techniques, not deep learning if it is possible (I was thinking about RCNN network, or mask-RCNN, but I can not find the detail description how can I make the training with my own data/images).

I have attached the images too.

Please, any suggestion to improve the detection from 50cm distance (the greater distance would more better).

Naturally, a new camera could help, but the goal is to generate the depth image too, not only the color image.

Thanks!

2 个评论
显示无隐藏无

Mathias Smeets 2022-8-2

Is it an option to crop the image in advance?

L 2022-8-2

Hello Mathias,

No, there is no option to crop the image.

The goal is to record the image from about 50cm and detect/extract the connector.

Also, the detection should work with various illumination conditions (daylight, artificial light, etc.)

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Image Analyst 2022-8-3

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1772715-how-can-i-detect-the-dark-black-socket-from-the-dark-or-very-reflective-bacground#answer_1020790

normxcorr2_demo.m

imadjust won't help. Increasing global brightness or contrast will not help with segmentation. Just think about it and you should know why.

Local contrast enhancement like with adapthisteq or imlocalbrighten might help.

If the connectors are of the same type, you might be able to use normalized cross correlation to find them. Demo attached. But it will depend on the template having the same size as the connector, which is unlikely from images collected from the field.

Otherwise if you had hundreds of photos and wanted to label them all with ground truth of known connector pixels, then you could use segnet.

14 个评论
显示 12更早的评论隐藏 12更早的评论

L 2022-8-3

Hello Image Analyst,

Thank you for the response. Your advice is very useful, as always.

I think you saw the attached images.

imadjust helped at the beginning to make more visible the socket, naturally it is not enough for reliable detection in every case, since the illumination conditions, position, size can change, also the demand is the 50cm distance between the camera and the socket.

I have tried the adapthisteq and imlocalbrighten functions. The problem remained. The detection from 50cm sometime fails. The detection from 20-25cm is good (since the socket is closer and there are less components around it).

The correlation is a good operation, however it cannot help in this situation. The connectors are always the same, but the connector size can vary in the image and it is impossible to know the template size. It is difficult to set the car and the camera at the same position in every case.

The goal is to use common image processing operations for the detection and extraction.

Maybe some very special industrial camera or sensor (profile sensor, etc.) can help, I don't know? I have ZED2i and ZED cameras (and a commercial photo camera and a phone camera).

I have many photos, it is not a problem to capture more.

I never used CNNs and segnet. Is there a tutorial (Matlab, Python) with details how to train the own data/images with segnet? The segnet would provide the connector area (like the mask-rcnn, or the rcnn), or it would segment and label the image regions including the connector too (then further processing is needed)? Also, what do you think about 6DOF Pose Estimation with DOPE? Did you know something about that? It should detect and label the bounding box around the object (I saw a video about this, I am not familiar with it).

Do you have an idea how to limit the capturing conditions from 50cm to ensure a better image (because the connector is very dark, without visible textures in the RGB image)?

Thank you in advance.

L 2022-8-4

编辑：L 2022-8-4

Thank you. I have accepted your answer, as always.

I see, but could I replace the imadjust with other function?

The first problem is when the car and the socket are the same black color. The second problem is when the background is very reflective and the socket is not visible enough. The third problem is when the socket is in shadow and it is not visible.

I have tried the thresholding without the imadjust. Mainly, the socket melts into the background, specially when the background is black. Futher, there are too many components in the image after the thresholding and it is hard to extract the socket, specially from 50cm. From 20cm the thresholded image is better, but still it is a problem to extract the socket.

Generally, is it possible to detect the socket realiably with classiccal image processing techniques (under certain conditions of image capturing)? Now, I am thinking about that too.

The segnet results with direct detection of the socket, or it segments the image in regions?

Yor segnet app is trained in Matlab, or you have used other platform?

Also, the problems of black background and socket invisibility can affect the segnet (or any other CNN)? I am not familiar with CNNs, but I think the low image quality and the darkness could be a problem to CNNs too. Generally, the illumination in this case is a problem due to the reflection from the car.

Is it possible to use some texture filter? The socket is mainlye textureless due to the darkness, and that is the reason I didn't try. I have tried the edge detection and the morphological gradient, but still the socket is not elevated suitably from the background.

L 2022-8-4

编辑：L 2022-8-4

example.png

The best result would be the binary mask of the connector, with its contacts. Hence, the accuracy should be high.

The other possibility could be the detection of the centroid too. Maybe after the centroid detection, in the second step the accuracy can be improved? Maybe this could be a good solution?

The goal is to detect reliably the connector for the scientific project, and the possible application could be the plugging of the cable (with robot, with automated arm, etc.), however, this is only the beginning of the project. The topic of this project is interesting from the image processing point of view, since it is very difficult the detection of the dark socket.

I am aware that is difficult the detection (specially with black environment), I have mounted two LED lights near the camera. It helped, however the detection fails in some examples. The light can cause reflections too, and they can blind the camera.

I have attached you a good result. I have cropped the good results.

L 2022-8-10

sample3.png

Hello,

This is an update to previous post. I did the adaptive thresholding. The image is attached. The global thresholding is not useful since the enviroment is white.

Now, I am thinking about simple operations to extract the mask of the socket area.

Maybe the combination of the edge image with this can produce a reliable solution (the goal is with any enviroment)?

L 2022-8-10

This is the example with a silver (alu-foil) background. The socket is illuminated with the same hand lamp as before, however now the sunlight is affecting the scene from the left side. The results of the adaptive thresholding and the Sobel edge detector are shown in the sample images.

The possible solution is to illuminate the socket in dark. However, is it possible to separate and extract the socket from this kind of input image (to get the mask of the socket)?

Possibly this is one of the worst cases.

请先登录，再进行评论。

Answer 2

Mathias Smeets 2022-8-3

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/1772715-how-can-i-detect-the-dark-black-socket-from-the-dark-or-very-reflective-bacground#answer_1020285

I don't know a specific solution.

However, here is a link that might help. I had a similar (easier) problem and I tried out a lot of different things on this link under 'segmentation techniques'.

Another thing you might try is using the functions imadjust (for enhancing contrast) and imlocalbrighten (for brightening) before you do the actual processing.

4 个评论
显示 2更早的评论隐藏 2更早的评论

L 2022-8-3

编辑：L 2022-8-3

I am sending you two good results when the detection is correct, and I am sending you the image after imadjust and few operations. In this example the detection fails. Generally, if there are several white areas with similar dimensions as the connector itself, that is the problem.

L 2022-8-3

I have attached now the images.

The file names explaine the images.

请先登录，再进行评论。

How can I detect the dark black socket from the dark or very reflective bacground?

2 个评论
显示无隐藏无

采纳的回答

14 个评论
显示 12更早的评论隐藏 12更早的评论

更多回答（1 个）

4 个评论
显示 2更早的评论隐藏 2更早的评论

另请参阅

类别

标签

Community Treasure Hunt

How can I detect the dark black socket from the dark or very reflective bacground?

2 个评论 显示 无隐藏 无

采纳的回答

14 个评论 显示 12更早的评论隐藏 12更早的评论

更多回答（1 个）

4 个评论 显示 2更早的评论隐藏 2更早的评论

另请参阅

类别

标签

Community Treasure Hunt

2 个评论
显示无隐藏无

14 个评论
显示 12更早的评论隐藏 12更早的评论

4 个评论
显示 2更早的评论隐藏 2更早的评论