Why imregtform registration fails on similar images

Question

Lorenzo 2024-9-4

1
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2150134-why-imregtform-registration-fails-on-similar-images

编辑： Matt J 2024-9-7

testCode.zip

在 MATLAB Online 中打开

Dear all, I am currently working on a project requiring image registration. I have to very similar images

However, a slight mismatch exists between them requiring an affine trasformation to get them perfectly matched. Unfortunately, it does not work and I cannot find a way to make it work. This is the result I get executing the code below. Both images in tif format and the code are in the zip file ready to run since I cannot upload tif here.

ref = imread('Reference.tif');
im = imread('Image.tif');
[optimizer, metric] = imregconfig('monomodal');
optimizer.GradientMagnitudeTolerance = 1e-8;
optimizer.MaximumIterations = 3000;
optimizer.MinimumStepLength = 1e-7;
transform = imregtform(im, ref, 'affine', optimizer, metric,'DisplayOptimization',0);
reg = imwarp(im,transform,'cubic','OutputView',imref2d(size(im)),'FillValues',0);
subplot(1,3,1)
imshow(ref,[]);
subplot(1,3,2)
imshow(reg,[]);

Can your provide any suggestion? What can I do to improve outcome of registration? I have several images similar to these to register to the reference one and I would like to understand what goes wrong here.

Thank you!

L.

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Matt J 2024-9-4

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2150134-why-imregtform-registration-fails-on-similar-images#answer_1510879

编辑：Matt J 2024-9-4

在 MATLAB Online 中打开

Images.mat

Perhaps as follows,

load Images

g=@(z) entropyfilt(z); f=@(z) z.*(z>=1.5);

im=f(g(im));

ref=f(g(ref));

ov=imref2d(size(im));

[optimizer, metric] = imregconfig('multimodal');

transform = imregtform(im,ov,ref,ov, 'translation', optimizer, metric,'DisplayOptimization',0);

[optimizer, metric] = imregconfig('monomodal');

transform = imregtform(im,ov,ref,ov, 'affine', optimizer, metric,...

'DisplayOptimization',0,Initial=transform);

load Images

reg = imwarp(im,transform,'cubic','FillValues',1,'OutputView',ov);

subplot(1,3,1)

imshow(ref,[]); title 'Reference'

subplot(1,3,2)

imshow(reg,[]); title 'Registered'

subplot(1,3,3)

imshow(imfuse(reg,ref,'diff')); title 'Fuse'

h=gcf; h.Position(3:4)=[903 188];

4 个评论
显示 2更早的评论隐藏 2更早的评论

Lorenzo 2024-9-5

在 MATLAB Online 中打开

data.mat

Hi first of all, thank you for your time.

I found you approach does work very well for that image and for that specific crop. The images are a crop for a larger one. But it is not reproducible for other images or crop size, even if they are still very similar. I now attach a more complete dataset with additional images. They are again very similar in intensity and with a minimum mismatch but registration still fails with some of them.

load data.mat

reference = data(:,:,1);

images = data(:,:,2:end);

g=@(z)entropyfilt(z); f=@(z) z.*(z>=1.5);

referencefilt = f(g(reference));

for ii = 2:size(images,3)

image = images(:,:,ii);

imagefilt = f(g(image));

ov = imref2d(size(image));

[optimizer, metric] = imregconfig('multimodal');

transform = imregtform(imagefilt,ov,referencefilt,ov, 'translation', optimizer, metric,'DisplayOptimization',0);

[optimizer, metric] = imregconfig('monomodal');

transform = imregtform(imagefilt,ov,referencefilt,ov, 'affine', optimizer, metric,'DisplayOptimization',0,Initial=transform);

registered = imwarp(image,transform,'cubic','FillValues',1,'OutputView',ov);

subplot(size(images,3)-1,3,3*ii-5); imshow(reference,[0.9,1.1]); title 'Reference';

subplot(size(images,3)-1,3,3*ii-4); imshow(image,[0.9,1.1]); title 'Image';

subplot(size(images,3)-1,3,3*ii-3); imshow(registered,[0.9,1.1]); title 'Registered';

end

The first image is properly registered why the others not at all. Can you provide an insight about your approach? Are you just setting the background to zero in such a way that the alghoritm works only on the object? Any idea why it fails on the other images and it cannot generalized?

Matt J 2024-9-5

编辑：Matt J 2024-9-5

在 MATLAB Online 中打开

data.mat

This seems to work. You need to plug the holes that form at the edges after the initial registration.

load data.mat

reference = data(:,:,1);

images = data(:,:,2:end);

N=size(images,3);

g=@(z)entropyfilt(z); f=@(z) z.*(z>=1.5);

referencefilt = f(g(reference));

for ii = 1:N

image = images(:,:,ii);

imagefilt = f(g(image));

%Similarity-only registration and warping

ov = imref2d(size(image));

[optimizer, metric] = imregconfig('multimodal');

tform0 = imregtform(imagefilt,ov,referencefilt,ov, 'similarity', optimizer, metric,'DisplayOptimization',0);

%Form an intermediate image by warping. Fill black areas with ones.

tmp = imwarp(image,tform0,'cubic','FillValues',0,'OutputView',ov);

tmp(tmp<0.5)=1;

%Refine with a full affine registration

[optimizer, metric] = imregconfig('monomodal');

tform1 = imregtform(f(g(tmp)),ov,referencefilt,ov, 'affine', optimizer, metric,'DisplayOptimization',0);

transform=affinetform2d(tform1.A*tform0.A);

registered = imwarp(image,transform,'cubic','FillValues',1,'OutputView',ov);

registered(registered<0.5)=1;

j=sub2ind([4,N],1,ii);

subplot(N,4,j); imshow(reference,[0.9,1.1]); title 'Reference';

subplot(N,4,j+1); imshow(registered,[0.9,1.1]); title 'Registered';

subplot(N,4,j+2); imshow(imfuse(image,reference,'diff'),[100,200]); title 'Diff Before';

subplot(N,4,j+3); imshow(imfuse(registered,reference,'diff'),[100,200]); title 'Diff After';

end

Lorenzo 2024-9-6

在 MATLAB Online 中打开

I am sorry to keep bothering, we are getting closer but unfortunately it is still not reliable. Adding additonal images to the dataset there are a few of them where registration still fails. I also noticed that there is slight mismatch between a few of them which is almost impossible to detect from the previous plots. However, I guess I can change the optimizer parameter to improve registration a bit.

Sorry but I have several groups of these images where that object is slightly moved and each group has its own reference. So sending all of them is not possible and not useful for this question because, all the images look the same. So my hope is to get it working on single group and apply the code to all. But I don't understand why is the registration so unreliable with such similar images.

This time the .mat file is too large for uploading so I send it as a link https://drive.google.com/file/d/1XvPnLBaoGK7I6lEuI8H6MXI51J6DMD7N/view?usp=sharing

Thank you for all your time.

L.

load data.mat
reference = data(:,:,1);
images = data(:,:,2:end);
N=size(images,3);
g=@(z)entropyfilt(z); f=@(z) z.*(z>=1.5);
referencefilt = f(g(reference));
for ii = 1:N
    disp(['Registering image ', num2str(ii),' of ', num2str(N)]);
    image = images(:,:,ii);
    imagefilt = f(g(image));
    %Similarity-only registration and warping
    ov = imref2d(size(image));
    [optimizer, metric] = imregconfig('multimodal');
    tform0 = imregtform(imagefilt,ov,referencefilt,ov, 'similarity', optimizer, metric,'DisplayOptimization',0);
    %Form an intermediate image by warping. Fill black areas with ones.
    tmp = imwarp(image,tform0,'cubic','FillValues',0,'OutputView',ov);
    tmp(tmp<0.5)=1;
    %Refine with a full affine registration
    [optimizer, metric] = imregconfig('monomodal');
    tform1 = imregtform(f(g(tmp)),ov,referencefilt,ov, 'affine', optimizer, metric,'DisplayOptimization',0);
    transform=affinetform2d(tform1.A*tform0.A);
    registered = imwarp(image,transform,'cubic','FillValues',1,'OutputView',ov);
    registered(registered<0.5)=1;
    registeredVolume(:,:,ii) = registered;
end
for ii = 1:N
    imshow(registeredVolume(:,:,ii),[0.88,1.1]);
    title(['Image ',num2str(ii)]);
    pause(0.9);
end
imshow(std(registeredVolume,1,3),[0,0.015]);
title(['Standard deviation of all images ',num2str(ii)]);

Matt J 2024-9-6

编辑：Matt J 2024-9-7

在 MATLAB Online 中打开

But I don't understand why is the registration so unreliable with such similar images.

From the point of view of most registration algorithms, the images are only "similar" if there is strong overlap between where image features are positioned in the moving and reference images. That is less and less the case as the image frames progress. The objects in the scene become significantly displaced from the reference image and overlap diminishes greatly. On top of that, you have edge truncation effects, and also differences and unevenness in illumination across frames.

Below, I have yet another variant which is working for me on the 18-frame movie that you sent, but it relies on the assumption that there will always be only incremental change in the image position from one frame to the next. If you continue to have problems, you could try backing off from an affine transform in the refinement step to a similarity transform (I wonder if you really need more than that). With fewer degrees of freedom in the transformation model, the registration will be more stable.

load data.mat
reference = data(:,:,1);
images = data(:,:,2:end);
[H,W,N]=size(images);
imagesfilt=preproc(images);
referencefilt = preproc(reference);
registeredVolume=nan(H,W,N);
tform0=transltform2d;
for i = 1:N
    disp(['Registering image ', num2str(i),' of ', num2str(N)]);
    image=images(:,:,i);
    imagefilt = imagesfilt(:,:,i);
    %Similarity-only registration and warping
    ov = imref2d(size(image));
    [optimizer, metric] = imregconfig('multimodal');
    tform0 = imregtform(imagefilt,ov,referencefilt,ov, 'translation', ...
                        optimizer, metric,'DisplayOptimization',0,Initial=tform0);
    %Form an intermediate image by warping. Fill black areas with ones.
    tmp = imwarp(image,tform0,'cubic','FillValues',0,'OutputView',ov);
    tmp(tmp<0.5)=1;
    registeredVolume(:,:,i) = tmp;
    %Refine with a full affine registration
    [optimizer, metric] = imregconfig('multimodal');
    tform1 = imregtform(preproc(tmp),ov,referencefilt,ov,...
                        'affine', optimizer, metric,'DisplayOptimization',0);
    transform=affinetform2d(tform1.A*tform0.A);
    registered = imwarp(image,transform,'cubic','FillValues',1,'OutputView',ov);
    registered(registered<0.5)=1;
    registeredVolume(:,:,i) = registered;
end
close all
for i = [1,3:3:N]
    registered=registeredVolume(:,:,i);
    image=images(:,:,i);
    figure(i);
     subplot(1,3,1); imshow(reference,[0.9,1.1]); title 'Target';
     subplot(1,3,2); imshow(registered,[0.9,1.1]); title 'Registered';
     subplot(1,3,3); imshow(imfuse(registered,reference,'diff'),[]); title 'Diff After';
end
function Inew=preproc(I)
   Inew = 1000*(1-I);
   Inew=Inew.*(abs(Inew)<=50);
end

请先登录，再进行评论。

Why imregtform registration fails on similar images

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

4 个评论
显示 2更早的评论隐藏 2更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

Why imregtform registration fails on similar images

0 个评论 显示 -2更早的评论隐藏 -2更早的评论

回答（1 个）

4 个评论 显示 2更早的评论隐藏 2更早的评论

另请参阅

类别

标签

产品

版本

Community Treasure Hunt

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

4 个评论
显示 2更早的评论隐藏 2更早的评论