How to indentify a broken pill package?

Question

Jannis Holtkoetter 2020-10-9

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/608773-how-to-indentify-a-broken-pill-package

回答： Jannis Holtkoetter 2020-10-22

Hello, I am writing a program, which should read in an image of a pill package and count how many pills are left over in the package. The code uses a combination of image processing functions including the canny transform into a gradient image and the hough transform for circles. Here are some images to understand what I am talking about (The numbers in the green circles show the order of the circles' metric values, from high to low)

Now I am trying to write some code, which can identify the difference between pills, which are already used and those, which are still left over in the package. Unfortunately the metric value does not give a good indication if the pill pocket is broken or not. I also calculated a ratio where I divided the number of gradient pixels by the total area of the circle. Unfortunately this value is also very unreliable for this problem.

Do you have any other ideas how to identify which of these pill pockets are broken? If it helps, the full pill package is always analyzed first and then, some informations are saved and can be loaded into the code again as a possible comparision. However, the images are never 100% the same, since they are all done by hand within an app.

I'm thankful for any tips, ideas and help!

0 个评论
显示 -2更早的评论隐藏 -2更早的评论

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Image Analyst 2020-10-9

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/608773-how-to-indentify-a-broken-pill-package#answer_509101

I'd probably start with just having a mask/template with ROIs over each pill pocket and take the standard deviation of the image intensities. If the standard deviation is high, then it's been used. If it's low, then the intensity if pretty uniform and the pill has not been pushed out.

Or you could try a deep learning app.

17 个评论
显示 15更早的评论隐藏 15更早的评论

Image Analyst 2020-10-9

编辑：Image Analyst 2020-10-9

在 MATLAB Online 中打开

labeling_explanation.m

I would have thought the SD's would be more different than that. But the broken ones are above 0.2 while the others are less. You just don't understand how the masks are ordered. It goes top down then left to right. In other words, whichever one is leftmost is #1, regardless if it's in the top row or not. For example we know that blobs 5, 6, and 7 are in the right column, but I see that 5 and 6 have 0.2 so that means they're in row 1 and 4 and are very slightly positioned to the left of the blob in row 2, column 2.

I suggest you add the region number to the overlay to see how they're ordered:

for k = 1 : numPills
    if stdDevs(k) > 10
        % Plot red X over it.
        plot(xy(k, 1), xy(k, 2), 'rx', 'MarkerSize', 30, 'LineWidth', 2);
    else
        % Plot green spot over it.
        plot(xy(k, 1), xy(k, 2), 'g.', 'MarkerSize', 30, 'LineWidth', 2);
    end
    txt = sprintf('  %d', k);
    text(xy(k, 1), xy(k, 2), txt, 'Color', 'b', 'FontSize', 20, 'FontWeight', 'bold');
end

If you want to you could sort them based on rows and columns. Use kmeans() to categorize the x centroids into 2 classes, and the y into 4 classes. Then use sort to resort them if you want. However that really had no bearing on the count of broken or unbroken.

To explain labeling, run this code:

% Illustrates how regions labels (i.e. numbers) are assigned to blobs randomly placed in an image.
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 22;
%-------------------------------------------------------------------------------------------------------------------------------------
% First create image with a bunch of regions in it.
% Define the number of regions
numRegions = 20;
% Define a max region size for the circles.
maxRadius = 30;
minRadius = 6;
% Define the image size
rows = 600;
columns = 800;
% Create a binary image
binaryImage = false(rows, columns);
% Get a list of all coordinates in the image.
% First create the image.
imageSizeX = 640;
imageSizeY = 480;
[columnsInImage, rowsInImage] = meshgrid(1:columns, 1:rows);
% Start adding circles of random size.
for k = 1 : numRegions
	% Create a logical image of a circle with specified
	% diameter, center, and image size.
	% Next create the circle in the image.
	radius = minRadius + (maxRadius - minRadius) * rand(1);
	centerX = maxRadius + (columns - 2 * maxRadius) * rand(1);
	centerY = maxRadius + (rows - 2 * maxRadius) * rand(1);
	fprintf('Pasting circle #%d of radius %.1f centered at (%.1f, %.1f)\n', k, radius, centerX, centerY);
	circlePixels = (rowsInImage - centerY).^2 ...
		+ (columnsInImage - centerX).^2 <= radius.^2;
	% circlePixels is a 2D "logical" array.
	% Paste this circle onto binaryImage
	binaryImage = binaryImage | circlePixels;
% 	imshow(binaryImage);
% 	drawnow;
end
%-------------------------------------------------------------------------------------------------------------------------------------
% Next let's do connected components labeling of the regions.
[labeledImage, numRegions] = bwlabel(binaryImage); % bwlabel() is where it actually does the connected components labeling.
% Let's assign each blob a different color to visually show the user the distinct blobs,
% just in case some overlapped which would cause them to merge into one single blob instead of two.
coloredLabels = label2rgb (labeledImage, 'hsv', 'k', 'shuffle'); % pseudo random color labels
% coloredLabels is an RGB image.  We could have applied a colormap instead (but only with R2014b and later)
imshow(coloredLabels);
axis('on', 'image');
caption = sprintf('Binary image of %d distinct, separate regions', numRegions);
title(caption, 'FontSize', 15);
%-------------------------------------------------------------------------------------------------------------------------------------
% Now find out their pixel locations so we can put up a line there along the left edge, which is how it determines the label number.
props = regionprops(binaryImage, 'PixelList', 'Centroid');
for k = 1 : numRegions
	thesePixels = props(k).PixelList;
	[leftColumn(k), index] = min(thesePixels(:, 1)); % Min x value.
	% Get the y value for the left edge of the blob.
	yt = thesePixels(index, 2);
	% Put up a line there
	xline(leftColumn(k), 'Color', 'w');
	% Put a numerical label at the centroid.
	xt = leftColumn(k); %props(k).Centroid(1);
	txt = sprintf(' %d', k);
	text(xt, yt, txt, 'Color', 'k', 'FontWeight', 'bold', 'FontSize', 12, 'VerticalAlignment', 'middle');
end
% Show all the left columns in the command window so you'll see they are sorted from left to right.
leftColumn
% Maximize the window.
g = gcf;
g.WindowState = 'maximized';

Now scroll over to the right and look at the special case where blobs 17 and 18 both started at column 683. Because the blue blob (17) is higher than the pink blob (18), it was assigned the number 17 and the lower pink blob was assigned number 18. But in general you can see that blobs are numbered from left to right. And your mask is not a perfect rectangular array so it's quite possible some blobs have numbers different than what you might think at first.

Jannis Holtkoetter 2020-10-10

Thanks again!

I got the mask by using the hough transform for circles and then using different information about the pills (all same radius, rough X-axis-location, etc. ) in order to filter out all possible circles which were detected in the Hough Transform but are not pills. This works for the main part of my images, however I can't share any code, since it is quite long and there are different bits of code involved in the complete code.

However, I tried the calculation of the mean intensity, but it seems to give me unclear results for some images. I mean you can clearly see, that the mean intensity is generally higher in the broken pills, however the values are very close to each other:

After doing several tests, the limit which differentiates a non broken pill from a broken pill seems to be somewhere between 0.13 and 0.14. However as you can see in this example, sometimes there can be a little dent or some light reflection which bumps up the mean intensity.

Do you have any idea how to improve the differentiation between the values, so that it is a little bit more clear which ones are the real broken ones?

Image Analyst 2020-10-12

Yes, I'd just start with the cropped bounding box image. And of course you'll need the cropped circular mask image also so you can get the EDT of that so you know what radius the pixel's intensity is measured at.

Jannis Holtkoetter 2020-10-12

在 MATLAB Online 中打开

I applied your recommendation of using the crop and feeding these images into a loop with the averageRadialProfile function. I have to say, that I used the grayscale image based on the local standard deviation filter (values between 0 and 1). Is that a problem and should I have used just a grayscale version of the original jpg image (values between 0 and 255? Does it change anything?

The caluclation is being done with the code, which you provided in the file "average_radial_profile.m", as shown in the following. I was looking into the documentation of the different functions, but had problems to fully understand what the averageRadialProfile actually describes...

edtImage = bwdist(binaryImage);
maxDistance = ceil(max(edtImage(:)));
% Allocate an array for the profile
profileSums = zeros(1, maxDistance);
profileCounts = zeros(1, maxDistance);
% Scan the original image getting gray level, and scan edtImage getting distance.
% Then add those values to the profile.
[rows, columns] = size(edtImage);
for column = 1 : columns
	for row = 1 : rows
		thisDistance = round(edtImage(row, column));
		if thisDistance <= 0
			continue;
		end
		profileSums(thisDistance) = profileSums(thisDistance) + double(grayImage(row, column));
		profileCounts(thisDistance) = profileCounts(thisDistance) + 1;
	end
end
% Divide the sums by the counts at each distance to get the average profile
averageRadialProfile = profileSums ./ profileCounts;

In the loop I put each averageRadialProfile array into one cell array which now contains the averageRadialProfile for each pill. If I plot it for the example above, it gives me this result (the plots with the red circle are broken pills). How would can I analyze these results in a quantified way, so that I can atleast draw some conclusion between the average radial profile and a broken pill?

Again, I would be very thankful for any help!

请先登录，再进行评论。

Answer 2

Jannis Holtkoetter 2020-10-22

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/608773-how-to-indentify-a-broken-pill-package#answer_521643

I just accepted the answer with regard to the fact, that I will be able to use the mean intensity of the local standard deviation image for each pill in combination with the mean intensity of a canny edge image for each pill to define if a pill can be marked as used or not. It is definetly not working perfect but it works for a lot of specific looking blisters.

I still do not really know, how to interpret the radial profile of each pills and therefore I have no use for it, however the provided code worked for me and I could compute each radial profile in my example. I'm still open for any explanation or suggestion for implementation.