Recognizing text from image of a calculator

Question

Soumyadip 2013-3-24

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/68387-recognizing-text-from-image-of-a-calculator

编辑： sandeep bayi 2017-3-21

Hi,

I have trying to extract the numbers from a calculator screen image. I am using my laptop webcam to detect the image and simultaneously display the result after image processing. Can anybody please help how to proceed??

1 个评论
显示 -1更早的评论隐藏 -1更早的评论

sandeep bayi 2017-3-21

编辑：sandeep bayi 2017-3-21

iam too working on the like wise project and need help for extracting the numbers from led display of seven segment data. Can anyone please help me?

请先登录，再进行评论。

请先登录，再回答此问题。

Answer 1

Image Analyst 2013-3-24

0
链接

此回答的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/68387-recognizing-text-from-image-of-a-calculator#answer_79720

在 MATLAB Online 中打开

Very very easy. You don't even need to do sophisticated OCR. You merely need to sample the image at known locations for known numbers. Trivial for a 7 segment display. Just read the image at 7 locations. Let's say the locations are

Now a "1" would have 2 and 5 lit, and all the others unlit.

A "2" would have 1, 3, 4, 5, & 7 lit and the others unlit.

You can make up a 128 element look up table and use that to read off the number, given an input pattern. it's the same concept but a little more complicated if you have something with more resolution than a 7 segment display.

23 个评论
显示 21更早的评论隐藏 21更早的评论

Image Analyst 2013-3-26

编辑：Image Analyst 2013-3-26

在 MATLAB Online 中打开

First of all, improve your image by using more light. Next, make sure your calculator is in the same location every time. Then just use the same concepts I spelled out. There are certain locations that will be blue and certain locations that will be gray. These locations will be different for every number. I don't think you should have to check more than about 7-10 regions to uniquely identify every number. Is there something in the algorithm that is still confusing? Or do you understand the concept and algorithm, but you just don't know how to put it into MATLAB code? Here's a hint. To get the red, green, and blue value you can use mean2:

meanRed = mean2(rgbImage(row1:row2, col1:col2, 1));
meanGreen = mean2(rgbImage(row1:row2, col1:col2, 2));
meanBlue = mean2(rgbImage(row1:row2, col1:col2, 3));
tolerance = 10; % or whatever.
zoneIsBlue(zoneNumber) = meanBlue > (meanRed + tolerance) & meanBlue > (meanGreen + tolerance)

Of course the rows and columns will have to refer to different locations for each number because they are not all in the same location. You could pre-identfy those regions with ginput(), imrect(), or rbbox(), which is easiest if your calculator is in the same location, as I recommended. If the calculator moves around and you, unfortunately, have no way at all to prevent that, then you can identify the blue region and try to find out how many numbers are showing by looking at the width of the blue region, then get your zones knowing that. You can look in my File Exchange for several color segmentation methods.

Image Analyst 2013-3-26

在 MATLAB Online 中打开

More light and a longer exposure time. And save it as PNG, not jpg. And of course a better camera and lens. Here's some code to get you started:

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
imtool close all;  % Close all imtool figures if you have the Image Processing Toolbox.
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 20;
% Read in a standard MATLAB color demo image.
folder = 'C:\Users\Soumyadip\Documents\Temporary';
baseFileName = 'dpvms8.jpg';
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
if ~exist(fullFileName, 'file')
  % Didn't find it there.  Check the search path for it.
  fullFileName = baseFileName; % No path this time.
  if ~exist(fullFileName, 'file')
    % Still didn't find it.  Alert user.
    errorMessage = sprintf('Error: %s does not exist.', fullFileName);
    uiwait(warndlg(errorMessage));
    return;
  end
end
rgbImage = imread(fullFileName);
% Get the dimensions of the image.  numberOfColorBands should be = 3.
[rows columns numberOfColorBands] = size(rgbImage);
% Display the original color image.
subplot(2, 2, 1);
imshow(rgbImage);
axis on;
title('Original Color Image', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
% Crop the image.
rgbImage = imcrop(rgbImage, [275, 320, 250 80]);
% Display the image.
subplot(2, 2, 2);
imshow(rgbImage);
axis on;
title('Cropped Color Image', 'FontSize', fontSize);
% imwrite(rgbImage, 'calculator.png');
% Extract the individual red, green, and blue color channels.
redChannel = rgbImage(:, :, 1);
greenChannel = rgbImage(:, :, 2);
blueChannel = rgbImage(:, :, 3);
tolerance = 7; % or whatever.
bluePixels = blueChannel > (redChannel + tolerance) & blueChannel > (greenChannel + tolerance);
% Display the image.
subplot(2, 2, 3);
imshow(bluePixels);
axis on;
title('Blue Pixels', 'FontSize', fontSize);
% meanRed = mean2(redChannel(row1:row2, col1:col2));
% meanGreen = mean2(greenChannel(row1:row2, col1:col2));
% meanBlue = mean2(blueChannel(row1:row2, col1:col2));

Image Analyst 2013-3-31

编辑：Image Analyst 2013-3-31

在 MATLAB Online 中打开

Here, I did it using color classification plus some standard image processing techniques to clean the image of spurious blobs so that only the numbers remain. I looked at the 3D color gamut and noticed that in LAB color space, the numbers had negative b values (bluish) while the background had positive b values (greenish/yellowish). Although it does an excellent job on the one image you provided, you'd still be better off getting a better camera to get less noise and using more light, which will also give less noise.

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
imtool close all;  % Close all imtool figures if you have the Image Processing Toolbox.
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 20;
% Read in a standard MATLAB color demo image.
folder = 'C:\Users\soumyadip\Documents\Temporary';
baseFileName = 'calculator.png';
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
if ~exist(fullFileName, 'file')
  % Didn't find it there.  Check the search path for it.
  fullFileName = baseFileName; % No path this time.
  if ~exist(fullFileName, 'file')
    % Still didn't find it.  Alert user.
    errorMessage = sprintf('Error: %s does not exist.', fullFileName);
    uiwait(warndlg(errorMessage));
    return;
  end
end
rgbImage = imread(fullFileName);
% Get the dimensions of the image.  numberOfColorBands should be = 3.
[rows, columns, numberOfColorBands] = size(rgbImage);
% Display the original color image.
subplot(2, 2, 1);
imshow(rgbImage);
title('Original Color Image', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
% Convert to lab color space.
cform = makecform('srgb2lab');
lab = applycform(im2double(rgbImage),cform);
LImage = lab(:,:, 1);
aImage = lab(:,:, 2);
bImage = lab(:,:, 3);
% Display them
subplot(2,2,2);
imshow(LImage, []);
title('L Image', 'FontSize', fontSize);
subplot(2,2,3);
imshow(aImage, []);
title('A Image', 'FontSize', fontSize);
subplot(2,2,4);
imshow(bImage);
title('B Image', 'FontSize', fontSize);
figure; % Open a new figure.
subplot(2, 2, 1);
imshow(bImage, []);
title('B Image', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
% Binarize to get negative b (bluish pixels).
originalBinaryImage = bImage < 0;
subplot(2, 2, 2);
imshow(originalBinaryImage, []);
title('Binary Image', 'FontSize', fontSize);
% Clean up the image.
binaryImage = bwareaopen(originalBinaryImage, 20);
% Dilate to merge regions together.
binaryImage = imdilate(binaryImage, true(9));
% Fill holes to make it more solid;
binaryImage = imfill(binaryImage, 'holes');
subplot(2, 2, 3);
imshow(binaryImage, []);
title('Cleaned Binary Image', 'FontSize', fontSize);
% Find the rows
verticalProfile = sum(binaryImage, 2);
subplot(2, 2, 4);
plot(verticalProfile, 'b-');
grid on;
title('Vertical Profile', 'FontSize', fontSize);
% Find the top and bottom rows - where the jumps are biggest.
[~, topRow] = max(diff(verticalProfile))
[~, bottomRow] = max(-diff(verticalProfile))
% Crop image - erase above top row and below bottom row
binaryImage(1:topRow,:) = false;
binaryImage(bottomRow:end,:) = false;
% Find the columns
horizontalProfile = sum(binaryImage, 1);
figure
subplot(2, 2, 1);
plot(horizontalProfile, 'b-');
grid on;
title('Horizontal Profile', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);
% Find the top and bottom rows - where the jumps are biggest.
[~, leftCol] = max(diff(horizontalProfile))
[~, rightCol] = max(-diff(horizontalProfile))
% Crop image - erase above top row and below bottom row
binaryImage(:, 1:leftCol) = false;
binaryImage(:, rightCol:end) = false;
subplot(2, 2, 2);
imshow(binaryImage, []);
title('Cleaned Binary Image', 'FontSize', fontSize);
% Extract the largest blob
% Find the areas.
labeledImage = bwlabel(binaryImage);
blobMeasurements = regionprops(labeledImage, 'area');
allAreas = [blobMeasurements.Area];
% Sort in descending order
[sortedAreas, sortIndexes] = sort(allAreas, 'descend');
% Extract the blob with the largest area.
maskImage = ismember(labeledImage, sortIndexes(1)) > 0;
% Mask this against the first binary image.
originalBinaryImage(~maskImage) = false;
subplot(2, 2, 3);
imshow(originalBinaryImage, []);
title('Cleaned Binary Image', 'FontSize', fontSize);

Image Analyst 2013-3-31

Sorry but I think I've provided you enough to go on. If I do this, then there will be another problem with another weird image, and I can't go on to do the whole project with you. And I don't have time to delve deeply into any one particular project. I think you have the general concepts enough for you to continue on your own and take pride in ownership of your code. If lab thresholding won't work, try thresholding on RGB and look for large black squares. Of course this won't work the minute you start to look at black calculators instead of white calculators - see what I mean?

Soumyadip 2013-3-31