# Recognizing numbers using OpenCV and python

AI Hao 2021-11-25 11:27:33
recognizing numbers using opencv python

## Use OpenCV and Python Identification Numbers

This article demonstrates how to use OpenCV and Python Recognize the numbers in the image .

In the first part of this tutorial , We will discuss what is a seven segment display , And how we apply computer vision and image processing operations to identify these types of numbers （ No need for machine learning ！）

# Seven segment display

You may already be familiar with the seven segment monitor , Even if you don't know a particular term . A good example of this display is your classic digital alarm clock ：

Each number on the alarm clock is represented by a seven segment component , As shown below ：

The seven segment display can present a total of 128 All possible states ：

We are only interested in 10 I'm interested in —— Numbers 0 To 9：

Our goal is to write OpenCV and Python Code to identify each of the ten digital states in the image .

# Design OpenCV Digital recognizer

We will use the thermostat image as input ：

The steps of identification ：

step 1： Position the... On the thermostat LCD. This can be done using edge detection , Because the plastic shell and LCD There is enough contrast between .

step 2： extract LCD. Given an input edge graph , I can find the outline and look for the outline of the rectangle —— The largest rectangular area should correspond to LCD. Perspective transformation will give me a good LCD extract .

step 3： Extract the digital area . Once I have LCD In itself , I can focus on extracting numbers . Due to the digital area and LCD There seems to be a contrast between the backgrounds , I believe that threshold and morphological operations can achieve this .

step 4： Identification Numbers . Use OpenCV Identifying actual numbers will involve ROI Divided into seven parts . From there, I can apply pixel counts to the threshold image to determine whether a given segment is “ open ” still “ Turn off ”.

So let's see how we use OpenCV and Python Complete this four step process for number recognition , Read on .

# Use computer vision and OpenCV Identification Numbers

Let's continue with this example . Create a new file , I'm going to call it identify_digits.py , And insert the following code ：

``````# import the necessary packages
from imutils.perspective import four_point_transform
from imutils import contours
import imutils
import cv2
# define the dictionary of digit segments so we can identify
# each digit on the thermostat
DIGITS_LOOKUP = {

(1, 1, 1, 0, 1, 1, 1): 0,
(0, 0, 1, 0, 0, 1, 0): 1,
(1, 0, 1, 1, 1, 1, 0): 2,
(1, 0, 1, 1, 0, 1, 1): 3,
(0, 1, 1, 1, 0, 1, 0): 4,
(1, 1, 0, 1, 0, 1, 1): 5,
(1, 1, 0, 1, 1, 1, 1): 6,
(1, 0, 1, 0, 0, 1, 0): 7,
(1, 1, 1, 1, 1, 1, 1): 8,
(1, 1, 1, 1, 0, 1, 1): 9
}
``````

Import what we need Python package . introduce mutils, This is my series of convenience functions , Can be used more easily OpenCV + Python. If you have not already installed imutils, It's time to use pip Install the package on your system ： Use OpenCV and Python Identification Numbers

``````pip install imutils
``````

Define a name DIGITS_LOOKUP Of Python Dictionaries . Their key to the table is a seven segment array . Array 1 Indicates that the given segment is open , Zero indicates that the segment is closed . The value is the actual number itself ：0-9.

Once we identify the segment in the thermostat display , We can pass the array to our DIGITS_LOOKUP In the table and obtain numerical values . As a reference , The dictionary uses the same as the figure above 2 The same segment order in . Let's continue with our example ：

``````# load the example image
# pre-process the image by resizing it, converting it to
# graycale, blurring it, and computing an edge map
image = imutils.resize(image, height=500)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
edged = cv2.Canny(blurred, 50, 200, 255)
``````

Then we preprocess the image in the following ways ：-

• Resize .
• Convert the image to grayscale .
• Use 5×5 The kernel applies Gaussian blur to reduce high-frequency noise .
• adopt Canny The edge detector calculates the edge map .

After applying these preprocessing steps , Our edge graph is shown below ：

Be careful LCD How the outline of the is clearly visible —— This completes the steps #1. We can now move on to the second 2 Step , extract LCD In itself ：

``````# find contours in the edge map, then sort them by their
# size in descending order
cnts = cv2.findContours(edged.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
cnts = sorted(cnts, key=cv2.contourArea, reverse=True)
displayCnt = None
# loop over the contours
for c in cnts:
# approximate the contour
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.02 * peri, True)
# if the contour has four vertices, then we have found
# the thermostat display
if len(approx) == 4:
displayCnt = approx
break
``````

In order to find LCD Area , We need to extract the contour of the region in the edge graph （ That is, contour ）.

Then we sort the contours by area , Make sure to place the larger contour line at the front of the list .

Give us a sorted outline list , Cycle them one by one and apply contour approximation .

If our approximate contour has four vertices , So let's assume we've found the thermostat display . This is a reasonable assumption , Because the largest rectangular area in our input image should be LCD In itself .

After getting four vertices , We can extract... Through four point perspective transformation LCD：

``````# extract the thermostat display, apply a perspective transform
# to it
warped = four_point_transform(gray, displayCnt.reshape(4, 2))
output = four_point_transform(image, displayCnt.reshape(4, 2))
``````

Applying this perspective transformation provides us with a top-down LCD Aerial view ：

get LCD This view of meets the requirements of 2 Step —— We are now ready to start from LCD Extract numbers from ：

``````# threshold the warped image, then apply a series of morphological
# operations to cleanup the thresholded image
thresh = cv2.threshold(warped, 0, 255,
cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (1, 5))
thresh = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel)
``````

In order to get the number itself , We need to threshold the distorted image , On a brighter background （ namely LCD The background of the display ） Show dark areas in （ The digital ）：

Then we apply a series of morphological operations to clean up the threshold image ：

Now we have a good segmentation image , We need to apply contour filtering again , Only this time we are looking for the actual number ：

``````# find contours in the thresholded image, then initialize the
# digit contours lists
cnts = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
digitCnts = []
# loop over the digit area candidates
for c in cnts:
# compute the bounding box of the contour
(x, y, w, h) = cv2.boundingRect(c)
# if the contour is sufficiently large, it must be a digit
if w >= 15 and (h >= 30 and h <= 40):
digitCnts.append(c)
``````

So , We find the contour in the threshold image . initialization digitsCnts list —— This list will store the outline of the number itself .

Cycle through each contour .

For each profile , We calculate the bounding box , Make sure the width and height are acceptable , If it is , Update digitsCnts list .

If we loop through digitsCnts Outline the interior and draw a bounding box on the image , The results will be as follows ：

Sure enough , We found the number on the LCD ！ The last step is to actually identify each number ：

``````# sort the contours from left-to-right, then initialize the
# actual digits themselves
digitCnts = contours.sort_contours(digitCnts,
method="left-to-right")[0]
digits = []
``````

ad locum , We're just based on (x, y) The coordinates sort the digital contours from left to right .

This sorting step is necessary , Because there is no guarantee that the outline has been sorted from left to right （ In the same direction we read numbers ）.

Next is the actual number recognition process ：

``````# loop over each of the digits
for c in digitCnts:
# extract the digit ROI
(x, y, w, h) = cv2.boundingRect(c)
roi = thresh[y:y + h, x:x + w]
# compute the width and height of each of the 7 segments
# we are going to examine
(roiH, roiW) = roi.shape
(dW, dH) = (int(roiW * 0.25), int(roiH * 0.15))
dHC = int(roiH * 0.05)
# define the set of 7 segments
segments = [
((0, 0), (w, dH)), # top
((0, 0), (dW, h // 2)), # top-left
((w - dW, 0), (w, h // 2)), # top-right
((0, (h // 2) - dHC) , (w, (h // 2) + dHC)), # center
((0, h // 2), (dW, h)), # bottom-left
((w - dW, h // 2), (w, h)), # bottom-right
((0, h - dH), (w, h)) # bottom
]
on = [0] * len(segments)
``````

Traverse each digital contour .

For each of these areas , We calculate the bounding box and extract the number ROI.

I include each number below ROI Of GIF Animation ：

Given number ROI, We now need to locate and extract the seven parts of the digital display .

according to ROI Dimension calculate the approximate width and height of each segment . Then we define a (x, y) Coordinates the list , These coordinates correspond to seven line segments . This list follows the figure above 2 Same segment order . This is an example GIF Animation , It draws a green box on the current clip being investigated ：

Last , Initialize our on list —— The values in this list 1 Indicates that the given segment is “ open ” Of , A value of zero indicates that the segment is “ closed ”. Given seven display segments (x, y) coordinate , It is quite easy to identify whether a segment is open or closed ：

``````# loop over the segments
for (i, ((xA, yA), (xB, yB))) in enumerate(segments):
# extract the segment ROI, count the total number of
# thresholded pixels in the segment, and then compute
# the area of the segment
segROI = roi[yA:yB, xA:xB]
total = cv2.countNonZero(segROI)
area = (xB - xA) * (yB - yA)
# if the total number of non-zero pixels is greater than
# 50% of the area, mark the segment as "on"
if total / float(area) > 0.5:
on[i]= 1
# lookup the digit and draw it on the image
digit = DIGITS_LOOKUP[tuple(on)]
digits.append(digit)
cv2.rectangle(output, (x, y), (x + w, y + h), (0, 255, 0), 1)
cv2.putText(output, str(digit), (x - 10, y - 10),
cv2.FONT_HERSHEY_SIMPLEX, 0.65, (0, 255, 0), 2)
``````

We start looping through each line segment (x, y) coordinate .

We extract fragments ROI, Then calculate the non-zero pixel number （ That is, in the fragment “ Turn on ” The number of pixels ）.

If the ratio of non-zero pixels to the total area of the segment is greater than 50%, Then we can assume that the paragraph is “on” And update our... Accordingly on list . After seven stages of the cycle , We can pass the list to DIGITS_LOOKUP To get the number itself .

Then we draw a bounding box around the number and display the number on the output image . Last , Our last code block prints the number onto our screen and displays the output image ：

``````# display the digits
print(u"{}{}.{} \u00b0C".format(*digits))
cv2.imshow("Input", image)
cv2.imshow("Output", output)
cv2.waitKey(0)
``````

Please note how we use Python and OpenCV Correctly identify LCD The numbers on the screen ：

# summary

In today's blog post , I demonstrated how to use OpenCV and Python To identify the numbers in the image .

This method is specifically used for seven segment displays （ That is, the digital display you usually see on the digital alarm clock ）.

By extracting each of the seven segments and applying basic threshold and morphological operations , We can determine which paragraphs are “ open ” Of , Which are “ Turn off ” Of .

From there, , We can do it in Python Find open... In dictionary data structure / Close the section to quickly determine the actual number —— No machine learning required ！

As I mentioned at the beginning of this blog post , The application of computer vision to identify the numbers in the thermostat image often makes the problem itself too complex —— Using a data recording thermometer will be more reliable , And the workload is much less .

I hope you like today's blog ！

https://pythonmana.com/2021/11/20211109010517010P.html