Building a $60 face recognition system with NVIDIA Jetson nano 2GB and python

building face recognition nvidia jetson


author |Adam Geitgey
compile |Flin
source |medium

new Nvidia Jetson Nano 2GB Development board ( Announced today !) It's a single board computer , The price is 59 dollar , Run with GPU Accelerated AI Software .

To 2020 year , You can buy it from one 59 Amazing performance in a $1 single board computer . Let's use it to create a simple version of the doorbell camera , The camera can track everyone who goes to the front door of the house . Through facial recognition , Even if these people dress differently , It can also immediately know if someone at your door has ever visited you .

What is? Nvidia Jetson Nano 2GB?

Jets o n Nano 2GB It's a single board computer , It has four cores 1.4GHz ARM CPU And built-in Nvidia Maxwell GPU. It's the cheapest Nvidia Jetson Model , For amateurs who buy raspberry pie .

If you're already familiar with the raspberry pie range , Besides Jetson Nano Equipped with Nvidia GPU Outside , This is almost exactly the same as other products . It can run GPU Accelerated applications ( Such as deep learning model ), It's much faster than a board like raspberry pie ( Does not support most deep learning frameworks GPU) Much faster .

There's a lot of AI Development board and accelerator module , but Nvidia Have a big advantage —— It's with the desktop AI Libraries are directly compatible with , You don't need to convert deep learning models into any special format to run them .

It uses almost everything based on Python The deep learning framework has been used by the same CUDA library GPU Speed up . This means that you can use existing based on Python The deep learning program of , It can be done in almost no modification Jetson Nano 2GB Run it on , And still get good performance ( As long as your app can be in 2GB Of RAM Up operation ).

It will be written for powerful servers Python The code is deployed at a price of 59 The capabilities on the dollar's stand-alone devices are excellent .

This new Jetson Nano 2GB Motherboard is also better than Nvidia The previous hardware version is more glossy .

first Jetson Nano The model is inexplicably lacking WiFi, But the model comes with a pluggable WiFi modular , So you don't have to add messy Ethernet cables . They also upgraded the power input to a more modern USB-C port , And in terms of software , Some rough edges have been worn off . for example , You don't have to perform basic operations like enabling swap files .

Nvidia Actively launched a price lower than 60 The dollar has authenticity GPU Simple and easy to use hardware device . It seems like they're targeting raspberry pie , And try to occupy Education / Enthusiast market . It will be interesting to see how the market reacts .

Let's assemble the system

For any hardware project , The first step is to collect all the parts we need :

1. Nvidia Jetson Nano 2GB a main board (59 dollar )

These boards are now available for booking ( By 2020 year 10 month 5 Japan ), It is expected that 10 Release at the end of month .

I don't know what the initial usability will be after the release , But the previous Jetson Nano The model was in short supply in the months after its launch .

Full disclosure : I got it for free from Inda Jetson Nano 2GB Development board as evaluation unit , But I don't have a financial or editorial relationship with NVIDIA . That's how I was able to write this guide ahead of time .

2. USB-C The power adapter ( You may already have one ?)

new type Jetson Nano 2GB Use USB-C Power supply . Not including power adapter , But you may already have a power adapter .

3. camera —— USB Webcam ( You might have one ?) Or raspberry pie camera module v2.x( about 30 dollar )

If you want to install a small camera in the case , So the raspberry pie camera module v2.x It's a good choice ( Be careful :v1.x The camera module will not work ). You can Amazon Or get them from all kinds of dealers .

some USB Webcam ( Such as Logitech Of C270 or C920) It can also be in Jetson Nano 2GB Upper normal operation , So if you already have one USB camera , It can also be used . Here's an incomplete list of cameras .

Before you buy a new product , Please don't be afraid to try to put any USB equipment . Not all features support Linux The driver , But some features will support . I inserted the value of buying on Amazon 20 The currency of the dollar HDMI To USB Adapter , It works well . therefore , I can use my high-end digital camera without any extra configuration HDMI Video source for .

You need something else , But you may be ready to :

At least with 32GB Spatial microSD card . We will install here Linux. You can reuse any existing microSD card .

One microSD Card reader : So that you can install Jetson Software .

A cable USB Keyboard and a wired USB mouse control Jetson Nano.

Any direct acceptance of HDMI( Not through HDMI-DVI converter ) Of The monitor Or TV , You can see what you're doing . Even if you don't use the monitor later Jetson Nano Initial settings , Also need a monitor .

load Jetson Nano 2GB Software

At the beginning, insert something into Jetson Nano Before , You need to download Jetson Nano Software image of .

Nvidia The default software image includes pre installed Python 3.6 and OpenCV Of Ubuntu Linux 18.04.

Here is what will be Jetson Nano The software is installed into SD The method on the card :

  1. from Nvidia download Jetson Nano Developer Kit SD Card image .
  1. download Etcher, The program will Jetson Software image write SD card .
  1. function Etcher And use it to write and download to SD Card Jetson Nano Developer Kit SD Card image . It takes about 20 minute .

It's time to unpack the rest of the hardware !

Insert all parts

First , Please take out your Jetson Nano 2GB:

The first step is to insert microSD card .microSD The card slot is completely hidden , But you can find it on the back of the bottom of the radiator :

You should also continue with the attached USB WiFi The adapter inserts the following USB One of the ports :

Next , You need to plug in the camera .

If you use raspberry pie v2.x Camera module , It will be connected through a ribbon cable . stay Jetson Find the ribbon cable slot on the , Pop up the connector , Plug in the cable , Then pop it up and close it . Make sure that the metal contacts on the ribbon cable are facing inward toward the radiator :

If you use USB Webcam , Just insert it into USB One of the ports , And ignore the ribbon cable port .

Now? , Insert all the other parts :

Insert the mouse and keyboard into USB port .

Use HDMI Plug the cable into the monitor .

Last , Insert USB-C Power cord to start it .

If you're using a raspberry pie camera module , The result is as follows :

perhaps , If you're using USB Video input device , It will look like this :

After you plug in the power cord ,Jetson Nano It will start automatically . After a few seconds , You should see that Linux The settings screen appears on the monitor . Please follow the steps below to create your account and connect to WiFi. It's simple .

install Linux and Python Library for face recognition

Once it's done Linux The initial setting of , We need to install several libraries that we will use in face recognition system .

stay Jetson Nano Desktop , Open one LXTerminal Window and run the following command . Every time you ask for a password , Please enter the password you entered when creating the user account :

sudo apt-get update
sudo apt-get install python3-pip cmake libopenblas-dev liblapack-dev libjpeg-dev

First , We need to update apt, It's standard Linux Software installation tools , We'll use it to install the other system libraries we need .

then , We're going to install some of the things that we don't need to pre install our software linux library .

Last , We need to install face_recognition Python Libraries and their dependencies , Including machine learning libraries dlib. You can do this automatically with the following single command :

sudo pip3 -v install Cython face_recognition

Because nothing can be used for Jetson Platform dlib and numpy Pre built copies of , Compile from this command library, so . therefore , Take this opportunity to have lunch , Because it could take an hour !

When it's finally done , Yours Jetson Nano 2GB Through the complete CUDA GPU Accelerate face recognition . Move on to the next interesting part !

Run the face recognition doorbell camera demo app

face_recognition The library is one that I wrote Python library , It makes use of DLIB Face recognition is super easy . It allows you to detect the face , Convert each detected face to a unique face code , Then compare these facial codes to see if they could be the same person —— Just a few lines of code .

Use the library , I built a doorbell camera app , The app can identify people who walk to your front door and track them every time they come back . The runtime looks like this :

First , Please download the code . I've added the complete code and comments here .

But here's an easier way to download your Jetson Nano On :

wget -O doorcam.py tiny.cc/doorcam2gb

At the top of the program , You need to edit a line of code to tell you that you are using USB Camera or raspberry pie camera module . You can edit files like this :

gedit doorcam.py

Follow the instructions , And then save it , sign out GEdit And run the code :

python3 doorcam.py

You'll see a video window on the desktop . Every time a new person walks up to the camera , It will record their faces and start tracking their time at your door . If the same person leaves and is in 5 Come back in minutes , It will re register and track them again . You can always press... On the keyboard “ q” sign out .

The app will automatically save everyone's information you see to a name called known_faces.dat In the file of . When you run the program again , It will use that data to remember previous visitors . If you want to clear the list of known faces , Just exit the program and delete the file .

Turn it into a stand-alone hardware device

thus , We have a development board that runs face recognition models , But it's still tied to the desktop , To achieve powerful functions and display effect . Let's see how to run it without having to plug in .

One of the cool things about modern single board computers is , They almost all support the same hardware standards , for example USB. That means you can buy a lot of cheap accessories on Amazon , For example, touch screen displays and batteries . You have a lot of input , Output and power options . This is what I ordered ( But anything like that can ):

One 7 Inch touch screen HDMI display , Use USB Power Supply :

And a generic USB-C Battery pack to power :

Let's connect it , Look at what it looks like when it's running as a stand-alone device . Just insert USB Battery not wall charger , And then HDMI The monitor is plugged in HDMI Port and USB port , Can act as screen and mouse input .

The effect is very good . The touch screen can look like a normal USB Operate like a mouse , No other configuration required . The only downside , If Jetson Nano 2GB More electricity is consumed than USB The amount of electricity that a battery pack can provide , It will reduce GPU The speed of . But it still works well .

With a little bit of creativity , You can package all of this into a project case , Use it as a prototype hardware device to test your own ideas . and , If you want to mass produce certain products , You can buy Jetson The production version of the motherboard , And use it to build real hardware products .

Doorbell camera Python Code walkthrough

Want to know how code works ? Let's work it out step by step .

The code starts by importing the library we're going to use . most important of all OpenCV(Python called cv2), We will use OpenCV Reading images from the camera , And face recognition for detecting and comparing faces .

import face_recognition
import cv2
from datetime import datetime, timedelta
import numpy as np
import platform
import pickle

then , We need to know how to access the camera —— Method and use of image acquisition from raspberry pie camera module USB The camera has different methods . therefore , Just change this variable to... Depending on your hardware True or False that will do :

# The settings here depend on your camera device type :
# - True = Raspberry pie 2.x camera module
# - False = USB webcam or other USB video input (like an HDMI capture device)
USING_RPI_CAMERA_MODULE = False

Next , We're going to create variables to store data about people walking in front of the camera . These variables will act as a simple database of known visitors .

known_face_encodings = []
known_face_metadata = []

The app is just a demo , So we store known faces in Python In the list . In real-world applications that deal with more faces , You may want to use a real database , But I want to keep this demo simple .

Next , We have the ability to save and load known facial data . This is the save function :

def save_known_faces():
with open("known_faces.dat", "wb") as face_data_file:
face_data = [known_face_encodings, known_face_metadata]
pickle.dump(face_data, face_data_file)
print("Known faces backed up to disk.")

This will use Python The built-in pickle Function to write known faces to disk . Data is loaded back in the same way , But I don't show it here .

Every time our program detects a new face , We all call a function to add it to the known face database :

def register_new_face(face_encoding, face_image):
known_face_encodings.append(face_encoding)
known_face_metadata.append({
"first_seen": datetime.now(),
"first_seen_this_interaction": datetime.now(),
"last_seen": datetime.now(),
"seen_count": 1,
"seen_frames": 1,
"face_image": face_image,
})

First , We store the face code representing the face in a list . then , We store a dictionary of matching data about faces in a second list . We're going to use it to track when we first met the person , How long have they been hanging around the cameras lately , The number of times they visited our house and the images of their faces .

We also need an auxiliary function to check whether there is an unknown face in the face database :

def lookup_known_face(face_encoding):
metadata = None
if len(known_face_encodings) == 0:
return metadata
face_distances = face_recognition.face_distance(
known_face_encodings,
face_encoding
)
best_match_index = np.argmin(face_distances)
if face_distances[best_match_index] < 0.65:
metadata = known_face_metadata[best_match_index]
metadata["last_seen"] = datetime.now()
metadata["seen_frames"] += 1
if datetime.now() - metadata["first_seen_this_interaction"]
> timedelta(minutes=5):
metadata["first_seen_this_interaction"] = datetime.now()
metadata["seen_count"] += 1
return metadata

We're doing something important here :

  1. Use face_recogntion library , We check how similar the unknown faces are to all previous visitors . Described face_distance() Function provides a numerical measure of the similarity between an unknown face and all known faces —— The smaller the number is. , The more similar the face is .

  2. If the face is very similar to one of our known visitors , Let's assume they're repeat visitors . under these circumstances , We will update their “ Last time I watched ” Time , And increase the number of times you see them in video frames .

  3. Last , If someone has seen this person in front of the camera in the last five minutes , So let's assume they're still here as part of the same visit . otherwise , Let's assume this is a new visit to our house , So we're going to reset the timestamp that tracks their most recent visits .

The rest of the program is the main loop —— An infinite cycle , In this cycle , We get video frames , Find faces in the image and process every face we see . This is the main core of the program . Let's see :

def main_loop():
if USING_RPI_CAMERA_MODULE:
video_capture =
cv2.VideoCapture(
get_jetson_gstreamer_source(),
cv2.CAP_GSTREAMER
)
else:
video_capture = cv2.VideoCapture(0)

The first step is to access the camera in any way that suits our computer hardware .

Now let's get the video frames :

while True:
# Grab a single frame of video
ret, frame = video_capture.read()
# Resize frame of video to 1/4 size
small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)
# Convert the image from BGR color
rgb_small_frame = small_frame[:, :, ::-1]

Every time you grab a frame of video , We all shrink it down to 1/4 Size . This will make the face recognition process run faster , But the cost is only to detect the larger face in the image . But because we're building a doorbell camera , The camera can only identify people near the camera , So it's not a problem .

We also have to deal with the fact that ,OpenCV Extract images from the camera , Each pixel is stored as blue-green - Red value , Instead of the standard red - green - Blue value . Before face recognition on images , We need to convert the image format .

Now? , We can detect all the faces in the image , And convert each face into a face code . Just two lines of code :

face_locations = face_recognition.face_locations(rgb_small_frame)
face_encodings = face_recognition.face_encodings(
rgb_small_frame,
face_locations
)

Next , We're going to go through every detected face , And determine if the face is someone we've met in the past or a new visitor :

for face_location, face_encoding in zip(
face_locations,
face_encodings):
metadata = lookup_known_face(face_encoding)
if metadata is not None:
time_at_door = datetime.now() -
metadata['first_seen_this_interaction']
face_label = f"At door {int(time_at_door.total_seconds())}s"
else:
face_label = "New visitor!"
# Grab the image of the face
top, right, bottom, left = face_location
face_image = small_frame[top:bottom, left:right]
face_image = cv2.resize(face_image, (150, 150))
# Add the new face to our known face data
register_new_face(face_encoding, face_image)

If we've met this person before , We're going to retrieve the metadata we've stored about their previous access .

without , We add them to our face database , And get their faces from video images to add to our database .

Now we've found all the people and identified them , We can traverse the detected faces again , Just draw a box around each face and label each face :

for (top, right, bottom, left), face_label in
zip(face_locations, face_labels):
# Scale back up face location
# since the frame we detected in was 1/4 size
top *= 4
right *= 4
bottom *= 4
left *= 4
# Draw a box around the face
cv2.rectangle(
frame, (left, top), (right, bottom), (0, 0, 255), 2
)
# Draw a label with a description below the face
cv2.rectangle(
frame, (left, bottom - 35), (right, bottom),
(0, 0, 255), cv2.FILLED
)
cv2.putText(
frame, face_label,
(left + 6, bottom - 6),
cv2.FONT_HERSHEY_DUPLEX, 0.8,
(255, 255, 255), 1
)

I also want to draw a run list of recent visitors at the top of the screen , It includes the number of times they visited your house :

To draw the image , We need to traverse all known faces , And look at the faces in front of the camera recently . For every recent visitor , We will draw their faces on the screen and draw the number of visits :

number_of_recent_visitors = 0
for metadata in known_face_metadata:
# If we met this person at the last minute ,
if datetime.now() - metadata["last_seen"]
< timedelta(seconds=10):
# Drawing known facial images
x_position = number_of_recent_visitors * 150
frame[30:180, x_position:x_position + 150] =
metadata["face_image"]
number_of_recent_visitors += 1
# Label the image with how many times they have visited
visits = metadata['seen_count']
visit_label = f"{visits} visits"
if visits == 1:
visit_label = "First visit"
cv2.putText(
frame, visit_label,
(x_position + 10, 170),
cv2.FONT_HERSHEY_DUPLEX, 0.6,
(255, 255, 255), 1
)

Last , We can display the current video frame on the screen , And draw all the comments on top of it :

cv2.imshow('Video', frame)

To make sure the program doesn't crash , We will every 100 Frame saves a list of known faces to disk :

if len(face_locations) > 0 and number_of_frames_since_save > 100:
save_known_faces()
number_of_faces_since_save = 0
else:
number_of_faces_since_save += 1

When the program exits , Just one or two lines of cleanup code can turn off the camera .

The program's startup code is at the bottom of the program :

if __name__ == "__main__":
load_known_faces()
main_loop()

All we have to do is load the known faces ( If any ), Then start the main loop , The loop always reads from the camera and displays the results on the screen .

The whole program is only about 200 That's ok , But it can detect visitors , Identify them and track them whenever they come to your house .

An interesting fact : This face tracking code runs in ads on many streets and bus stops , To track who's watching the ad and how long it lasts . before , It sounds so far away , But now you spend 60 Dollars can buy the same thing !

add-in

This program is an example , Explains how to use in cheap Jetson Nano 2GB A small amount of running on board Python 3 Code to build powerful systems .

If you want to turn it into a real doorbell camera system , You can add a feature like this : When the system detects a new person at the door , It will use Twilio Send you a text message , Not just on your monitor . Or you can try replacing a simple in memory face database with a real database .

You can also try to convert this program into a completely different program . Read a video , Find content in images , And then the pattern of action is the basis of all kinds of computer vision systems . Try to change the code , See what you can think of ! When you come home to your door , How about having it play your own customized theme music ? You can look at some other examples of face recognition , To learn how to do a similar operation .

Learn about Nvidia Jetson More about the platform

If you want to know about the use of Nvidia Jetson Hardware platform to build more information ,Nvidia There will be new free Jetson Training courses . Check out their website for more information .

They also have great community resources , for example JetsonHacks Website .

If you want to learn more about the use of Python structure ML and AI More information about the system , Check out my other books and articles on my website .

Link to the original text :https://medium.com/@ageitgey/build-a-face-recognition-system-for-60-with-the-new-nvidia-jetson-nano-2gb-and-python-46edbddd7264

Welcome to join us AI Blog station :
http://panchuang.net/

sklearn Machine learning Chinese official documents :
http://sklearn123.com/

Welcome to pay attention to pan Chuang blog resource summary station :
http://docs.panchuang.net/

版权声明
本文为[Artificial intelligence meets pioneer]所创,转载请带上原文链接,感谢

  1. 利用Python爬虫获取招聘网站职位信息
  2. Using Python crawler to obtain job information of recruitment website
  3. Several highly rated Python libraries arrow, jsonpath, psutil and tenacity are recommended
  4. Python装饰器
  5. Python实现LDAP认证
  6. Python decorator
  7. Implementing LDAP authentication with Python
  8. Vscode configures Python development environment!
  9. In Python, how dare you say you can't log module? ️
  10. 我收藏的有关Python的电子书和资料
  11. python 中 lambda的一些tips
  12. python中字典的一些tips
  13. python 用生成器生成斐波那契数列
  14. python脚本转pyc踩了个坑。。。
  15. My collection of e-books and materials about Python
  16. Some tips of lambda in Python
  17. Some tips of dictionary in Python
  18. Using Python generator to generate Fibonacci sequence
  19. The conversion of Python script to PyC stepped on a pit...
  20. Python游戏开发,pygame模块,Python实现扫雷小游戏
  21. Python game development, pyGame module, python implementation of minesweeping games
  22. Python实用工具,email模块,Python实现邮件远程控制自己电脑
  23. Python utility, email module, python realizes mail remote control of its own computer
  24. 毫无头绪的自学Python,你可能连门槛都摸不到!【最佳学习路线】
  25. Python读取二进制文件代码方法解析
  26. Python字典的实现原理
  27. Without a clue, you may not even touch the threshold【 Best learning route]
  28. Parsing method of Python reading binary file code
  29. Implementation principle of Python dictionary
  30. You must know the function of pandas to parse JSON data - JSON_ normalize()
  31. Python实用案例,私人定制,Python自动化生成爱豆专属2021日历
  32. Python practical case, private customization, python automatic generation of Adu exclusive 2021 calendar
  33. 《Python实例》震惊了,用Python这么简单实现了聊天系统的脏话,广告检测
  34. "Python instance" was shocked and realized the dirty words and advertisement detection of the chat system in Python
  35. Convolutional neural network processing sequence for Python deep learning
  36. Python data structure and algorithm (1) -- enum type enum
  37. 超全大厂算法岗百问百答(推荐系统/机器学习/深度学习/C++/Spark/python)
  38. 【Python进阶】你真的明白NumPy中的ndarray吗?
  39. All questions and answers for algorithm posts of super large factories (recommended system / machine learning / deep learning / C + + / spark / Python)
  40. [advanced Python] do you really understand ndarray in numpy?
  41. 【Python进阶】Python进阶专栏栏主自述:不忘初心,砥砺前行
  42. [advanced Python] Python advanced column main readme: never forget the original intention and forge ahead
  43. python垃圾回收和缓存管理
  44. java调用Python程序
  45. java调用Python程序
  46. Python常用函数有哪些?Python基础入门课程
  47. Python garbage collection and cache management
  48. Java calling Python program
  49. Java calling Python program
  50. What functions are commonly used in Python? Introduction to Python Basics
  51. Python basic knowledge
  52. Anaconda5.2 安装 Python 库(MySQLdb)的方法
  53. Python实现对脑电数据情绪分析
  54. Anaconda 5.2 method of installing Python Library (mysqldb)
  55. Python implements emotion analysis of EEG data
  56. Master some advanced usage of Python in 30 seconds, which makes others envy it
  57. python爬取百度图片并对图片做一系列处理
  58. Python crawls Baidu pictures and does a series of processing on them
  59. python链接mysql数据库
  60. Python link MySQL database