You can also read a translated version of this file in Chinese 简体中文版 or in Korean 한국어.
We believe that face recognition will open up a ton of possibilities in how we interact not just with each other, but with objects as well – whether it’s with Glass or not. To help you in your journey of exploring face recognition, we have below a long list of face detection and recognition APIs that you can use for your applications.
Recognize and manipulate faces from Python or from the command line withthe world's simplest face recognition library.
Built using dlib's state-of-the-art face recognitionbuilt with deep learning. The model has an accuracy of 99.38% on theLabeled Faces in the Wild benchmark.
- Built using dlib's state-of-the-art face recognition built with deep learning. The model has an accuracy of 99.38% on the Labeled Faces in the Wild benchmark. This also provides a simple facerecognition command line tool that lets you do face recognition on a folder of images from the command line! Features Find faces in pictures.
- FaceFirst's face recognition system is creating a safer planet through face recognition security software for retailers, airports, law enforcement and more.
- Facial recognition has already been a hot topic of 2018. Now, with the announcement of the iPhone X’s Face ID technology, facial recognition has become an even more popular topic. We wanted to help you get started using facial recognition in your own apps & software, so here is a list of 10 best facial recognition APIs of 2018!
- Researchers at Carnegie Mellon University have put together an open source facial recognition program based on Google’s FaceNet research. Called OpenFace, the developers say that it can recognize.
- OpenCV is an open source computer vision library that has tons of modules like object detection, face recognition, and augmented reality. Although this library is written in C, it also offers battle-tested Java bindings.
This also provides a simple
face_recognition
command line tool that letsyou do face recognition on a folder of images from the command line!Features
Find faces in pictures
Find all the faces that appear in a picture:
Find and manipulate facial features in pictures
Get the locations and outlines of each person's eyes, nose, mouth and chin.
Finding facial features is super useful for lots of important stuff. Download mathematica for students. But you can also use it for really stupid stufflike applying digital make-up (think 'Meitu'):
Identify faces in pictures
Recognize who appears in each photo.
You can even use this library with other Python libraries to do real-time face recognition:
See this example for the code.
Online Demos
User-contributed shared Jupyter notebook demo (not officially supported):
Installation
Requirements
- Python 3.3+ or Python 2.7
- macOS or Linux (Windows not officially supported, but might work)
Installation Options:
Installing on Mac or Linux
First, make sure you have dlib already installed with Python bindings:
Then, install this module from pypi using
pip3
(or pip2
for Python 2):Alternatively, you can try this library with Docker, see this section.
If you are having trouble with installation, you can also try out apre-configured VM.
Installing on an Nvidia Jetson Nano board
- Jetson Nano installation instructions
- Please follow the instructions in the article carefully. There is current a bug in the CUDA libraries on the Jetson Nano that will cause this library to fail silently if you don't follow the instructions in the article to comment out a line in dlib and recompile it.
Installing on Raspberry Pi 2+
Installing on Windows
While Windows isn't officially supported, helpful users have posted instructions on how to install this library:
Installing a pre-configured Virtual Machine image
- Download the pre-configured VM image (for VMware Player or VirtualBox).
Usage
Command-Line Interface
When you install
face_recognition
, you get two simple command-lineprograms:face_recognition
- Recognize faces in a photograph or folder full forphotographs.face_detection
- Find faces in a photograph or folder full for photographs.
face_recognition
command line tool
The
face_recognition
command lets you recognize faces in a photograph orfolder full for photographs.First, you need to provide a folder with one picture of each person youalready know. There should be one image file for each person with thefiles named according to who is in the picture:
Next, you need a second folder with the files you want to identify:
Then in you simply run the command
face_recognition
, passing inthe folder of known people and the folder (or single image) with unknownpeople and it tells you who is in each image:There's one line in the output for each face. The data is comma-separatedwith the filename and the name of the person found.
An
unknown_person
is a face in the image that didn't match anyone inyour folder of known people.face_detection
command line tool
The
face_detection
command lets you find the location (pixel coordinatates)of any faces in an image.Just run the command
face_detection
, passing in a folder of imagesto check (or a single image):It prints one line for each face that was detected. The coordinatesreported are the top, right, bottom and left coordinates of the face (in pixels).
Adjusting Tolerance / Sensitivity
If you are getting multiple matches for the same person, it might be thatthe people in your photos look very similar and a lower tolerance valueis needed to make face comparisons more strict.
You can do that with the
--tolerance
parameter. The default tolerancevalue is 0.6 and lower numbers make face comparisons more strict:If you want to see the face distance calculated for each match in orderto adjust the tolerance setting, you can use
--show-distance true
:More Examples
If you simply want to know the names of the people in each photograph but don'tcare about file names, you could do this:
Speeding up Face Recognition
Face recognition can be done in parallel if you have a computer withmultiple CPU cores. For example, if your system has 4 CPU cores, you canprocess about 4 times as many images in the same amount of time by usingall your CPU cores in parallel.
If you are using Python 3.4 or newer, pass in a
--cpus <number_of_cpu_cores_to_use>
parameter:You can also pass in
--cpus -1
to use all CPU cores in your system.Python Module
You can import the
face_recognition
module and then easily manipulatefaces with just a couple of lines of code. It's super easy!API Docs: https://face-recognition.readthedocs.io.
Automatically find all the faces in an image
See this exampleto try it out.
You can also opt-in to a somewhat more accurate deep-learning-based face detection model. Microsoft visual studio 2010 download for windows 7 64 bit.
Note: GPU acceleration (via NVidia's CUDA library) is required for goodperformance with this model. You'll also want to enable CUDA supportwhen compliling
dlib
.See this exampleto try it out.
If you have a lot of images and a GPU, you can alsofind faces in batches.
Automatically locate the facial features of a person in an image
See this exampleto try it out.
Recognize faces in images and identify who they are
See this exampleto try it out.
Python Code Examples
All the examples are available here.
Face Detection
Facial Features
Facial Recognition
Creating a Standalone Executable
If you want to create a standalone executable that can run without the need to install
python
or face_recognition
, you can use PyInstaller. However, it requires some custom configuration to work with this library. See this issue for how to do it.Articles and Guides that cover face_recognition
- My article on how Face Recognition works: Modern Face Recognition with Deep Learning
- Covers the algorithms and how they generally work
- Face recognition with OpenCV, Python, and deep learning by Adrian Rosebrock
- Covers how to use face recognition in practice
- Raspberry Pi Face Recognition by Adrian Rosebrock
- Covers how to use this on a Raspberry Pi
- Face clustering with Python by Adrian Rosebrock
- Covers how to automatically cluster photos based on who appears in each photo using unsupervised learning
How Face Recognition Works
If you want to learn how face location and recognition work instead ofdepending on a black box library, read my article.
Caveats
- The face recognition model is trained on adults and does not work very well on children. It tends to mixup children quite easy using the default comparison threshold of 0.6.
- Accuracy may vary between ethnic groups. Please see this wiki page for more details.
Since
face_recognition
depends on dlib
which is written in C++, it can be tricky to deploy an appusing it to a cloud hosting provider like Heroku or AWS.To make things easier, there's an example Dockerfile in this repo that shows how to run an app built with
face_recognition
in a Docker container. With that, you should be able to deployto any service that supports Docker images.You can try the Docker image locally by running:
docker-compose up --build
Linux users with a GPU (drivers >= 384.81) and Nvidia-Docker installed can run the example on the GPU: Open the docker-compose.yml file and uncomment the
dockerfile: Dockerfile.gpu
and runtime: nvidia
lines.Having problems?
If you run into problems, please read the Common Errors section of the wiki before filing a github issue.
Thanks
- Many, many thanks to Davis King (@nulhom)for creating dlib and for providing the trained facial feature detection and face encoding modelsused in this library. For more information on the ResNet that powers the face encodings, check outhis blog post.
- Thanks to everyone who works on all the awesome Python data science libraries like numpy, scipy, scikit-image,pillow, etc, etc that makes this kind of stuff so easy and fun in Python.
- Thanks to Cookiecutter and theaudreyr/cookiecutter-pypackage project templatefor making Python project packaging way more tolerable.
by Manish Bansal
Ever since the Artificial Intelligence boom began — or the iPhone X advertisement featuring the face unlock feature hit TV screens — I’ve wanted to try this technology. However, once I started googling about it, I typically only found code examples in Python. And being a Java enthusiast for seven years, I got demotivated seeing that. Therefore, I finally decided to hunt for Java open source libraries for this.
Currently, there are various Java libraries out there. But the most popular one I found was OpenCV.
OpenCV is an open source computer vision library that has tons of modules like object detection, face recognition, and augmented reality. Although this library is written in C++, it also offers battle-tested Java bindings.
However, there is one issue. As part of its software release, it offers only a few modules (with Java bindings) out of the box — and facial recognition is not one of them. Therefore, to use it, you need to manually build it.
Wait! What? Why?
Yes — the reason cited by the OpenCV community is that the modules are not completely stable. Therefore, they are not bundled along with the standard release. Hence, they maintain them in a separate repository here.
If you have no or very little C++ experience (like me), you must have already started to feel dizzy about building a C++ library yourself. But don’t worry, I am here to hold your hand and walk you through this tedious process. So let’s begin, shall we?
Building OpenCV for Java from scratch
You can find various resources for step by step instructions like this, this, and this. However, none of them worked perfectly for me, as one thing or another was missing. The closest I found, which helped me, is this one. However, you do not need to refer to it. You can follow below steps and you will be good.
First, you need to have the below software on your PC. Here, I am building a 64-bit version of the library as I own a 64 bit PC. But you can build it for 32-bit as well.
The required software is:
- Cmake (I used 3.6.0 rc-4 version).
- Ant (used internally for building JAR)
- MinGW — W64 GCC-8.1.0
A word about MinGW: Here, to build this library, we need C++ compilers. You can use Visual Studio tools (VS), which is far better. However, I did not have the luxury to do that, as I built it on my office laptop and VS is licensed software unavailable to Java people here. Therefore, I had to use open source tools, and the best one is MinGW (Minimalist GNU for Windows).
Also, it is very important to use the correct version of MinGW. Download version x86_64-posix-seh, as there is thread support in this version. I have not tried all other versions. But version x86_64-win32-sjlj does not work at all.
To give some more perspective, the build is done by the utility called make which comes as part of MinGW (bin/mingw32-make.exe). make is a task runner for C++ like “Ant” is for Java. But C++ code and make scripts are very much platform-dependent. Hence, to make the distributables platform-independent, the utility CMake is used. CMake generates platform-dependent make scripts.
Generating build configurations using CMake
Step 1: Download the source code zip of both the opencv and opencv_contrib, and extract them into a directory. Further, create a folder called “build” in the same directory (I created “build_posix” as visible in the screenshots).
Step 2: Open CMake. Point “where is the source code” to the opencv extracted folder. Further, point “where to build the binaries” to the “build” folder you created.
Step 3: Add the 64 bit JDK 1.8 bin folder, the MinGW bin folder, and the Ant bin folder to the “PATH” environment variables. This is important, as CMake will look in the environment variables for configuration. If this is not done, then we will have to configure CMake manually in step 5.
In case you have multiple JDKs in your system and you already have some different JDK in “PATH” & you don’t want to add JDK 1.8 in “PATH”, you can skip this. But do configure it manually in step 5.
Step 4: Press the “Configure” button and select “ MinGw Makefiles” and “finish”. After this, CMake will start configuring your project. It will take a while and, after it finishes configuring, it will show the current available configurations.
In case you are wondering if the configurations generated for you are correct, you can refer to the logs which got generated for me here and compare.
Step 5: Now comes the most important part — changing the configurations. First, click the checkboxes “Grouped” and “Advanced” to organize the configurations.
- Verify that ANT_EXECUTABLE (search “ANT_EXECUTABLE” in the search box) and all five “JAVA” configurations are pointing to the 64-bit JDK 1.8. If Step 3 was done properly, then this will be correct. Otherwise, correct them.
- Un-check Python (search “Python”) related check boxes under “BUILD” and “INSTALL” groups as we don’t need Python builds.
- Disable “WITH_MSMF” and “WITH_IPP & WITH_TBB”. These libs are only available for VS.
- Edit “OPENCV_EXTRA_MODULES_PATH” under “OPENCV” group and set it to the “modules” folder under the “opencv_contrib” source folder you extracted earlier.
After this, press the “Configure” button again. This will do the final configurations. You can refer to the logs which got generated for me here.
Note: Make sure to compare your “Configure” logs generated with the one I shared in pastebin above. If you find some major difference, then first try correcting your configurations and press “Configure” again. Otherwise, there are chances that your build will fail and that it will be more difficult to debug.
Step 6: After this, press “Generate”. It will take few seconds and then close CMake.
Compiling OpenCV
Is Facial Recognition Software Legal
Now, if all the configurations generated above are correct, this task will be a breeze (of 2–3 hours!). Just open the command prompt, go to the “build” folder, and execute the command below.
Here,
-j5
is added, which instructs the make utility to run five jobs in parallel. This will make your build faster, at least theoretically.Further, do not forget to push the logs to a text file. These might get too big, in which case your command prompt window might truncate it. You need them in case compilation fails. You can refer to the compilation logs generated in my case here.
Note: The order of log statements might not be the same for you, as the build is happening in five parallel threads.
Once the build is over, you can check the “bin” and “lib” folders inside your “build” directory. Inside “bin”, you will have all your opencv*.exe’s and libopencv*.dll’s and your compiled JAR. Further, “lib” will have your main dll (libopencv_javaxxx.dll) along with some more dependent files.
Hands on with OpenCV face recognition API
Now that you have built the library, you first need to set up the environment variables, as well as the user library, in Eclipse.
- Create a variable OPENCV_JAVA_BIN and point it to the “bin” folder generated inside your “build” directory.
- Create OPENCV_JAVA_LIB and point it to the “lib” folder generated inside your “build” directory.
- Append both the above variables to the “PATH” variable.
- Open up your Eclipse and create a new user library which you will be using for your face recognition project. Go to “Window” > “Preferences”. From the menu, navigate under “Java” > “Build Path” > “User Libraries” and choose “New…” . Enter a name for the library — for example, opencv — and select the newly created user library. Choose “Add External JARs…” , and browse to select “opencv-3xx.jar” from your computer.
After this, there is no need to link the native library, as this was added to your “path” variables in Step 3.
Once you are done with this setup, you can clone my Git repository from here and import the project into your Eclipse workspace. Further, you will need to add JDK 1.8 as well as the opencv user library (just created above) to this project. Once you are done, you will be ready to test your newly built OpenCV library.
As of this writing, there are three programs in this project.
- HelloWorld: you can run this to test if your OpenCV library setup is ok. If this does not work properly, you need to sort this out first. The only issues you will encounter at this point will be related to system environment variables or user library setup.
- FaceDetection: you can use this to test the face detection module. It is a different module from face recognition. This is a module which gets shipped along with standard release of OpenCV. As of this writing, we can provide an image as an input to the program, and it will detect all the faces inside the image. The output image has green rectangles drawn on all the detected faces.
- FaceRecognition: the OpenCV facerec module includes three algorithms:
- Eigenfaces
- Fisherfaces
- Local Binary Patterns Histograms.
For technical details on all these algorithms, you can refer this official article. For demonstration purposes, I will show you how to use the Eigenfaces algorithm.
First, you need to download training data from the face database. This data contains ten different images for each of 40 distinct subjects (400 images). For some subjects, the images were taken at different times, varying the lighting, facial expressions (open / closed eyes, smiling / not smiling), and facial details (glasses / no glasses). After extracting them on your computer, you need to prepare a .csv file containing the path of each image, along with their corresponding label.
To make it easy, I have one TrainingData.txt in my Git repository. However, you need to edit the file and alter the paths of images as per your computer directory location.
Note: the downloaded face database contains images in .pgm format. This format is not supported by Windows. To actually convert them to .jpg, I added PGMToJPGConverter to my repository. You can use to this to convert the images and have an actual look at the training data.
After this, you can run the face recognition program. Below are the steps performed in the program:
- OpenCV library is loaded as usual.
- The .csv file is read, and two ArrayList(s) are created. One for the matrix of images and other for their corresponding labels.
- Out of the 400 input images, the last entry in the list data structure is removed and saved for testing the trained model later.
- After that, the remaining 399 images are used for training the Eigenfaces algorithm.
- Once training is complete, the model is asked to predict the label of the image we removed in step 3.
Here, we can observe that the algorithm is able to predict the label of our test subject with a confidence value of 1807. The lower the value, the better the prediction. Similarly, you can perform this exercise with two other algorithms. The C++ code can be downloaded from here and here.
Update (27th Dec 2018): In case you find building the openCV java bindings painful, then i have a good news for you. Recently, I have found an easier way to get all the openCV dependencies for java. For complete details, please refer my another article.
Congratulations!! ? You made it to the end. And if you liked ?this article, hit that clap button below ?. It means a lot to me and it helps other people see the story.