facenet-pytorch-master.zip

# Face Recognition Using Pytorch [](https://pepy.tech/project/facenet-pytorch) [](https://codecov.io/gh/timesler/facenet-pytorch) | Python | 3.10 | 3.9 | 3.8 | | :---: | :---: | :---: | :---: | | Status | [](https://github.com/timesler/facenet-pytorch/actions?query=workflow%3A%22Python+3.10%22+branch%3Amaster) | [](https://github.com/timesler/facenet-pytorch/actions?query=workflow%3A%22Python+3.9%22+branch%3Amaster) | [](https://github.com/timesler/facenet-pytorch/actions?query=workflow%3A%22Python+3.8%22+branch%3Amaster) | [](https://xscode.com/timesler/facenet-pytorch) This is a repository for Inception Resnet (V1) models in pytorch, pretrained on VGGFace2 and CASIA-Webface. Pytorch model weights were initialized using parameters ported from David Sandberg's [tensorflow facenet repo](https://github.com/davidsandberg/facenet). Also included in this repo is an efficient pytorch implementation of MTCNN for face detection prior to inference. These models are also pretrained. To our knowledge, this is the fastest MTCNN implementation available. ## Table of contents * [Table of contents](#table-of-contents) * [Quick start](#quick-start) * [Pretrained models](#pretrained-models) * [Example notebooks](#example-notebooks) + [*Complete detection and recognition pipeline*](#complete-detection-and-recognition-pipeline) + [*Face tracking in video streams*](#face-tracking-in-video-streams) + [*Finetuning pretrained models with new data*](#finetuning-pretrained-models-with-new-data) + [*Guide to MTCNN in facenet-pytorch*](#guide-to-mtcnn-in-facenet-pytorch) + [*Performance comparison of face detection packages*](#performance-comparison-of-face-detection-packages) + [*The FastMTCNN algorithm*](#the-fastmtcnn-algorithm) * [Running with docker](#running-with-docker) * [Use this repo in your own git project](#use-this-repo-in-your-own-git-project) * [Conversion of parameters from Tensorflow to Pytorch](#conversion-of-parameters-from-tensorflow-to-pytorch) * [References](#references) ## Quick start 1. Install: ```bash # With pip: pip install facenet-pytorch # or clone this repo, removing the '-' to allow python imports: git clone https://github.com/timesler/facenet-pytorch.git facenet_pytorch # or use a docker container (see https://github.com/timesler/docker-jupyter-dl-gpu): docker run -it --rm timesler/jupyter-dl-gpu pip install facenet-pytorch && ipython ``` 1. In python, import facenet-pytorch and instantiate models: ```python from facenet_pytorch import MTCNN, InceptionResnetV1 # If required, create a face detection pipeline using MTCNN: mtcnn = MTCNN(image_size=<image_size>, margin=<margin>) # Create an inception resnet (in eval mode): resnet = InceptionResnetV1(pretrained='vggface2').eval() ``` 1. Process an image: ```python from PIL import Image img = Image.open(<image path>) # Get cropped and prewhitened image tensor img_cropped = mtcnn(img, save_path=<optional save path>) # Calculate embedding (unsqueeze to add batch dimension) img_embedding = resnet(img_cropped.unsqueeze(0)) # Or, if using for VGGFace2 classification resnet.classify = True img_probs = resnet(img_cropped.unsqueeze(0)) ``` See `help(MTCNN)` and `help(InceptionResnetV1)` for usage and implementation details. ## Pretrained models See: [models/inception_resnet_v1.py](models/inception_resnet_v1.py) The following models have been ported to pytorch (with links to download pytorch state_dict's): |Model name|LFW accuracy (as listed [here](https://github.com/davidsandberg/facenet))|Training dataset| | :- | :-: | -: | |[20180408-102900](https://github.com/timesler/facenet-pytorch/releases/download/v2.2.9/20180408-102900-casia-webface.pt) (111MB)|0.9905|CASIA-Webface| |[20180402-114759](https://github.com/timesler/facenet-pytorch/releases/download/v2.2.9/20180402-114759-vggface2.pt) (107MB)|0.9965|VGGFace2| There is no need to manually download the pretrained state_dict's; they are downloaded automatically on model instantiation and cached for future use in the torch cache. To use an Inception Resnet (V1) model for facial recognition/identification in pytorch, use: ```python from facenet_pytorch import InceptionResnetV1 # For a model pretrained on VGGFace2 model = InceptionResnetV1(pretrained='vggface2').eval() # For a model pretrained on CASIA-Webface model = InceptionResnetV1(pretrained='casia-webface').eval() # For an untrained model with 100 classes model = InceptionResnetV1(num_classes=100).eval() # For an untrained 1001-class classifier model = InceptionResnetV1(classify=True, num_classes=1001).eval() ``` Both pretrained models were trained on 160x160 px images, so will perform best if applied to images resized to this shape. For best results, images should also be cropped to the face using MTCNN (see below). By default, the above models will return 512-dimensional embeddings of images. To enable classification instead, either pass `classify=True` to the model constructor, or you can set the object attribute afterwards with `model.classify = True`. For VGGFace2, the pretrained model will output logit vectors of length 8631, and for CASIA-Webface logit vectors of length 10575. ## Example notebooks ### *Complete detection and recognition pipeline* Face recognition can be easily applied to raw images by first detecting faces using MTCNN before calculating embedding or probabilities using an Inception Resnet model. The example code at [examples/infer.ipynb](examples/infer.ipynb) provides a complete example pipeline utilizing datasets, dataloaders, and optional GPU processing. ### *Face tracking in video streams* MTCNN can be used to build a face tracking system (using the `MTCNN.detect()` method). A full face tracking example can be found at [examples/face_tracking.ipynb](examples/face_tracking.ipynb).  ### *Finetuning pretrained models with new data* In most situations, the best way to implement face recognition is to use the pretrained models directly, with either a clustering algorithm or a simple distance metrics to determine the identity of a face. However, if finetuning is required (i.e., if you want to select identity based on the model's output logits), an example can be found at [examples/finetune.ipynb](examples/finetune.ipynb). ### *Guide to MTCNN in facenet-pytorch* This guide demonstrates the functionality of the MTCNN module. Topics covered are: * Basic usage * Image normalization * Face margins * Multiple faces in a single image * Batched detection * Bounding boxes and facial landmarks * Saving face datasets See the [notebook on kaggle](https://www.kaggle.com/timesler/guide-to-mtcnn-in-facenet-pytorch). ### *Performance comparison of face detection packages* This notebook demonstrates the use of three face detection packages: 1. facenet-pytorch 1. mtcnn 1. dlib Each package is tested