# Face parts detection with YOLOv8 ����
## Introduction
In this project I use the most recent implementation of YOLO by Ultralytics, [YOLOv8](https://github.com/ultralytics/ultralytics). The goal is to train an algorithm that is able to detect separate face parts without having to use landmark detectors that don't do well when part of the face is occluded or missing. My goal is to also combine frontal, semi-frontal and profile face datasets so that the YOLO model works well on all of them.
It is also a great opportunity to try out the `supervision` library by [Roboflow](https://github.com/roboflow/supervision). Despite it's still in beta, it looks really helpful for some common YOLO-related tasks such as drawing the detections.
![A live demo of YOLOv8 nano](images/live_demo.gif)
## Motivation
All I want these models for is data exploration and check what face parts can be seen in an image. Note that I'm talking about detecting _face parts_, which is not the same as detecting _faces_. I've been asked many times: why not using facial landmark detectors? And the reason is that these do not work well with close-up images, like this one:
![An example of a close-up image where facial landmark detection is not possible](images/closeup_example.jpg)
_Image source: [Pexels](https://images.pexels.com/photos/977601/pexels-photo-977601.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1)_
I know there are several works about facial landmark detection for occluded faces (such as ["Robust face landmark estimation under occlusion"](https://pdollar.github.io/files/papers/BurgosArtizzuICCV13rcpr.pdf)), but a picture of the entire face is always needed. If I wanted to be able to detect face parts in close-up images, I would have to develop something new. And that's what I've done.
## Data
For this experiment I'm using a variety of facial landmark detection datasets. Each dataset came in a different structure, so I had to deal with that in `prepare_full_dataset.py`:
- **Existing datasets**: all these datasets were processed by converting each group of facial landmarks (eye, mouth, nose, eyebrows) to a bounding box compatible with YOLO.
- [Helen dataset](http://www.ifp.illinois.edu/~vuongle2/helen/)
- [Menpo2D dataset](https://github.com/jiankangdeng/MenpoBenchmark)
- [AFW (Annotated Faces in the Wild) dataset](https://ibug.doc.ic.ac.uk/resources/facial-point-annotations/)
- [LaPa (Landmark-guided face Parsing) dataset](https://github.com/JDAI-CV/lapa-dataset)
- [FASSEG (FAce Semantic SEGmentation) dataset](https://github.com/massimomauro/FASSEG-repository): The original labels consider eyebrows and hair as a single class (hair), so I manually annotated subsets V2 and V3. This dataset includes many low-quality profile images, and it will probably improve performance.
- **Custom datasets**:
- [Pexels](https://pexels.com): I downloaded 257 images from this website and annotated them using [CVAT](https://app.cvat.ai/). As of today, I've annotated four batches of images, and I've tried to include pictures where parts of the face are missing.
**I am not sharing any of these datasets**: they are not mine and they are 100% accessible from their corresponding sites. I may release the Pexels dataset that I create, though.
## Results
### Data quality
Some datasets such as Helen may generate noisy examples when the images have more than one face but only one set of landmarks (i.e. the ones corresponding to the "main" face in the image). This is probably affecting the precision because the model is actually detecting all the faces in these images (which is good, though). Other datasets such as AFW have as many landmarks as faces in the images.
![A training batch with some images with incomplete labels](images/example_incomplete_labels.jpg)
### Performance
In this section you can see the performance of the _nano_ model. It struggles with eyebrows, but it works really well for eyes and noses. I would need to add more close-up images of each part to increase the number of incomplete or occluded faces.
![Yolov8-nano F1 curve](images/F1_curve.png)
Here are the metrics of the _nano_ model:
![YOLOv8-nano results](images/results.png)
## Reports
Use `run.py` to run the model on a folder with images to obtain a CSV with all the detections.
没有合适的资源?快使用搜索试试~ 我知道了~
yolov8系列-- Using YOLOv8 to detect face parts.zip
共15个文件
py:6个
png:2个
jpg:2个
需积分: 5 0 下载量 82 浏览量
2024-02-24
21:43:44
上传
评论
收藏 35.6MB ZIP 举报
温馨提示
yolov8系列-- Using YOLOv8 to detect face parts
资源推荐
资源详情
资源评论
收起资源包目录
yolov8系列-- Using YOLOv8 to detect face parts.zip (15个子文件)
kwan1120
live_demo.py 2KB
utils.py 722B
EDA
prepare_fasseg_data.py 1KB
LICENSE 34KB
run.py 3KB
requirements.txt 1KB
.gitignore 33B
images
example_incomplete_labels.jpg 352KB
F1_curve.png 204KB
closeup_example.jpg 62KB
results.png 200KB
live_demo.gif 34.86MB
train.py 1KB
Readme.md 4KB
prepare_full_dataset.py 23KB
共 15 条
- 1
资源评论
Kwan的解忧杂货铺
- 粉丝: 1w+
- 资源: 3625
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功