<div align="left">
## 注意下载model放置在LapDepth文件夹下:链接:https://pan.baidu.com/s/1Hp_GT3eVwhXN6uXapm0KAQ 提取码:5lgz
## You Only :eyes: Once for Panoptic :car: Perception
> [**You Only Look at Once for Panoptic driving Perception**](https://link.springer.com/article/10.1007/s11633-022-1339-y)
>
> by Dong Wu, Manwen Liao, Weitian Zhang, [Xinggang Wang](https://xinggangw.info/)<sup> :email:</sup>, [Xiang Bai](https://scholar.google.com/citations?user=UeltiQ4AAAAJ&hl=zh-CN), [Wenqing Cheng](http://eic.hust.edu.cn/professor/chengwenqing/), [Wenyu Liu](http://eic.hust.edu.cn/professor/liuwenyu/) [*School of EIC, HUST*](http://eic.hust.edu.cn/English/Home.htm)
>
> (<sup>:email:</sup>) corresponding author.
>
> *arXiv technical report ([Machine Intelligence Research2022](https://link.springer.com/article/10.1007/s11633-022-1339-y))*
---
[中文文档](https://github.com/hustvl/YOLOP/blob/main/README%20_CH.md)
### The Illustration of YOLOP
![yolop](pictures/yolop.png)
### Contributions
* We put forward an efficient multi-task network that can jointly handle three crucial tasks in autonomous driving: object detection, drivable area segmentation and lane detection to save computational costs, reduce inference time as well as improve the performance of each task. Our work is the first to reach real-time on embedded devices while maintaining state-of-the-art level performance on the `BDD100K `dataset.
* We design the ablative experiments to verify the effectiveness of our multi-tasking scheme. It is proved that the three tasks can be learned jointly without tedious alternating optimization.
* We design the ablative experiments to prove that the grid-based prediction mechanism of detection task is more related to that of semantic segmentation task, which is believed to provide reference for other relevant multi-task learning research works.
### Results
[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/yolop-you-only-look-once-for-panoptic-driving/traffic-object-detection-on-bdd100k)](https://paperswithcode.com/sota/traffic-object-detection-on-bdd100k?p=yolop-you-only-look-once-for-panoptic-driving)
#### Traffic Object Detection Result
| Model | Recall(%) | mAP50(%) | Speed(fps) |
| -------------- | --------- | -------- | ---------- |
| `Multinet` | 81.3 | 60.2 | 8.6 |
| `DLT-Net` | 89.4 | 68.4 | 9.3 |
| `Faster R-CNN` | 81.2 | 64.9 | 8.8 |
| `YOLOv5s` | 86.8 | 77.2 | 82 |
| `YOLOP(ours)` | 89.2 | 76.5 | 41 |
#### Drivable Area Segmentation Result
| Model | mIOU(%) | Speed(fps) |
| ------------- | ------- | ---------- |
| `Multinet` | 71.6 | 8.6 |
| `DLT-Net` | 71.3 | 9.3 |
| `PSPNet` | 89.6 | 11.1 |
| `YOLOP(ours)` | 91.5 | 41 |
#### Lane Detection Result:
| Model | mIOU(%) | IOU(%) |
| ------------- | ------- | ------ |
| `ENet` | 34.12 | 14.64 |
| `SCNN` | 35.79 | 15.84 |
| `ENet-SAD` | 36.56 | 16.02 |
| `YOLOP(ours)` | 70.50 | 26.20 |
#### Ablation Studies 1: End-to-end v.s. Step-by-step:
| Training_method | Recall(%) | AP(%) | mIoU(%) | Accuracy(%) | IoU(%) |
| --------------- | --------- | ----- | ------- | ----------- | ------ |
| `ES-W` | 87.0 | 75.3 | 90.4 | 66.8 | 26.2 |
| `ED-W` | 87.3 | 76.0 | 91.6 | 71.2 | 26.1 |
| `ES-D-W` | 87.0 | 75.1 | 91.7 | 68.6 | 27.0 |
| `ED-S-W` | 87.5 | 76.1 | 91.6 | 68.0 | 26.8 |
| `End-to-end` | 89.2 | 76.5 | 91.5 | 70.5 | 26.2 |
#### Ablation Studies 2: Multi-task v.s. Single task:
| Training_method | Recall(%) | AP(%) | mIoU(%) | Accuracy(%) | IoU(%) | Speed(ms/frame) |
| --------------- | --------- | ----- | ------- | ----------- | ------ | --------------- |
| `Det(only)` | 88.2 | 76.9 | - | - | - | 15.7 |
| `Da-Seg(only)` | - | - | 92.0 | - | - | 14.8 |
| `Ll-Seg(only)` | - | - | - | 79.6 | 27.9 | 14.8 |
| `Multitask` | 89.2 | 76.5 | 91.5 | 70.5 | 26.2 | 24.4 |
#### Ablation Studies 3: Grid-based v.s. Region-based:
| Training_method | Recall(%) | AP(%) | mIoU(%) | Accuracy(%) | IoU(%) | Speed(ms/frame) |
| --------------- | --------- | ----- | ------- | ----------- | ------ | --------------- |
| `R-CNNP Det(only)` | 79.0 | 67.3 | - | - | - | - |
| `R-CNNP Seg(only)` | - | - | 90.2 | 59.5 | 24.0 | - |
| `R-CNNP Multitask` | 77.2(-1.8)| 62.6(-4.7)| 86.8(-3.4)| 49.8(-9.7)| 21.5(-2.5)| 103.3 |
| `YOLOP Det(only)` | 88.2 | 76.9 | - | - | - | - |
| `YOLOP Seg(only)` | - | - | 91.6 | 69.9 | 26.5 | - |
| `YOLOP Multitask` | 89.2(+1.0)| 76.5(-0.4)| 91.5(-0.1)| 70.5(+0.6)| 26.2(-0.3)| 24.4 |
**Notes**:
- The works we has use for reference including `Multinet` ([paper](https://arxiv.org/pdf/1612.07695.pdf?utm_campaign=affiliate-ir-Optimise%20media%28%20South%20East%20Asia%29%20Pte.%20ltd._156_-99_national_R_all_ACQ_cpa_en&utm_content=&utm_source=%20388939),[code](https://github.com/MarvinTeichmann/MultiNet)),`DLT-Net` ([paper](https://ieeexplore.ieee.org/abstract/document/8937825)),`Faster R-CNN` ([paper](https://proceedings.neurips.cc/paper/2015/file/14bfa6bb14875e45bba028a21ed38046-Paper.pdf),[code](https://github.com/ShaoqingRen/faster_rcnn)),`YOLOv5s`([code](https://github.com/ultralytics/yolov5)) ,`PSPNet`([paper](https://openaccess.thecvf.com/content_cvpr_2017/papers/Zhao_Pyramid_Scene_Parsing_CVPR_2017_paper.pdf),[code](https://github.com/hszhao/PSPNet)) ,`ENet`([paper](https://arxiv.org/pdf/1606.02147.pdf),[code](https://github.com/osmr/imgclsmob)) `SCNN`([paper](https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/download/16802/16322),[code](https://github.com/XingangPan/SCNN)) `SAD-ENet`([paper](https://openaccess.thecvf.com/content_ICCV_2019/papers/Hou_Learning_Lightweight_Lane_Detection_CNNs_by_Self_Attention_Distillation_ICCV_2019_paper.pdf),[code](https://github.com/cardwing/Codes-for-Lane-Detection)). Thanks for their wonderful works.
- In table 4, E, D, S and W refer to Encoder, Detect head, two Segment heads and whole network. So the Algorithm (First, we only train Encoder and Detect head. Then we freeze the Encoder and Detect head as well as train two Segmentation heads. Finally, the entire network is trained jointly for all three tasks.) can be marked as ED-S-W, and the same for others.
---
### Visualization
#### Traffic Object Detection Result
![detect result](pictures/detect.png)
#### Drivable Area Segmentation Result
![](pictures/da.png)
#### Lane Detection Result
![](pictures/ll.png)
**Notes**:
- The visualization of lane detection result has been post processed by quadratic fitting.
---
### Project Structure
```python
├─inference
│ ├─images # inference images
│ ├─output # inference result
├─lib
│ ├─config/default # configuration of training and validation
│ ├─core
│ │ ├─activations.py # activation function
│ │ ├─evaluate.py # calculation of metric
│ │ ├─function.py # training and validation of model
│ │ ├─general.py #calculation of metric、nms、conversion of data-format、visualization
│ │ ├─loss.py # loss function
│ │ ├─postprocess.py # postprocess(refine da-seg and ll-seg, unrelated to paper)
│ ├─dataset
│ │ ├─AutoDriveDataset.py # Superclass dataset,general function
│ │ ├─bdd.py # Subclass dataset,specific function
│ │ ├─hust.py # Subclass dataset(Campus sce
没有合适的资源?快使用搜索试试~ 我知道了~
温馨提示
自动驾驶全景感知辅助系统软件是在 windows10 系统平台上,以 python3.9为语言环境并依托 qt 库开发的辅助驾驶软件。其能同时处理交通目标检测、可行驶区域划分、车道线检测、前车距离估计、显示深度图五个视觉感知任务,并且速度优异、能保持较好精度进行工作。 qt项目,工具源码,适合毕业设计、课程设计作业,所有源码均经过严格测试,可以直接运行,可以放心下载使用。有任何使用问题欢迎随时与博主沟通,第一时间进行解答! 软件开发设计:PHP、QT、应用软件开发、系统软件开发、移动应用开发、网站开发C++、Java、python、web、C#等语言的项目开发与学习资料 硬件与设备:单片机、EDA、proteus、RTOS、包括计算机硬件、服务器、网络设备、存储设备、移动设备等 操作系统:LInux、IOS、树莓派、安卓开发、微机操作系统、网络操作系统、分布式操作系统等。此外,还有嵌入式操作系统、智能操作系统等。 云计算与大数据:数据集、包括云计算平台、大数据分析、人工智能、机器学习等,云计算是一种基于互联网的计算方式,通过这种方式,共享的软硬件资源和信息可以按需提供给计算机和其他设备。
资源推荐
资源详情
资源评论
收起资源包目录
毕业设计&课程设计-自动驾驶全景感知辅助系统软件是在 windows10 系统平台.zip (123个子文件)
output.avi 6KB
output.avi 6KB
infer_files.cpp 8KB
main.cpp 6KB
yololayer.cu 12KB
input1.gif 3.54MB
output1.gif 3.41MB
output2.gif 3.16MB
input2.gif 3.12MB
.gitignore 103B
logging.h 16KB
utils.h 5KB
yololayer.h 4KB
cuda_utils.h 417B
yolov5.hpp 15KB
common.hpp 15KB
zedcam.hpp 1KB
LapDepth_demo.ipynb 317KB
I-80_Eastshore_Fwy.jpg 304KB
kitti_demo.jpg 197KB
output_onnx.jpg 191KB
detect_onnx.jpg 176KB
nyu_demo.jpg 115KB
adb4871d-4d063244.jpg 84KB
7dd9ef45-f197db95.jpg 82KB
cars-vehicles-traffic-640x427.jpg 78KB
8e1c1ab0-a8b92173.jpg 77KB
test.jpg 66KB
0ace96c3-48481887.jpg 66KB
9aa94005-ff1d4c9a.jpg 43KB
3c0e7240-96e390d2.jpg 36KB
da_onnx.jpg 23KB
ll_onnx.jpg 17KB
out_kitti_demo.jpg 14KB
out_kitti_demo.jpg 14KB
out_kitti_demo.jpg 14KB
out_I-80_Eastshore_Fwy.jpg 10KB
out_nyu_demo.jpg 9KB
out_I-80_Eastshore_Fwy.jpg 9KB
out_cars-vehicles-traffic-640x427.jpg 9KB
out_nyu_demo.jpg 7KB
LICENSE 34KB
LICENSE 1KB
sync_project_frames_multi_threads.m 3KB
splits.mat 3KB
README.md 15KB
README _CH.md 12KB
README.md 9KB
blank.mp4 5KB
yolop-640-640.onnx 34.24MB
yolop-1280-1280.onnx 30.58MB
yolop-320-320.onnx 30.34MB
ll.png 695KB
detect.png 687KB
da.png 662KB
FIG1.png 659KB
yolop.png 147KB
img.png 59KB
model.py 54KB
function.py 23KB
YOLOP.py 23KB
general.py 19KB
light.py 19KB
main.py 19KB
train.py 16KB
ui_main.py 15KB
main_ui.py 15KB
eval.py 15KB
transform_list.py 14KB
common2.py 12KB
evaluate.py 11KB
common.py 11KB
augmentations.py 11KB
trainer.py 10KB
AutoDriveDataset.py 10KB
DepthDetect.py 9KB
loss.py 9KB
postprocess.py 8KB
export_onnx.py 8KB
demo2.py 8KB
train.py 8KB
datasets_list.py 8KB
demo.py 7KB
DemoDataset.py 7KB
demo.py 7KB
plot.py 6KB
utils.py 6KB
utils.py 6KB
test_onnx.py 6KB
autoanchor.py 5KB
test.py 5KB
option.py 5KB
default.py 5KB
extract_official_train_test_set_from_mat.py 4KB
calculate_error.py 4KB
gen_bdd_seglabel.py 3KB
hust.py 3KB
bdd.py 3KB
logger.py 3KB
activations.py 2KB
共 123 条
- 1
- 2
资源评论
妄北y
- 粉丝: 1w+
- 资源: 1万+
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功