# Semantic-Mono-Depth
![image](images/SemanticMonoDepth.PNG)
## Abstract
Depth estimation from a single image represents a very exciting challenge in computer vision. While other image-based depth sensing techniques leverage on the geometry between different viewpoints(e.g., stereo or structure from motion), the lack of these cues within a single image renders ill-posed the monocular depth estimation task. For inference, state-of-the-art encoder-decoder architectures for monocular depth estimation rely on effective feature representations learned at training time. For unsupervised training of these models, geometry has been effectively exploited by suitable images warping losses computed from views acquired by a stereo rig or a moving camera. In this paper, we make a further step forward showing that learning semantic information from images enables to improve effectively monocular depth estimation as well. In particular, by leveraging on semantically labeled images together with unsupervised signals gained by geometry through an image warping loss, we propose a deep learning approach aimed at joint semantic segmentation and depth estimation. Our overall learning framework is semi-supervised, as we deploy groundtruth data only in the semantic domain. At training time, our network learns a common feature representation for both tasks and a novel cross-task loss function is proposed. The experimental findings show how, jointly tackling depth prediction and semantic segmentation, allows to improve depth estimation accuracy. In particular, on the KITTI dataset our network outperforms state-of-the-art methods for monocular depth estimation.
## Requirements
* `Tensorflow 1.5 or higher` (recomended)
* `python packages` such as opencv, matplotlib
## Download pretrain models
Checkpoints can be downloaded from [here](https://drive.google.com/open?id=1n4qPzso_uyodgevi3w0qCXduTsPXqlub)
## Inference and evaluation
```
python monodepth_main.py --dataset kitti --mode test --data_path $DATA_PATH --output_dir $OUTPUT_DIR --filename ./utils/filenames/kitti_semantic_stereo_2015_test_split.txt --task depth --checkpoint_path $checkpoint_path --encoder $ENCODER
python ./utils/evaluate_kitti.py --split kitti_test --predicted_disp_path $OUTPUT_DIR/disparities_pp.npy --gt_path $DATA_PATH
```
DATA_PATH=`path_to_dataset`
OUTPUT_DIR=`path_to_output_folder`
ENCODER=`vgg` or `resnet`
没有合适的资源?快使用搜索试试~ 我知道了~
资源推荐
资源详情
资源评论
收起资源包目录
三维_基于深度学习实现的单目立体视觉深度估计算法实现_优质项目实战.zip (15个子文件)
三维_基于深度学习实现的单目立体视觉深度估计算法实现_优质项目实战
utils.py 9KB
monodepth_model.py 24KB
monodepth_dataloader.py 7KB
bilinear_sampler.py 4KB
utils
evaluation_utils.py 7KB
filenames
kitti_semantic_stereo_2015_train_split.txt 15KB
cityscapes_semantic_train_files.txt 424KB
kitti_semantic_stereo_2015_test_split.txt 4KB
shuffler.py 488B
evaluate_kitti.py 6KB
visualize_semantic.py 7KB
monodepth_main.py 14KB
images
SemanticMonoDepth.PNG 1.4MB
README.md 2KB
average_gradients.py 2KB
共 15 条
- 1
资源评论
极智视界
- 粉丝: 2w+
- 资源: 1459
下载权益
C知道特权
VIP文章
课程特权
开通VIP
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功