# Mask R-CNN for Object Detection and Segmentation
This is an implementation of [Mask R-CNN](https://arxiv.org/abs/1703.06870) on Python 3, Keras, and TensorFlow. The model generates bounding boxes and segmentation masks for each instance of an object in the image. It's based on Feature Pyramid Network (FPN) and a ResNet101 backbone.
![Instance Segmentation Sample](assets/street.png)
The repository includes:
* Source code of Mask R-CNN built on FPN and ResNet101.
* Training code for MS COCO
* Pre-trained weights for MS COCO
* Jupyter notebooks to visualize the detection pipeline at every step
* ParallelModel class for multi-GPU training
* Evaluation on MS COCO metrics (AP)
* Example of training on your own dataset
The code is documented and designed to be easy to extend. If you use it in your research, please consider referencing this repository. If you work on 3D vision, you might find our recently released [Matterport3D](https://matterport.com/blog/2017/09/20/announcing-matterport3d-research-dataset/) dataset useful as well.
This dataset was created from 3D-reconstructed spaces captured by our customers who agreed to make them publicly available for academic use. You can see more examples [here](https://matterport.com/gallery/).
# Getting Started
* [demo.ipynb](samples/demo.ipynb) Is the easiest way to start. It shows an example of using a model pre-trained on MS COCO to segment objects in your own images.
It includes code to run object detection and instance segmentation on arbitrary images.
* [train_shapes.ipynb](samples/shapes/train_shapes.ipynb) shows how to train Mask R-CNN on your own dataset. This notebook introduces a toy dataset (Shapes) to demonstrate training on a new dataset.
* ([model.py](mrcnn/model.py), [utils.py](mrcnn/utils.py), [config.py](mrcnn/config.py)): These files contain the main Mask RCNN implementation.
* [inspect_data.ipynb](samples/coco/inspect_data.ipynb). This notebook visualizes the different pre-processing steps
to prepare the training data.
* [inspect_model.ipynb](samples/coco/inspect_model.ipynb) This notebook goes in depth into the steps performed to detect and segment objects. It provides visualizations of every step of the pipeline.
* [inspect_weights.ipynb](samples/coco/inspect_weights.ipynb)
This notebooks inspects the weights of a trained model and looks for anomalies and odd patterns.
# Step by Step Detection
To help with debugging and understanding the model, there are 3 notebooks
([inspect_data.ipynb](samples/inspect_data.ipynb), [inspect_model.ipynb](samples/inspect_model.ipynb),
[inspect_weights.ipynb](samples/inspect_weights.ipynb)) that provide a lot of visualizations and allow running the model step by step to inspect the output at each point. Here are a few examples:
## 1. Anchor sorting and filtering
Visualizes every step of the first stage Region Proposal Network and displays positive and negative anchors along with anchor box refinement.
![](assets/detection_anchors.png)
## 2. Bounding Box Refinement
This is an example of final detection boxes (dotted lines) and the refinement applied to them (solid lines) in the second stage.
![](assets/detection_refinement.png)
## 3. Mask Generation
Examples of generated masks. These then get scaled and placed on the image in the right location.
![](assets/detection_masks.png)
## 4.Layer activations
Often it's useful to inspect the activations at different layers to look for signs of trouble (all zeros or random noise).
![](assets/detection_activations.png)
## 5. Weight Histograms
Another useful debugging tool is to inspect the weight histograms. These are included in the inspect_weights.ipynb notebook.
![](assets/detection_histograms.png)
## 6. Logging to TensorBoard
TensorBoard is another great debugging and visualization tool. The model is configured to log losses and save weights at the end of every epoch.
![](assets/detection_tensorboard.png)
## 6. Composing the different pieces into a final result
![](assets/detection_final.png)
# Training on MS COCO
We're providing pre-trained weights for MS COCO to make it easier to start. You can
use those weights as a starting point to train your own variation on the network.
Training and evaluation code is in `samples/coco/coco.py`. You can import this
module in Jupyter notebook (see the provided notebooks for examples) or you
can run it directly from the command line as such:
```
# Train a new model starting from pre-trained COCO weights
python3 samples/coco/coco.py train --dataset=/path/to/coco/ --model=coco
# Train a new model starting from ImageNet weights
python3 samples/coco/coco.py train --dataset=/path/to/coco/ --model=imagenet
# Continue training a model that you had trained earlier
python3 samples/coco/coco.py train --dataset=/path/to/coco/ --model=/path/to/weights.h5
# Continue training the last model you trained. This will find
# the last trained weights in the model directory.
python3 samples/coco/coco.py train --dataset=/path/to/coco/ --model=last
```
You can also run the COCO evaluation code with:
```
# Run COCO evaluation on the last trained model
python3 samples/coco/coco.py evaluate --dataset=/path/to/coco/ --model=last
```
The training schedule, learning rate, and other parameters should be set in `samples/coco/coco.py`.
# Training on Your Own Dataset
Start by reading this [blog post about the balloon color splash sample](https://engineering.matterport.com/splash-of-color-instance-segmentation-with-mask-r-cnn-and-tensorflow-7c761e238b46). It covers the process starting from annotating images to training to using the results in a sample application.
In summary, to train the model on your own dataset you'll need to extend two classes:
```Config```
This class contains the default configuration. Subclass it and modify the attributes you need to change.
```Dataset```
This class provides a consistent way to work with any dataset.
It allows you to use new datasets for training without having to change
the code of the model. It also supports loading multiple datasets at the
same time, which is useful if the objects you want to detect are not
all available in one dataset.
See examples in `samples/shapes/train_shapes.ipynb`, `samples/coco/coco.py`, `samples/balloon/balloon.py`, and `samples/nucleus/nucleus.py`.
## Differences from the Official Paper
This implementation follows the Mask RCNN paper for the most part, but there are a few cases where we deviated in favor of code simplicity and generalization. These are some of the differences we're aware of. If you encounter other differences, please do let us know.
* **Image Resizing:** To support training multiple images per batch we resize all images to the same size. For example, 1024x1024px on MS COCO. We preserve the aspect ratio, so if an image is not square we pad it with zeros. In the paper the resizing is done such that the smallest side is 800px and the largest is trimmed at 1000px.
* **Bounding Boxes**: Some datasets provide bounding boxes and some provide masks only. To support training on multiple datasets we opted to ignore the bounding boxes that come with the dataset and generate them on the fly instead. We pick the smallest box that encapsulates all the pixels of the mask as the bounding box. This simplifies the implementation and also makes it easy to apply image augmentations that would otherwise be harder to apply to bounding boxes, such as image rotation.
To validate this approach, we compared our computed bounding boxes to those provided by the COCO dataset.
We found that ~2% of bounding boxes differed by 1px or more, ~0.05% differed by 5px or more,
and only 0.01% differed by 10px or more.
* **Learning Rate:** The paper uses a learning rate of 0.02, but we found that to be
too high, and often causes the weights to explode, especially when using a small batch
size. It might be related to differences between how Caffe and TensorFlow compute
gradients (sum vs mean across batches and
没有合适的资源?快使用搜索试试~ 我知道了~
Mask RCNN源代码
共511个文件
png:230个
jpg:66个
txt:58个
5星 · 超过95%的资源 需积分: 49 686 下载量 73 浏览量
2018-07-13
21:05:19
上传
评论 23
收藏 49.35MB RAR 举报
温馨提示
Mask RCNN源代码,附有数据集,可以直接在windows中运行
资源推荐
资源详情
资源评论
收起资源包目录
Mask RCNN源代码 (511个子文件)
setup.cfg 99B
.gitignore 569B
MANIFEST.in 58B
workspace.ini 415B
vcs.ini 92B
encoding.ini 64B
codestyle.ini 62B
inspect_balloon_model.ipynb 9.99MB
inspect_model.ipynb 9.84MB
inspect_data.ipynb 7.84MB
inspect_balloon_data.ipynb 7.8MB
inspect_nucleus_model.ipynb 6.59MB
inspect_nucleus_data.ipynb 4.21MB
demo.ipynb 1.48MB
inspect_weights.ipynb 1.21MB
train_shapes.ipynb 99KB
9.jpg 32KB
9.jpg 32KB
23.jpg 31KB
19.jpg 30KB
45.jpg 29KB
35.jpg 29KB
37.jpg 28KB
34.jpg 28KB
44.jpg 28KB
49.jpg 28KB
39.jpg 28KB
36.jpg 28KB
31.jpg 28KB
41.jpg 28KB
30.jpg 27KB
26.jpg 27KB
43.jpg 27KB
50.jpg 27KB
20.jpg 27KB
22.jpg 27KB
42.jpg 27KB
32.jpg 27KB
16.jpg 27KB
40.jpg 27KB
21.jpg 27KB
38.jpg 26KB
27.jpg 26KB
33.jpg 26KB
25.jpg 26KB
17.jpg 26KB
48.jpg 26KB
47.jpg 26KB
51.jpg 26KB
46.jpg 26KB
52.jpg 26KB
7.jpg 26KB
7.jpg 26KB
12.jpg 26KB
18.jpg 25KB
13.jpg 25KB
2.jpg 25KB
2.jpg 25KB
10.jpg 25KB
10.jpg 25KB
1.jpg 25KB
1.jpg 25KB
24.jpg 25KB
56.jpg 24KB
28.jpg 24KB
29.jpg 24KB
53.jpg 24KB
6.jpg 24KB
6.jpg 24KB
4.jpg 23KB
4.jpg 23KB
11.jpg 23KB
15.jpg 23KB
54.jpg 23KB
5.jpg 22KB
5.jpg 22KB
55.jpg 22KB
14.jpg 22KB
8.jpg 22KB
8.jpg 22KB
3.jpg 22KB
3.jpg 22KB
9.json 44KB
23.json 42KB
19.json 41KB
45.json 40KB
35.json 39KB
37.json 38KB
34.json 38KB
44.json 38KB
49.json 38KB
39.json 38KB
36.json 38KB
31.json 38KB
41.json 38KB
30.json 37KB
26.json 37KB
43.json 37KB
50.json 37KB
20.json 37KB
共 511 条
- 1
- 2
- 3
- 4
- 5
- 6
TomHardy
- 粉丝: 2838
- 资源: 37
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功
- 1
- 2
- 3
- 4
- 5
- 6
前往页