# ros2-tensorflow
Use Tensorflow to load pretrained neural networks and perform inference through ROS2 interfaces.
<img src="/data/detection.png" alt="Rviz2 detection output" width="50%" height="50%"/>
The output can be directly visualized through Rviz
## Requirements
In order to build the `ros2-tensorflow` package, the following dependencies are needed
Required dependencies:
- [ROS2 Foxy](https://docs.ros.org/en/foxy/Installation.html)
Rosdep dependencies:
- [OpenCV Python](https://pypi.org/project/opencv-python/)
- [Tensorflow](https://www.tensorflow.org/install/)
- [Vision Msgs](https://github.com/Kukanani/vision_msgs)
Optional dependencies:
- [Tensorflow Object Detection Models](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/installation.md) for object detection tasks
- [TensorFlow-Slim](https://github.com/google-research/tf-slim) for object segmentation tasks
The provided Dockerfile contains an Ubuntu 18.04 environment with all the dependencies and this package already installed.
To use the Dockerfile:
$ git clone https://github.com/alsora/ros2-tensorflow.git
$ cd ros2-tensorflow/docker
$ bash build.sh
$ bash run.sh
## Build
This section describes how to build the `ros2-tensorflow` package and the required depenencies in case you are not using the provided Dockerfile.
Get the source code and create the ROS 2 workspace
$ git clone https://github.com/alsora/ros2-tensorflow.git $HOME/ros2-tensorflow
$ mkdir -p $HOME/tf_ws/src
$ cd $HOME/tf_ws
$ ln -s $HOME/ros2-tensorflow/ros2-tensorflow src
Install required dependencies using rosdep
$ rosdep install --from-paths src --ignore-src --rosdistro foxy -y
Install the Tensorflow Object Detection Models (optional).
Make sure to specify the correct Python version according to your system.
$ sudo apt-get install -y protobuf-compiler python-lxml python-tk
$ pip install --user Cython contextlib2 jupyter matplotlib Pillow
$ git clone https://github.com/tensorflow/models.git /usr/local/lib/python3.8/dist-packages/tensorflow/models
$ cd usr/local/lib/python3.8/dist-packages/tensorflow/models/research
$ protoc object_detection/protos/*.proto --python_out=.
$
$ echo 'export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python3.8/dist-packages/tensorflow/models/research' >> $HOME/.bashrc
Install Tensorflow Slim (optional)
$ pip install tf_slim
Build and install the `ros2-tensorflow` package
$ colcon build
$ source install/local_setup.sh
## Usage
The basic usage consists in creating a ROS 2 node which loads a Tensorflow model and another ROS 2 node that acts as a client and receives the result of the inference.
It is possible to specify which model a node should load.
Note that if the model is specified via url, as it is by default, the first time the node is executed a network connection will be required in order to download the model.
#### Object Detection Task
Test the object detection server by running in separate terminals
$ ros2 run tf_detection_py server
$ ros2 run tf_detection_py client_test
Setup a real object detection pipeline using a stream of images coming from a ROS 2 camera node
$ rviz2
$ ros2 run tf_detection_py server
$ ros2 run image_tools cam2image --ros-args -p frequency:=2.0
#### Image Classification Task
Test the image classification server by running in separate terminals
$ ros2 run tf_classification_py server
$ ros2 run tf_classification_py client_test
## Loading different models
The repository contains convenient APIs for loading Tensorflow models into the ROS 2 nodes.
Models are defined using the `ModelDescriptor` class, which contains all the information required for loading a model and performing inference on it.
It can either contain a path where the model can be found on the machine or an URL where the model can be downloaded the first time.
Different model formats are also supported, such as frozen models and saved models.
Some known supported models are already present as examples.
See [classification models](ros2-tensorflow/tf_classification_py/tf_classification_py/models.py) and [detection models](ros2-tensorflow/tf_detection_py/tf_detection_py/models.py)
The [Tensorflow models repository](https://github.com/tensorflow/models) contains many pretrained models that can be used.
For example, you can get additional Tensorflow model for object detection from the [detection model zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md#coco-trained-models).
没有合适的资源?快使用搜索试试~ 我知道了~
ros2-tensorflow:用于Tensorflow中的计算机视觉任务的ROS2节点
共66个文件
py:34个
xml:5个
rst:5个
需积分: 35 3 下载量 87 浏览量
2021-05-04
13:05:14
上传
评论
收藏 182KB ZIP 举报
温馨提示
ros2-tensorflow 使用Tensorflow加载预训练的神经网络并通过ROS2接口执行推理。 输出可以通过Rviz直接可视化 要求 为了构建ros2-tensorflow软件包,需要以下依赖项 所需的依赖项: Rosdep依赖项: 可选依赖项: 用于对象检测任务的 用于对象分割任务 提供的Dockerfile包含一个具有所有依赖项的Ubuntu 18.04环境,并且该软件包已安装。 要使用Dockerfile: $ git clone https://github.com/alsora/ros2-tensorflow.git $ cd ros2-tensorflow/docker $ bash build.sh $ bash run.sh 建造 本节介绍了如何构建ros2-tensorflow软件包以及在不使用提供的Dockerfile的情况下所需的依赖关系。 获
资源详情
资源评论
资源推荐
收起资源包目录
ros2-tensorflow-master.zip (66个子文件)
ros2-tensorflow-master
.gitignore 1KB
README.md 5KB
LICENSE 11KB
ros2-tensorflow
tf_detection_py
setup.cfg 99B
tf_detection_py
models.py 2KB
examples
client_test.py 2KB
server.py 999B
face_detection.py 2KB
__init__.py 0B
detection_node.py 9KB
__init__.py 0B
detection_models.py 6KB
test
test_copyright.py 871B
test_pep257.py 884B
__init__.py 0B
test_flake8.py 851B
package.xml 1008B
CHANGELOG.rst 2KB
resource
tf_detection_py 0B
setup.py 1KB
ros2_tensorflow
CMakeLists.txt 179B
package.xml 823B
CHANGELOG.rst 223B
tf_interfaces
CMakeLists.txt 688B
package.xml 973B
CHANGELOG.rst 444B
srv
ImageDetection.srv 67B
ImageClassification.srv 71B
ros2_tf_core
setup.cfg 92B
test
test_copyright.py 871B
ros2_tf_core
__init__,py 0B
test_pep257.py 884B
__init__.py 0B
test_flake8.py 851B
ros2_tf_core
tensorflow_node.py 1KB
models.py 9KB
img_conversion.py 2KB
__init__.py 0B
qos.py 921B
package.xml 872B
CHANGELOG.rst 1KB
resource
ros2_tf_core 0B
setup.py 1KB
tf_classification_py
setup.cfg 109B
test
test_copyright.py 871B
test_pep257.py 884B
__init__.py 0B
test_flake8.py 851B
package.xml 1019B
CHANGELOG.rst 1KB
tf_classification_py
models.py 1KB
classification_node.py 4KB
examples
client_test.py 2KB
server.py 1KB
__init__.py 0B
__init__.py 0B
resource
tf_classification_py 0B
setup.py 1KB
CONTRIBUTING.md 633B
docker
Dockerfile 4KB
build.sh 277B
run.sh 585B
scripts
coco
Makefile 248B
label_map_util.patch 400B
build_opencv.sh 320B
data
detection.png 129KB
共 66 条
- 1
蓝精神
- 粉丝: 26
- 资源: 4721
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功
评论0