AI 图像识别和视频分析

# ImageAI : Object Detection ## --------------------------------------------------- ## Introducing TheiaEngine.  We the creators of ImageAI are glad to announce [TheiaEngine](https://www.genxr.co/theia-engine), the next-generation computer Vision AI API capable of all computer vision tasks in a single API call and available via REST API to all programming languages. Features include - **Detect 300+ objects** ( 220 more objects than ImageAI) - **Provide answers to any content or context questions** asked on an image - very useful to get information on any object, action or information without needing to train a new custom model for every tasks - **Generate scene description and summary** - **Convert 2D image to 3D pointcloud and triangular mesh** - **Semantic Scene mapping of objects, walls, floors, etc** - **Stateless Face recognition and emotion detection** - **Image generation and augmentation from prompt** - etc. Visit [https://www.genxr.co/theia-engine](https://www.genxr.co/theia-engine) to try the demo and join in the beta testing today. ## --------------------------------------------------- ### TABLE OF CONTENTS - <a href="#firstdetection" > :white_square_button: First Object Detection</a> - <a href="#objectextraction" > :white_square_button: Object Detection, Extraction and Fine-tune</a> - <a href="#customdetection" > :white_square_button: Custom Object Detection</a> - <a href="#detectionspeed" > :white_square_button: Detection Speed</a> - <a href="#hidingdetails" > :white_square_button: Hiding/Showing Object Name and Probability</a> - <a href="#inputoutputtype" > :white_square_button: Image Input & Output Types</a> - <a href="#documentation" > :white_square_button: Documentation</a> ImageAI provides very convenient and powerful methods to perform object detection on images and extract each object from the image. The object detection class supports RetinaNet, YOLOv3 and TinyYOLOv3. To start performing object detection, you must download the RetinaNet, YOLOv3 or TinyYOLOv3 object detection model via the links below: * **[RetinaNet](https://github.com/OlafenwaMoses/ImageAI/releases/download/3.0.0-pretrained/retinanet_resnet50_fpn_coco-eeacb38b.pth)** _(Size = 130 mb, high performance and accuracy, with longer detection time)_ * **[YOLOv3](https://github.com/OlafenwaMoses/ImageAI/releases/download/3.0.0-pretrained/yolov3.pt)** _(Size = 237 mb, moderate performance and accuracy, with a moderate detection time)_ * **[TinyYOLOv3](https://github.com/OlafenwaMoses/ImageAI/releases/download/3.0.0-pretrained/tiny-yolov3.pt)** _(Size = 34 mb, optimized for speed and moderate performance, with fast detection time)_ Once you download the object detection model file, you should copy the model file to the your project folder where your .py files will be. Then create a python file and give it a name; an example is FirstObjectDetection.py. Then write the code below into the python file: ### FirstObjectDetection.py <div id="firstdetection" ></div> ```python from imageai.Detection import ObjectDetection import os execution_path = os.getcwd() detector = ObjectDetection() detector.setModelTypeAsYOLOv3() detector.setModelPath( os.path.join(execution_path , "yolov3.pt")) detector.loadModel() detections = detector.detectObjectsFromImage(input_image=os.path.join(execution_path , "image2.jpg"), output_image_path=os.path.join(execution_path , "image2new.jpg"), minimum_percentage_probability=30) for eachObject in detections: print(eachObject["name"] , " : ", eachObject["percentage_probability"], " : ", eachObject["box_points"] ) print("--------------------------------") ``` Sample Result: Input Image  Output Image  ``` laptop : 87.32235431671143 : (306, 238, 390, 284) -------------------------------- laptop : 96.86298966407776 : (121, 209, 258, 293) -------------------------------- laptop : 98.6301600933075 : (279, 321, 401, 425) -------------------------------- laptop : 99.78572130203247 : (451, 204, 579, 285) -------------------------------- bed : 94.02391314506531 : (23, 205, 708, 553) -------------------------------- apple : 48.03136885166168 : (527, 343, 557, 364) -------------------------------- cup : 34.09906327724457 : (462, 347, 496, 379) -------------------------------- cup : 44.65090036392212 : (582, 342, 618, 386) -------------------------------- person : 57.70219564437866 : (27, 311, 341, 437) -------------------------------- person : 85.26121377944946 : (304, 173, 387, 253) -------------------------------- person : 96.33603692054749 : (415, 130, 538, 266) -------------------------------- person : 96.95255160331726 : (174, 108, 278, 269) -------------------------------- ``` Let us make a breakdown of the object detection code that we used above. ```python from imageai.Detection import ObjectDetection import os execution_path = os.getcwd() ``` In the 3 lines above , we import the **ImageAI object detection** class in the first line, import the `os` in the second line and obtained the path to folder where our python file runs. ```python detector = ObjectDetection() detector.setModelTypeAsYOLOv3() detector.setModelPath( os.path.join(execution_path , "yolov3.pt")) detector.loadModel() ``` In the 4 lines above, we created a new instance of the `ObjectDetection` class in the first line, set the model type to YOLOv3 in the second line, set the model path to the YOLOv3 model file we downloaded and copied to the python file folder in the third line and load the model in the fourth line. ```python detections = detector.detectObjectsFromImage(input_image=os.path.join(execution_path , "image2.jpg"), output_image_path=os.path.join(execution_path , "image2new.jpg")) for eachObject in detections: print(eachObject["name"] , " : ", eachObject["percentage_probability"], " : ", eachObject["box_points"] ) print("--------------------------------") ``` In the 2 lines above, we ran the `detectObjectsFromImage()` function and parse in the path to our image, and the path to the new image which the function will save. Then the function returns an array of dictionaries with each dictionary corresponding to the number of objects detected in the image. Each dictionary has the properties `name` (name of the object), `percentage_probability` (percentage probability of the detection) and `box_points` (the x1,y1,x2 and y2 coordinates of the bounding box of the object). Should you want to use the RetinaNet which is appropriate for high-performance and high-accuracy demanding detection tasks, you will download the RetinaNet model file from the links above, copy it to your python file's folder, set the model type and model path in your python code as seen below: ```python detector = ObjectDetection() detector.setModelTypeAsRetinaNet() detector.setModelPath( os.path.join(execution_path , "retinanet_resnet50_fpn_coco-eeacb38b.pth")) detector.loadModel() ``` However, if you desire TinyYOLOv3 which is optimized for speed and embedded devices, you will download the TinyYOLOv3 model file from the links above, copy it to your python file's folder, set the model type and model path in your python code as seen below: ```python detector = ObjectDetection() detector.setModelTypeAsTinyYOLOv3() detector.setModelPath( os.path.join(execution_path , "tiny-yolov3.pt")) detector.loadModel() ``` ## Object Detection, Extraction and Fine-tune <div id="objectextraction" ></div> In the examples we used above, we ran the object detection on an image and it returned the detected objects in an array as well as save a new image with rectangular markers drawn on each object. In our next examples, we will be able to extract each object from the input image and save it independently. In the example code below which is very identical to the previous object