GeoYolo-SLAM/MaskRCNN_ROS/include/MaskRCNN/samples/demo.ipynb

297 lines
1.5 MiB
Plaintext
Raw Normal View History

2025-04-09 16:05:54 +08:00
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Mask R-CNN Demo\n",
"\n",
"A quick intro to using the pre-trained model to detect and segment objects."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Using TensorFlow backend.\n"
]
}
],
"source": [
"import os\n",
"import sys\n",
"import random\n",
"import math\n",
"import numpy as np\n",
"import skimage.io\n",
"import matplotlib\n",
"import matplotlib.pyplot as plt\n",
"\n",
"# Root directory of the project\n",
"ROOT_DIR = os.path.abspath(\"../\")\n",
"\n",
"# Import Mask RCNN\n",
"sys.path.append(ROOT_DIR) # To find local version of the library\n",
"from mrcnn import utils\n",
"import mrcnn.model as modellib\n",
"from mrcnn import visualize\n",
"# Import COCO config\n",
"sys.path.append(os.path.join(ROOT_DIR, \"samples/coco/\")) # To find local version\n",
"import coco\n",
"\n",
"%matplotlib inline \n",
"\n",
"# Directory to save logs and trained model\n",
"MODEL_DIR = os.path.join(ROOT_DIR, \"logs\")\n",
"\n",
"# Local path to trained weights file\n",
"COCO_MODEL_PATH = os.path.join(ROOT_DIR, \"mask_rcnn_coco.h5\")\n",
"# Download COCO trained weights from Releases if needed\n",
"if not os.path.exists(COCO_MODEL_PATH):\n",
" utils.download_trained_weights(COCO_MODEL_PATH)\n",
"\n",
"# Directory of images to run detection on\n",
"IMAGE_DIR = os.path.join(ROOT_DIR, \"images\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Configurations\n",
"\n",
"We'll be using a model trained on the MS-COCO dataset. The configurations of this model are in the ```CocoConfig``` class in ```coco.py```.\n",
"\n",
"For inferencing, modify the configurations a bit to fit the task. To do so, sub-class the ```CocoConfig``` class and override the attributes you need to change."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Configurations:\n",
"BACKBONE_SHAPES [[256 256]\n",
" [128 128]\n",
" [ 64 64]\n",
" [ 32 32]\n",
" [ 16 16]]\n",
"BACKBONE_STRIDES [4, 8, 16, 32, 64]\n",
"BATCH_SIZE 1\n",
"BBOX_STD_DEV [ 0.1 0.1 0.2 0.2]\n",
"DETECTION_MAX_INSTANCES 100\n",
"DETECTION_MIN_CONFIDENCE 0.5\n",
"DETECTION_NMS_THRESHOLD 0.3\n",
"GPU_COUNT 1\n",
"IMAGES_PER_GPU 1\n",
"IMAGE_MAX_DIM 1024\n",
"IMAGE_MIN_DIM 800\n",
"IMAGE_PADDING True\n",
"IMAGE_SHAPE [1024 1024 3]\n",
"LEARNING_MOMENTUM 0.9\n",
"LEARNING_RATE 0.002\n",
"MASK_POOL_SIZE 14\n",
"MASK_SHAPE [28, 28]\n",
"MAX_GT_INSTANCES 100\n",
"MEAN_PIXEL [ 123.7 116.8 103.9]\n",
"MINI_MASK_SHAPE (56, 56)\n",
"NAME coco\n",
"NUM_CLASSES 81\n",
"POOL_SIZE 7\n",
"POST_NMS_ROIS_INFERENCE 1000\n",
"POST_NMS_ROIS_TRAINING 2000\n",
"ROI_POSITIVE_RATIO 0.33\n",
"RPN_ANCHOR_RATIOS [0.5, 1, 2]\n",
"RPN_ANCHOR_SCALES (32, 64, 128, 256, 512)\n",
"RPN_ANCHOR_STRIDE 2\n",
"RPN_BBOX_STD_DEV [ 0.1 0.1 0.2 0.2]\n",
"RPN_TRAIN_ANCHORS_PER_IMAGE 256\n",
"STEPS_PER_EPOCH 1000\n",
"TRAIN_ROIS_PER_IMAGE 128\n",
"USE_MINI_MASK True\n",
"USE_RPN_ROIS True\n",
"VALIDATION_STEPS 50\n",
"WEIGHT_DECAY 0.0001\n",
"\n",
"\n"
]
}
],
"source": [
"class InferenceConfig(coco.CocoConfig):\n",
" # Set batch size to 1 since we'll be running inference on\n",
" # one image at a time. Batch size = GPU_COUNT * IMAGES_PER_GPU\n",
" GPU_COUNT = 1\n",
" IMAGES_PER_GPU = 1\n",
"\n",
"config = InferenceConfig()\n",
"config.display()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create Model and Load Trained Weights"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true,
"scrolled": false
},
"outputs": [],
"source": [
"# Create model object in inference mode.\n",
"model = modellib.MaskRCNN(mode=\"inference\", model_dir=MODEL_DIR, config=config)\n",
"\n",
"# Load weights trained on MS-COCO\n",
"model.load_weights(COCO_MODEL_PATH, by_name=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Class Names\n",
"\n",
"The model classifies objects and returns class IDs, which are integer value that identify each class. Some datasets assign integer values to their classes and some don't. For example, in the MS-COCO dataset, the 'person' class is 1 and 'teddy bear' is 88. The IDs are often sequential, but not always. The COCO dataset, for example, has classes associated with class IDs 70 and 72, but not 71.\n",
"\n",
"To improve consistency, and to support training on data from multiple sources at the same time, our ```Dataset``` class assigns it's own sequential integer IDs to each class. For example, if you load the COCO dataset using our ```Dataset``` class, the 'person' class would get class ID = 1 (just like COCO) and the 'teddy bear' class is 78 (different from COCO). Keep that in mind when mapping class IDs to class names.\n",
"\n",
"To get the list of class names, you'd load the dataset and then use the ```class_names``` property like this.\n",
"```\n",
"# Load COCO dataset\n",
"dataset = coco.CocoDataset()\n",
"dataset.load_coco(COCO_DIR, \"train\")\n",
"dataset.prepare()\n",
"\n",
"# Print class names\n",
"print(dataset.class_names)\n",
"```\n",
"\n",
"We don't want to require you to download the COCO dataset just to run this demo, so we're including the list of class names below. The index of the class name in the list represent its ID (first class is 0, second is 1, third is 2, ...etc.)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# COCO Class names\n",
"# Index of the class in the list is its ID. For example, to get ID of\n",
"# the teddy bear class, use: class_names.index('teddy bear')\n",
"class_names = ['BG', 'person', 'bicycle', 'car', 'motorcycle', 'airplane',\n",
" 'bus', 'train', 'truck', 'boat', 'traffic light',\n",
" 'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird',\n",
" 'cat', 'dog', 'horse', 'sheep', 'cow', 'elephant', 'bear',\n",
" 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie',\n",
" 'suitcase', 'frisbee', 'skis', 'snowboard', 'sports ball',\n",
" 'kite', 'baseball bat', 'baseball glove', 'skateboard',\n",
" 'surfboard', 'tennis racket', 'bottle', 'wine glass', 'cup',\n",
" 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',\n",
" 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza',\n",
" 'donut', 'cake', 'chair', 'couch', 'potted plant', 'bed',\n",
" 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote',\n",
" 'keyboard', 'cell phone', 'microwave', 'oven', 'toaster',\n",
" 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors',\n",
" 'teddy bear', 'hair drier', 'toothbrush']"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Run Object Detection"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Processing 1 images\n",
"image shape: (476, 640, 3) min: 0.00000 max: 255.00000\n",
"molded_images shape: (1, 1024, 1024, 3) min: -123.70000 max: 120.30000\n",
"image_metas shape: (1, 89) min: 0.00000 max: 1024.00000\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA6IAAAK7CAYAAADoVjJgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvLmOJdmWpvft2YZzfIgp885d7AaKoNQCFb4JJb5CvUIJ\nFAiCMiXyKfgE1KiREgkCXWDVrTtlZkS4+zl2zGxPa1PYp7KUFgjiVlYV2j4ggIBHhLuFnT2stf5/\nLdVa4+Dg4ODg4ODg4ODg4ODgp0L/cz/AwcHBwcHBwcHBwcHBwX9aHInowcHBwcHBwcHBwcHBwU/K\nkYgeHBwcHBwcHBwcHBwc/KQciejBwcHBwcHBwcHBwcHBT8qRiB4cHBwcHBwcHBwcHBz8pByJ6MHB\nwcHBwcHBwcHBwcFPypGIHhwcHBwcHBwcHBwcHPykHInowcHBwcHBwcHBwcHBwU/KkYgeHBwcHBwc\nHBwcHBwc/KQciejBwcHBwcHBwcHBwcHBT8qRiB4cHBwcHBwcHBwcHBz8pByJ6MHBwcHBwcHBwcHB\nwcFPypGIHhwcHBwcHBwcHBwcHPykHInowcHBwcHBwcHBwcHBwU/KkYgeHBwcHBwcHBwcHBwc/KQc\niejBwcHBwcHBwcHBwcHBT8qRiB4cHBwcHBwcHBwcHBz8pNh/7gf4/8pf/vtfOOCXTQupbOSyM58G\ntIHWhForNQvzOKJoxLjys5/9muWaeXm5scdKLULJhZoLJWcezw9Ihrhm0OC8wRgFWsCC9QZlFLUJ\nKSVU1Rg0qMa+b3z69JFhHPnd73+HVHDO430AQHJBadBeUShIBY3GYvHakWtBW8M4DqQtUwtIEUqu\nxLSidMNYhTYaAaz1+ODRtlFKopZMa4IxBqM0zlqMMexrxBhLyYW3ywVjFN57nPNopam1YoxBa01K\niSoV6z0uDEDFOEWVTE6JeTqRt8K2JPatEGxgTzvWOr79+DOaZETdwC6YOfLw4USujRjhPH1A9E5w\njWnQeFtBwS1mvl421h1adujqMFVjnaFKptHw3rFuO2Gc8CFwu12ZToFhcljjKJvh+rqwXDZyEqbx\nRBgMSidErbz/+UTTG5rM2Y9YOyACe8xc14UGWO8IYcQYz7bl/n79wOX6ikjBWIVzFhFBJINkDJVx\n8qRtpqYZ5zyfP3/HMMDjk2eYYLktbHtj2zSXt8rz8zuMtsQ9YzB4a7BaIVJ5eXlhns5M05mG4ruv\n36EtWKfY4gLA8/MHgp/5u7/9HVVgHEceHyemeSPliNaKEBzSCtZanHNoY3h7ewPAajAUnAucHx4Z\nhpF937lc35jGwPk8Ercbjw8zJUfWdeHd+w9c1oXaGmEauMWFbdvQ2nE+PfH1y4XaNMZarBLiuvIw\nnwlu5I9/+IJzI3bwaK/Y68btrVJSxTnN8/Mj1+uVUirjcCLGSGsNpcEYMFbRWkEpIXiDSMU5x+A9\nSKM1hXcDTTT/4f/+Pd/8/B1PzwFtC9p6Lq+Ft6+Z20UoCZQohEomk/ZIcBOmDbx8vvHx2yfC3Mjt\nytvtggsPzPbEYwh8+gWU3FgukRQj//6//Dm//91nfvu3V+I28/HTwMPTTBgtqB07gDEaqZrrJQGO\nt5dXbrdXfvObX4JLNFXQWnGaB75+/UIpkcfzCas8NWtyhFI0YTBcr6+s641xHJjHEVqjlr53lTLk\nmMkp4jzM84SzjloEbRz7nikZtA1oPSDNoq1hmByvX1+hgMVRE0gzoBXYBraw5gtZCmiHcwPejsQt\n8vLyhXEMTFNApPD29sovf/lLNAaqMJ9GrssrRkMIHmjknNFKE5wHIFiHlMSergwDoAq1VnKqvLwu\nWDcSwkxr4BWIZLJEakv4MRCmEes8OWdCsKS8s+0Ljx/e87d/8wNff1hxduI0PxDTSmuZ0zwQY6Nm\njdIKPwoaxTgMaKX5/vsf+PTx52g9cLnsfPfdKx8//RyU5uuXL2gaT6dHjHYs64adKmZK4BK1ZQb/\njLIVaTutwjSc+fr5lR++/4GHOfCrbz9RYuG73//Adi1888snnj7ONK34w+9ubDs0BcYVBv/Idm2s\n1xvSFh4ff0Yujdu6gzKMk6PITpGV56eJGCMKzTBMAGitqbWwx5UqiRBGrB6IO1xeG2k3lKyQmhkm\nx/kx4KfGdfse6wthtDhr2fcEYpCsqEnwziMFcu6fl7GK0+mENY5liUzjmdoglUyrmdNceXqG80mI\n+wXvZ7T2NARUojRHUx606u/UeqxzSKu0Wpknj7eG9XqlaTDWojBsa0IphXcG7zVQeH56ZJ4mROCH\n779DRBMTvF52Ymkoo3BeYV1jjysgWGvxPhBjRgrUorgtO8MwMAwe5zWtZobBYaxQyw7NkndFTZrW\nAlUM2x7Z4o5Q8d5yOk94b3l5/cw0nkmpcb1uAJwfR5wNpKi5XTXf/92C5Mr7b0f8GHGjpim43f+P\nkPCh8v7DzOgHtlvi+rZRq+b9h3eEoKmyYyys60qtDaM9697vMtDEPeHsQI6Vfc2kPaMxOOPRThP9\nDWOE0zjweD4zDxMvP7yxXDbiXlmuN8ZpYnwYsHNDqx570BrBBbQKLOtOSoXT6cT1diEMjmEKXF4v\nfPvpG86nmSaFHz7/CeM0SmukQRgnbq+vUCoPD++4LDf84BknD6pwW6/kBIiFaqhVY0zD+YoyCWmZ\n4APTcCYnMEqhAEEoWEpS7GvltiS0NTRbabagfUNrhdEa1TR5hXHODCNorXj5kjBqZBpn5jlQa+b1\nZWXfMsboH/fZMDhODyNvly9cXhdq1rx//gZkQGGBRm071b7REFobsNZhXcYqBcWz3lawD4gvVPd3\nfHj3Dt8mWqpI/oFffPsLgnskRni5/UCNGkWP3cr9vm8N1m3nu+9eMDowjQ8M84lbWjC2n8WD92xr\nZV0qZTc4NfCnP/6eYQi8e/fEum9s+5Wn94bf/LsHPn0auS2VkhrOGqxN7Fvk9XPly+8gTAYhIjph\nB810GilSiSlhUDw+PtJEs62CMZbL2xvTOPHp0ydeX78S84YfDI+PM2/XN7QF4+7vFsPD+EBg4PX7\nK9/9/isxNYwbwDgu5YpxmvN8Zh4GXj5/5vr2Rs6ZYTyRiqIpzzid+PYDTJNljxu//fu/Z5zPnE7P\nODuyLJHvvvuecZ54/+Edy+2VJhqtAjTP5XKjZqFVaFVBBaMcRntQhh++/oAdLONpoFIAQaHQQK2V\nMQwoHcl5IW6G0T1h8OSUqbLSTAatUNoxzc+8vXzHvlw4j8+YNqGVQmswznJdIjFHMI3zu4G3t1fi\nntDWEsYRpRTWaLyzSMmc5wlvLfu2c9kKqUScU7x7egIsyxJZlx2nHWHQPL8/8e7dGSTyhz9+j9KW\n9x8+cl2vaCM0Vcg149xILYpaiP/r//J//eHPmd/9q0lEgV8Bf/PP/RD/v1FA++d+iIODfwLUf+Rr\n/ymu9f/Yezj48/EvcU0dn/k/Gf8SP+6Dn44jZDo4+BfJ/wb8V3/Ob2j++q//+s/5/f7J+O/+h//2\nOaX8VynuAIQh4IO7K1YNrXqlr+RCE2EYPfu+k2vqlaFR883PnplPHm0EmiBS0WiMdeQSAUFZjQ+e\ncRwwxlJrY9si+5aQ3FBorHUopSm1su07pVQeHx8JIdBa62pPCDhn0VYzn2eMNkip5FSoWX5UTq/X\nBSlCrV3VlVqQVtEGjDMYp6kitNaQ1qDJj8/gnMNoAw3uf0QThXWeh/MD3/7sW6y1SO1qitaacRr7\nc+n+/1QKrNM4B9L6u7PGMQ4nFJZtTUiFaZrxLqAdhNnw8GyZHgqnd8L5vWF6HsBatg1uqyJHR0y3\nrmxZhzUWbS2pNK63RIwNq3rFXjVNzpWYCjkXpLZ75bQhUgjBo4ygTUOpQsoXhjBgdaAkyDkzny2P\n7zwPz55KpFSB5tAtEHfIRVPFgjhKUcTYWG+F5ZrRDIAhpZ1t20CprjyhuF6uGG0Z/EDwHoVCJJHz\njWV5pUlDo1EonLWEwaN
"text/plain": [
"<matplotlib.figure.Figure at 0x7fe822eae630>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Load a random image from the images folder\n",
"file_names = next(os.walk(IMAGE_DIR))[2]\n",
"image = skimage.io.imread(os.path.join(IMAGE_DIR, random.choice(file_names)))\n",
"\n",
"# Run detection\n",
"results = model.detect([image], verbose=1)\n",
"\n",
"# Visualize results\n",
"r = results[0]\n",
"visualize.display_instances(image, r['rois'], r['masks'], r['class_ids'], \n",
" class_names, r['scores'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}