This is a simple GUI-based Widget based on matplotlib in Python to facilitate quick and efficient crowd-sourced generation of annotation masks and bounding boxes using a simple interactive User Interface. Annotation can be in terms of polygon points covering all parts of an object (see instructions in README) or it can simply be a bounding box, for which you click and drag the mouse button. Optionally, one could choose to use a pretrained Mask RCNN model to come up with initial segmentations. This shifts the work load from painstakingly annotating all the objects in every image to altering wrong predictions made by the system which maybe simpler once an efficient model is learnt.
Python 3.5+ is required to run the Mask RCNN code. If only the GUI tool is used, Python2.7 or Python3.5+ can be used.
Before running the code, install required pre-requisite python packages using pip.
cat requirements.txt | xargs -n 1 -L 1 pip install
Clone the repo.
git clone https://github.com/Deep-Magic/COCO-Style-Dataset-Generator-GUI.git
cd COCO-Style-Dataset-Generator-GUI/
python3 segment.py -i images/ -c classes/products.txt
First download the Mask RCNN repo in the same parent directory of COCO-Style-Dataset-Generator-GUI/.
git clone https://github.com/Deep-Magic/Mask_RCNN.git
To run the particular model for the demo, download the pretrained weights from HERE!!!. Download and extract pretrained_weights/ into the repository.
Then, in a separate text file, list the target labels/classes line-by-line to be displayed along with the dataset for class labels. For example, look at classes/products.txt
python3 segment.py -i images -f -p ../Mask_RCNN/ -w pretrained_weights/imagenet_10/mask_rcnn_bags_0006.h5 -c classes/products.txt
USAGE: segment.py [-h] -i IMAGE_DIR [-f] [-p MASKRCNN_DIR] [-w WEIGHTS_PATH] [-c CLASS_FILE]
| Shorthand | Flag Name | Description |
|---|---|---|
| -h | --help | Show this help message and exit |
| -i IMAGE_DIR | --image_dir IMAGE_DIR | Path to the image dir |
| -c CLASS_FILE | --class_file CLASS_FILE | Path to object labels |
| -f | --feedback | Whether or not to include AI feedback |
| -p MASKRCNN_DIR | --maskrcnn_dir MASKRCNN_DIR | Path to Mask RCNN Repo |
| -w WEIGHTS_PATH | --weights_path WEIGHTS_PATH | Path to Mask RCNN checkpoint save file |
In this demo, all the green patches over the objects are the rough masks generated by a pretrained Mask RCNN network.
Key-bindings/ Buttons
EDIT MODE (when a is pressed and polygon is being edited)
'a' toggle vertex markers on and off. When vertex markers are on, you can move them, delete them
'd' delete the vertex under point
'i' insert a vertex at point near the boundary of the polygon.
Left click Use on any point on the polygon boundary and move around by dragging to alter shape of polygon
REGULAR MODE
Scroll Up Zoom into image
Scroll Down Zoom out of image
Left Click Create a point for a polygon mask around an object
Right Click Complete the polygon currently formed by connecting all selected points
Left Click Drag Create a bounding box rectangle from point 1 to point 2 (works only when there are no polygon points on screen for particular object)
'a' Press key on top of overlayed polygon (from Mask RCNN or previous annotations) to select it for editing
'r' Press key on top of overlayed polygon (from Mask RCNN or previous annotations) to completely remove it
BRING PREVIOUS ANNOTATIONS Bring back the annotations from the previous image to preserve similar annotations.
SUBMIT To be clicked after Right click completes polygon! Finalizes current segmentation mask and class label picked. After this, the polygon cannot be edited.
NEXT Save all annotations created for current file and move on to next image.
PREV Goto previous image to re-annotate it. This deletes the annotations created for the file before the current one in order to rewrite the fresh annotations.
RESET If when drawing the polygon using points, the polygon doesn't cover the object properly, reset will let you start fresh with the current polygon. This deletes all the points on the image.
The green annotation boxes from the network can be edited by pressing on the Keyboard key a when the mouse pointer is on top of a particular such mask. Once you press a, the points making up that polygon will show up and you can then edit it using the key bindings specified. Because of the nature of the polygon points being so close to each other, it's easier to delete all the cluttered points at some place of deformation and simply dragging points around to make the image. I suggest not using the insert vertex option i as deleting and dragging is just faster. Once you're done editing the polygon, press a again to finalize the edits. At this point, it will become possible to submit that particular annotation and move on to the next one.
FILE FUNCTIONALITY
cut_objects.py Cuts objects based on bounding box annotations using dataset.json file and creates occlusion-based augmented images dataset.
create_json_file.py Takes a directory of annotated images (use segment.py to annotate into text files) and returns a COCO-style JSON file.
extract_frames.py Takes a directory of videos and extracts all the frames of all videos into a folder labeled adequately by the video name.
pascal_to_coco.py Takes a PASCAL-style dataset directory with JPEGImages/ and Annotations/ folders and uses the bounding box as masks to create a COCO-style JSON file.
segment.py Read the instructions above.
segment_bbox_only.py Same functionality but optimized for easier annotation of bbox-only type of datasets.
test_*.py Unit tests.
visualize_dataset.py Visualize the annotations created using the tool.
visualize_json_file.py Visualize the dataset JSON file annotations on the entire dataset.
compute_dataset_statistics.py Find distribution of objects in the dataset by counts.
combine_json_files.py Combine different JSON files together into a single dataset file.
delete_images.py Delete necessary images from the JSON dataset.
NOTE: Please use python <FILENAME>.py -h for details on how to use each of the above files.