jupyter-innotater

Annotate data including image bounding boxes inline within your Jupyter notebook in Python. Innotater's flexible API allows easy selection of interactive controls to suit your datasets exactly.

Now works with JupyterLab (2.0+ recommended)

1 - Overview

In a data science or machine learning project, you may prepare and study images or other data within a Jupyter notebook then need to annotate the data to augment the training or fix errors in your source data.

Since you are already working within a Jupyter notebook, the Innotater works inline allowing you to interact with your data and annotate it quickly and easily, syncing straight back to your input data arrays or matrices.

Within Jupyter, you can easily home in on problem input data - perhaps only misclassified images - so you can step through and adjust bounding boxes just for those items.

The Innotater widget is designed with a flexible API making it quick and easy to get started exploring your dataset, guessing how to work with your data without explicit configuration where possible.

The project is currently in ALPHA development phase, and I appreciate all feedback on any problems including details on how the current code works or fails to work for the structure of your particular projects.

Full documentation is now on ReadTheDocs

2 - Examples

You can easily combine Innotater's interactive components to suit your project. Here are some examples.

Images and Bounding Boxes

Load some images from filenames in an array, initialise empty bounding boxes.

Then set up Innotater to display the images so you can draw updated bounding boxes directly.

from jupyter_innotater import *
import numpy as np, os

images = os.listdir('./foods/')
targets = np.zeros((len(images), 4)) # Initialise bounding boxes as x,y = 0,0, width,height = 0,0

Innotater( ImageInnotation(images, path='./foods'), BoundingBoxInnotation(targets) )

The widget allows you to interactively draw bounding boxes for any of the images, and the targets variable is always kept in sync with your changes.

Advance to the next image by clicking 'Next' or pressing 'n' on the keyboard (provided the Innotater has focus).

import pandas as pd
df = pd.DataFrame(targets, columns=['x','y','w','h'])
df.to_csv('./bounding_boxes.csv', index=False)

The above saves your work - the bounding boxes you've drawn - as a CSV file. Without saving, your numbers will be lost if the kernel restarts.

Multi-Classification of Text Strings

Load some text strings e.g. of movie reviews

Set up Innotater to display the reviews so you can mark their sentiment as positive, negative, or neutral.

Use indexes parameter to display only the reviews we want to check.

from jupyter_innotater import *

reviews = ['I really liked this movie', 'It was OK', 'Do not watch!', 'Ignore me']
sentiments = [1] * len(reviews)
sentiment_classes = ['0 - Positive', '1 - Neutral', '2 - Negative']

Innotater(
    TextInnotation(reviews), 
    MultiClassInnotation(sentiments, classes=sentiment_classes),
    indexes=[0,1,2]
)

The widget allows you to interactively step through the reviews selecting the classification, and the sentiments variable is always kept in sync.

Multiple Bounding Boxes and Multi-classification

For each photo, draw multiple bounding boxes and specify for each bounding box what is depicted (from a dropdown).

There is also an 'Exclude' checkbox per photo so we can select if that photo should be excluded from the dataset - you would write some code to remove those before full processing later on.

Also display the filename in a textbox, for information only.

from jupyter_innotater import *
import numpy as np, os

animalfns = sorted(os.listdir('./animals/'))

repeats = 4

# Per-photo data
targets_exclude = np.zeros((len(animalfns), 1), dtype='int') # Binary flag to indicate want to exclude from dataset

# Repeats within each photo
breeds = ['Cat - Shorthair tabby', 'Cat - Shorthair ginger', 'Dog - Labrador', 'Dog - Beagle', 'Dog - Terrier']
targets_breed = np.zeros((len(animalfns), len(breeds)), dtype='int')
targets_bboxes = np.zeros((len(animalfns), repeats, 4), dtype='int') # (x,y,w,h) for each animal

Innotater(
    [
        ImageInnotation(animalfns, path='./animals', width=370, height=280), # Display the image itself
        TextInnotation(animalfns, multiline=False) # Display the image filename
    ],
    [
        BinaryClassInnotation(targets_exclude, name='Exclude'), # Checkbox
        RepeatInnotation(
            (BoundingBoxInnotation, targets_bboxes), # Individual animal bounding box
            (MultiClassInnotation, targets_breed,
                {'name':'Breed', 'classes':breeds, 'dropdown':True}), # Per-animal breed dropdown
            max_repeats=repeats, min_repeats=1
        )
    ]
)

Jupyter Notebook Examples

The notebooks in the Example folder contain many examples you can run directly in Jupyter notebook. You can try them out for free in a Binder environment by clicking here: Innotater on Binder

3 - Installation

Install from PyPi (recommended)

JupyterLab (version 3.0+ required):

pip install jupyter_innotater

4 - Contact for Feedback

Please get in touch with any feedback or questions: [email protected]. It will be especially useful to understand the structure of your project and what is needed to augment your data - e.g. extra shape types. There are many ideas on the roadmap, and your input is vital for prioritising these.

5 - License

This code is released under an MIT license.

6 - Documentation

Full documentation is now on ReadTheDocs:

• Stable version as installed through pip
• Latest version in development