Skip to content

Commit

Permalink
Release 0.8.1 (HumanSignal#472)
Browse files Browse the repository at this point in the history
* Massive LSF rework (HumanSignal#403)

* LSF with new deserialization and changed structure

* Update LSF js

* Hostname for enternal LS usage. Docker with hostname passthrough.

* Username and password to docker.

* update s3 signature version to s3v4 to support more aws regions (HumanSignal#402)

* Release 0.7.4.post0 (HumanSignal#400)

* Release 0.7.4.post0 label-studio-converter version.

* Docs and version.

* update s3 signature version to s3v4 to support more aws regions

Co-authored-by: Max <[email protected]>

* Update ml.md (HumanSignal#404)

* Release 0.7.4.post0 (HumanSignal#400)

* Release 0.7.4.post0 label-studio-converter version.

* Docs and version.

* Update ml.md

* Update ml.md

Co-authored-by: Max <[email protected]>

* Logs for ML backend.

* LSF build.

* Ui fixes.

* Fixes and addions for task validation in cloud storages.

* Check for raise.

* Back raise.

* fix un-error for ML backend, add parse_config docstring

* Move project loading line.

* methods

* Bugfixes with project_obj and docs.

* New LSF build.

* Improvements of navigation with hotkeys.

* Minimum formatting of completed_at (HumanSignal#397)

* Minimum formatting of completed_at

* Fix skip formatting None timestamp

* Doc fixes.

* Fix order of completion picking

* Docs fixes.

* Pretty fixes.

* New LSF build.

* Mobile UI fix.

* More mobile fixes.

* Min height for tr in tasks.

* Docs fixes.

* Usage stats docs.

* Docs fixes.

* Docs about frontend.

* Docs frontend fixes.

* Docs fixes.

* Docs for LS modules.

* Docs fixes.

* LSF build from tmp/deserialize.

* New LSF build.

* LSF build from tmp/deserialize.

* New LSF build from tmp/deserialize.

* maintain ordering in source.json and task.json (HumanSignal#414)

Co-authored-by: dhruv <[email protected]>

* Docs preview in iframe.

* New LSF build from tmp/deserialize.

* Loader for open preview.

* Keypoint docs fix with stroke width.

* Docs fix with ml-backend-url.

* Active learning docs in ml backend.

* ML examples docs.

* New LSF build from tmp/deserialize.

* Docs CORS new image added.

* Latest LSF tmp/deserialize build

* Fix LSF bug with initialization

* Fix headshot export dimmer

* Reduce debug logs on next task

* Remove huge data from console while debug

* Hide skip button on tasks page

* Add table example with text fields

* New LSF build from tmp/deserialize

* Move external fonts to LS. Bug fix with accordion and checkboxes.

* Change version 0.7.5rc1

* configure blueprint

* enable PR builds for all target branches

* ML patch

* Update docs

* Latest LSF build with tmp/deserialization

* Fix buttons "Start Predictions" and "Remove ML backend" for Firefox (HumanSignal#439)

* Fix correct templates_dir from config.json

* Fix deprecated deploy to azure button

* Litte fix with deploy buttons

* Fix docs

* Show task code column

* UI fixes

* Reworked size of left and right panels on tasks page

* Fix size of right panel. rc3 release

* Make left and right panels true responsible

* Add call to ml backend to get task prediction on api/task/<id>

* Add Table type to _DATA_TYPES

* Add Table example data

* Add playground examples

* Lates LSF build with Text/HyperText fixes

* Add host & protocol & port help for external domain

* Upgrade pip in docker

* Update tag docs

* Change version to rc5

* Fix docs with init without project name

* Make short version for task and completion paths on tasks page

* Increase wait intervals in e2e tests

* Sync platform with opensource in TS previews

* Make tooltip for paths on tasks page.

* Make pretty print for show task data

* Fix displying of short paths for storages

* Fix tests

* Fix e2e tests

* Fix api_predict (HumanSignal#480)

analytics instance is in globals, not in project

* Fix stats

* Rename exception treatment

* Refactor project/config

* Refactor some api urls. Update docs about how to make predictions

* Fix docs

* Refactor blueprint.py

* Rework ml backend apis

* Move start_browser. Update LSF build

* Fix ascending order

* Import urls without SSL check

* Merge with master

* Add ML backend for Image Object Detection

* Restructure docs

* Revert labeling

* Fix one link

* Enhance ML docs, make version checker

* Use id in export

* Add workaround for python3.5

* Update LSF c883b7368

* Add deprecated API

* Check integration with blueprint. Fix external hostname without port

* Fix tests

* Update LSF 57cbe6

* Add blueprint usage example

* Fix mmdetection doc

* Delete duplicated text

* Add help text to blueprint example

* Fix misspells

* Add readme about ls integration

* Add more readme to blueprint

* Add more readme into blueprint usage example

* Fix redirects on welcome and setup pages

* Remove unused comment

* Fix ml.md from Michael

* stylistic changes and a few typos

Co-authored-by: hlomzik <[email protected]>
Co-authored-by: Wouter Devriendt <[email protected]>
Co-authored-by: jeannefukumaru <[email protected]>
Co-authored-by: nik <[email protected]>
Co-authored-by: Hikonobu Kurihara <[email protected]>
Co-authored-by: Dhruv Mishra <[email protected]>
Co-authored-by: dhruv <[email protected]>
Co-authored-by: Thomas Grainger <[email protected]>
Co-authored-by: Dario Saccavino <[email protected]>
Co-authored-by: hepaajan <[email protected]>
Co-authored-by: Mikhail Maluyk <[email protected]>
  • Loading branch information
12 people authored Nov 20, 2020
1 parent cf0269e commit 4688a36
Showing 76 changed files with 2,682 additions and 1,531 deletions.
4 changes: 1 addition & 3 deletions .github/workflows/master.yaml
Original file line number Diff line number Diff line change
@@ -4,8 +4,6 @@ on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
types: [opened, synchronize, reopened]

jobs:
build:
@@ -28,7 +26,7 @@ jobs:
pip install -e .
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
if [ -f label_studio/tests/requirements.txt ]; then pip install -r label_studio/tests/requirements.txt; fi
- name: init project
- name: Init project
run: |
label-studio init my_project
- name: Test with pytest
1 change: 1 addition & 0 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
@@ -9,6 +9,7 @@ on:
- '*'
- '*/*'
- '!master'
pull_request:

jobs:
build:
18 changes: 14 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -105,19 +105,21 @@ If you want to build a local image, run:
docker build -t heartexlabs/label-studio:latest .
```

## Using docker-compose
## Run docker-compose

You can also start serving at `http://localhost:8080` using docker-compose.

### First time to run the app
**First time to run the app**
```bash
INIT_COMMAND='--init' docker-compose up -d
```
### Run the app with existing project data

**Run the app with existing project data**
```bash
docker-compose up -d
```
### Run the app reseting project data

**Run the app reseting project data**
```bash
INIT_COMMAND='--init --force' docker-compose up -d
```
@@ -173,6 +175,9 @@ The list of supported use cases for data annotation. Please contribute your own
| **Comparison** | |
| Pairwise | Comparing entities in pairs to judge which of each entity is preferred |
| Ranking | Sort items in the list according to some property |
| **Time Series** | |
| Classification | |
| Segmentation | |

## Machine Learning Integration

@@ -187,6 +192,11 @@ That gives you the opportunities to use:
- **Active Learning**: Perform labeling in active learning mode - select only most complex examples
- **Prediction Service**: Instantly create running production-ready prediction service

## Label Studio Integration to your services

* You can use [Label Studio Frontend](https://github.com/heartexlabs/label-studio-frontend) as a separated React library, [read more here](https://labelstud.io/guide/frontend.html).
* Label Studio Backend (this repo) can be integrated to your app via Flask Blueprints. [See example of integration here](https://github.com/heartexlabs/label-studio/blob/master/blueprint_usage_example.py).

## Label Studio for Teams, Startups, and Enterprises :office:

Label Studio for Teams is our enterprise edition (cloud & on-prem), that includes a data manager, high-quality baseline models, active learning, collaborators support, and more. Please visit the [website](https://www.heartex.ai/) to learn more.
43 changes: 43 additions & 0 deletions blueprint_usage_example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
""" This is an example about how Label Studio could be included into your flask app as blueprint
1. You need to initialize LS project in the same directory where this file is placed:
> label-studio init my_project
2. Run this demo flask app within Label Studio python environment:
> python blueprint_usage_example.py
3. Go to http://localhost:5000/
"""
import json
from types import SimpleNamespace
from flask import Flask, make_response, g
from label_studio.blueprint import (blueprint as label_studio_blueprint,
LabelStudioConfig, set_external_hostname, project_get_or_create)

app = Flask('my-ml-platform', static_url_path='')
app.secret_key = 'some-secret-key'
app.config['WTF_CSRF_ENABLED'] = False
app.url_map.strict_slashes = False # it's very important to disable this option
app.register_blueprint(label_studio_blueprint, url_prefix='/label-studio/')

# check label_studio.utils.argparser to know all options: *_parser.add_argument(option_name, ...)
input_args = {'project_name': 'my_project', 'command': 'start', 'root_dir': '.', 'force': False}
set_external_hostname('http://localhost:5000/label-studio')
app.label_studio = LabelStudioConfig(input_args=SimpleNamespace(**input_args))


@app.route('/')
def index():
# get label studio project instance
project = project_get_or_create()
# get all tasks with completions from target storage and print them (target_storage ~ BaseStorage)
tasks_with_completions = [task for task in project.target_storage.items()]
output = json.dumps(tasks_with_completions, indent=2)
return make_response('Welcome to our ML platform with '
'<a href="/label-studio">Label Studio Project "' + project.name + '"</a>' +
'<br><br>Task data from project: '
'<pre style="width:500px">' + output + '</pre>', 200)


if __name__ == '__main__':
app.run(debug=True)
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
@@ -8,6 +8,6 @@ services:
volumes:
- ./my_project:/label-studio/my_project
command: "label-studio start my_project ${INIT_COMMAND} --host 0.0.0.0"
ports:
ports:
- "8080:8080"
restart: always
2 changes: 1 addition & 1 deletion docs/source/blog/release-050.md
Original file line number Diff line number Diff line change
@@ -141,4 +141,4 @@ Now Label Studio is also maintained and distributed as Docker container - [run o

### Multisession mode

You can launch Label Studio in [multisession mode](/guide/#Multisession-mode) - then each browser session dynamically creates its own project.
You can launch Label Studio in [multisession mode](/guide/multiuser/#Multisession-mode) - then each browser session dynamically creates its own project.
97 changes: 80 additions & 17 deletions docs/source/guide/FAQ.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: FAQ
type: guide
order: 101
order: 106
---

Frequently asked questions about setup and usage.
@@ -12,11 +12,11 @@ The most common mistake while resource loading is <b>CORS</b> (Cross-Origin Reso

<br>
<center>
<img src='../images/cors-lsf-error.png' style="max-width:300px; opacity: 0.8">
<img src='../images/cors-lsf-error.png' style="max-width:300px; width: 100%; opacity: 0.8">
<br/><br/>
<img src='/images/cors-error.png' style="max-width:500px; opacity: 0.8">
<img src='/images/cors-error.png' style="max-width:500px; width: 100%; opacity: 0.8">
<br/><br/>
<img src='/images/cors-error-2.png' style="max-width:500px; opacity: 0.8">
<img src='/images/cors-error-2.png' style="max-width:500px; width: 100%; opacity: 0.8">
</center>

* If you have an access to the hosting server as admin then you need to allow CORS for the web server. For nginx you can try to add <a href="javascript:void(0)" onclick="$('#nginx-cors-code').toggle()">these lines</a> to `/etc/nginx/nginx.conf` into your `location` section:
@@ -63,29 +63,92 @@ The most common mistake while resource loading is <b>CORS</b> (Cross-Origin Reso

<br/>

## How to make pre-annotations & pre-labeling
You can import pre-annotated tasks into LS. Pre-annotations will be automatically shown on Labeling page. Prepare your tasks with `predictions` field which is very similar to `completions` and then import your tasks to LS. [Read more](tasks.html#Basic-format) about task format and predictions.
## How to make pre-annotations & pre-labeling & predictions
You can import pre-annotated tasks into LS. Pre-annotations will be automatically shown on Labeling page. Prepare your tasks with `predictions` field which is very similar to `completions` and then import your tasks to LS. [Read more](tasks.html#Basic-format) about task format and predictions. The same format of predictions is used for the ML backend output.
<br>

<center><img src="../images/completions-predictions-scheme.png" style="width: 100%; max-width: 481px; opacity: 0.9"></center>

> Check completion format on Setup page or on Tasks page at `</>` (Show task data) button. Then make `result` field in your prediction is similar to the completion.
> You need to use different ids within any task elements, completions, predictions and thier `result` items. It's our LSF requirement.
Let's use the following labeling config:

```xml
<View>
<Choices name="choice" toName="image" showInLine="true">
<Choice value="Boeing" background="blue"/>
<Choice value="Airbus" background="green" />
</Choices>

<RectangleLabels name="label" toName="image">
<Label value="Airplane" background="green"/>
<Label value="Car" background="blue"/>
</RectangleLabels>

<Image name="image" value="$image"/>
</View>
```

After the project setup is finished you can import this task (just copy this right into the input field on the Import page):

```json
[{
{
"data": {
"my_text": "Opossum is great"
"image": "http://localhost:8080/static/samples/sample.jpg"
},

"predictions": [{
"result": [{
"from_name": "sentiment_class",
"to_name": "message",
"type": "choices",
"value": {
"choices": ["Positive"]
"result": [
{
"id": "result1",
"type": "rectanglelabels",
"from_name": "label", "to_name": "image",
"original_width": 600, "original_height": 403,
"image_rotation": 0,
"value": {
"rotation": 0,
"x": 4.98, "y": 12.82,
"width": 32.52, "height": 44.91,
"rectanglelabels": ["Airplane"]
}
},
{
"id": "result2",
"type": "rectanglelabels",
"from_name": "label", "to_name": "image",
"original_width": 600, "original_height": 403,
"image_rotation": 0,
"value": {
"rotation": 0,
"x": 75.47, "y": 82.33,
"width": 5.74, "height": 7.40,
"rectanglelabels": ["Car"]
}
},
{
"id": "result3",
"type": "choices",
"from_name": "choice", "to_name": "image",
"value": {
"choices": ["Airbus"]
}
}
}]
]
}]
}]
}
```

In this example there are 3 results inside of 1 prediction:
* `result1` - the first bounding box
* `result2` - the second bounding box
* `result3` - choice selection

And the result will look as the following:

<center><img src="../images/predictions-loaded.jpg" style="width: 100%; max-width: 700px"></center>

## How to display labels on bounding boxes, polygons and other regions
<center>
<img src='../images/lsf-settings.png'>
@@ -109,7 +172,7 @@ LSB web server always use `0.0.0.0` address for start. But if you really need to
</center>


## What units are x, y, width and height in results for images?
## What units are x, y, width and height for image results?

x, y, width and height are in percents of image dimensions.

47 changes: 47 additions & 0 deletions docs/source/guide/concept.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
title: Main concept & design
type: guide
order: 704
---

## Overview

Label Studio is a self-contained Web application for multi-typed data labeling and exploration. The _backend_ is written in pure Python powered by [Flask](https://github.com/pallets/flask). The _frontend_ part is a backend-agnostic [React](https://reactjs.org/) + [MST](https://github.com/mobxjs/mobx-state-tree) app, included as a precompiled script.

Here are the main concepts behind Label Studio's workflow:

<div style="margin:auto; text-align:center; width:100%"><img src="/images/label-studio-ov.jpg" style="opacity: 0.7"/></div>

- **Tasks** represent an individual dataset items. Label Studio is a multi-type labeling tool - you can [import](tasks.html) either text, image, audio URL, HTML text or any number and combination of these data resources.
- **Completions** are the labeling results in [JSON format](export.html#Completion-fields). They could be [exported](export.html) in various common formats, ready to use in machine learning pipelines.
- **Predictions** are the optional labeling results in [the same format](export.html#Completion-fields), but unlike completions they are used for generating pre-labeling during the annotation process, or validating the model predictions.
- [**Machine learning backend** connects](ml.html) popular machine learning frameworks to Label Studio for active learning & generating model predictions on-the-fly.
- **Labeling config** is a simple [XML tree with **tags**](setup.html#Labeling-config) used to configure UI elements, connect input data to output labeling scheme.
- **Project** encompasses tasks, config, predictions and completions all-in-one in an isolated directory
- **Frontend Labeling UI** is accessible from any browser, distributed as precompiled js/css scripts and could be [easily extendable with new labeling tags](frontend.html). You can also [embed Label Studio UI into your applications](frontend.html#Quickstart).


### Main modules

The main modules of LS are
* [Label Studio Backend](https://github.com/heartexlabs/label-studio/) (LSB, main repository)
* [Label Studio Frontend](https://github.com/heartexlabs/label-studio-frontend) (LSF, editor)
* [Machine Learning Backends](https://github.com/heartexlabs/label-studio/tree/master/label_studio/ml) (MLB)

<br>
<div style="margin:auto; text-align:center;"><img src="/images/ls-modules-scheme.png" style="opacity: 0.8"/></div>

### Relations among tasks, completions and results

Here you can see relations among labeling objects: tasks, completions, results, etc.

One user provides one completion, it’s atomic, and it consists of the result items. Result items can have relations between themselves with the specified direction of three types: left-right, right-left, or bidirectional. Normalizations are additional information in the custom string format about the current result item.

<br>
<center><img src="/images/labeling-scheme.png" style="width: 100%; opacity: 0.6"></center>
<br>
Completions and Predictions are very similar. But predictions must be generated automatically by ML models.

**Usage statistics**

Label Studio collects anonymous usage statistics without any sensitive info about page request number and data types from the labeling config. It helps us to improve the labeling quality and gives a better understanding about the next development.
37 changes: 37 additions & 0 deletions docs/source/guide/deployment.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
title: Deployment scenarios
type: guide
order: 907
---


## WSGIServer instead of Flask

Use `--use-gevent` option on start to enable WSGI server. It wraps around app.run with gevent's WSGIServer to enable the server to better handle concurrent requests.

```
label-studio start test --use-gevent
```

## HTTPS & SSL

You can enable https protocol for Flask or WSGIServer. You need to generate SSL certificate and key for it, e.g.:

```
openssl req -newkey rsa:2048 -nodes -keyout key.pem -x509 -days 365 -out certificate.pem
```

Than you need to use `--cert` and `--key` option on start:

```
label-studio start test --cert certificate.pem --key key.pem
```


## Health check

LS has a special endpoint for health checks:

```
/api/health
```
2 changes: 1 addition & 1 deletion docs/source/guide/export.md
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@ type: guide
order: 105
---

Your annotations are stored in [raw completion format](#Completion-format) inside `my_project_name/completions` directory, one file per labeled task named as `task_id.json`.
Your annotations are stored in [raw completion format](#Export-formats) inside `my_project_name/completions` directory, one file per labeled task named as `task_id.json`.

You can optionally convert and export raw completions to a more common format by doing one of the following:

2 changes: 1 addition & 1 deletion docs/source/guide/frontend.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Frontend library
type: guide
order: 705
order: 706
---

Frontend, as its name suggests, is the frontend library called [«Label Studio Frontend»](https://github.com/heartexlabs/label-studio-frontend) (**LSF**) based on React and mobx-state-tree, distributed as an NPM package. You can include it in your applications without «Label Studio Backend» (**LSB**) part and provide data annotation support to your users. It can be customized and extended.
Loading

0 comments on commit 4688a36

Please sign in to comment.