Skip to content

Commit

Permalink
fix: OPTIC-391: Tasks rendered incomplete by low agreement should not…
Browse files Browse the repository at this point in the history
… count on project cards (HumanSignal#5684)

### PR fulfills these requirements
- [X] Commit message(s) and PR title follows the format
`[fix|feat|ci|chore|doc]: TICKET-ID: Short description of change made`
ex. `fix: DEV-XXXX: Removed inconsistent code usage causing intermittent
errors`
- [X] Tests for the changes have been added/updated (for bug
fixes/features)
- [ ] Docs have been added/updated (for bug fixes/features)
- [X] Best efforts were made to ensure docs/code are concise and
coherent (checked for spelling/grammatical errors, commented out code,
debug logs etc.)
- [X] Self-reviewed and ran all changes on a local instance (for bug
fixes/features)



#### Change has impacts in these area(s)
_(check all that apply)_
- [ ] Product design
- [ ] Backend (Database)
- [X] Backend (API)
- [ ] Frontend


### Describe the reason for change
#### What
Given an enterprise user is utilizing the Low Agreement strategy to
assign tasks to annotators, this was not correctly reflecting in the
project cards for Admin/Manager/Owner often having a higher number
reported than it should be.

#### Why
The total completed tasks were not accounting for the low agreement
aspect, and reporting just the base raw `is_labeled` of the project
based tasks.

#### How
By refactoring the ProjectManager to be extensible, this allowed the
annotated fields to be replaced at a granular level. In this case only
`finished_task_number` required a differing calculation. The
implementation of which can be seen in the parent PR to this one, along
with tests.
  • Loading branch information
bmartel authored Apr 10, 2024
1 parent 6b28a6d commit 1b21dc8
Showing 1 changed file with 15 additions and 13 deletions.
28 changes: 15 additions & 13 deletions label_studio/projects/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,6 @@


class ProjectManager(models.Manager):
def for_user(self, user):
return self.filter(organization=user.active_organization)

COUNTER_FIELDS = [
'task_number',
'finished_task_number',
Expand All @@ -73,21 +70,26 @@ def for_user(self, user):
'skipped_annotations_number',
]

ANNOTATED_FIELDS = {
'task_number': annotate_task_number,
'finished_task_number': annotate_finished_task_number,
'total_predictions_number': annotate_total_predictions_number,
'total_annotations_number': annotate_total_annotations_number,
'num_tasks_with_annotations': annotate_num_tasks_with_annotations,
'useful_annotation_number': annotate_useful_annotation_number,
'ground_truth_number': annotate_ground_truth_number,
'skipped_annotations_number': annotate_skipped_annotations_number,
}

def for_user(self, user):
return self.filter(organization=user.active_organization)

def with_counts(self, fields=None):
return self.with_counts_annotate(self, fields=fields)

@staticmethod
def with_counts_annotate(queryset, fields=None):
available_fields = {
'task_number': annotate_task_number,
'finished_task_number': annotate_finished_task_number,
'total_predictions_number': annotate_total_predictions_number,
'total_annotations_number': annotate_total_annotations_number,
'num_tasks_with_annotations': annotate_num_tasks_with_annotations,
'useful_annotation_number': annotate_useful_annotation_number,
'ground_truth_number': annotate_ground_truth_number,
'skipped_annotations_number': annotate_skipped_annotations_number,
}
available_fields = ProjectManager.ANNOTATED_FIELDS
if fields is None:
to_annotate = available_fields
else:
Expand Down

0 comments on commit 1b21dc8

Please sign in to comment.