Spurious track switch

[olivier-cuttlefish]

Hello, I haven't found any similar issues by searching for track switch in the github so I am opening a discussion here.

I am having a hard time understanding how this kind of track switch would happen. Here, orange track went out of frames. Blue track had succesfully predicted instances before and after the switch, with no discontinuities. Yet, the track identitie have been switched. This would easily be fixable using manual code but I would still like to understand what would be the issue.

Here are relevant informations.
In the picture, trail_length is at 250 frames.
Video is recorded at 25 FPS.
This is a bottomup model (not ID).
Tracking parameters are the following:

    sleap-track --tracking.tracker simplemaxtracks \
                --tracking.max_tracks 2 \
                --tracking.max_tracking true \
                --tracking.pre_cull_to_target 2 \
                --tracking.match hungarian \
                --tracking.track_window 25 \
                --tracking.clean_instance_count 2 \
                --tracking.similarity instance \

A secondary question, less important though. In these parameters, I am culling to two tracks both before and after, which I assume is redundant. What would be the pros and cons of culling to target number instances before or after tracking ?

Thank you again for your help and amazing tool.

[talmo]

Hi @olivier-cuttlefish,

I think the issue here is that the track window is a bit too large.

Try setting it to 5. With simplemaxtracks you'll be using a track local queue so it should be able to bridge across longer gaps anyway, so you still get the tracks linked even if they are occluded for longer than that.

Let us know if that helps!

Talmo

[olivier-cuttlefish]

Hi Talmo,

Thank you for the quick reply. This indeed helped reduce the number of spontaneous track switch, thank you! However I still don't understand why this happened in the example I showed above, as both trajectories did not get close. I am trying to build an understanding of how the tracking works but it seems less straightforward than what I understood from the documentation

Also, regarding cull before or after tracking, what would be the pros and cons of these ?
I am asking because I am suspecting it to be an issue in the following examples, that I get a lot:


Here in the first frame, detection and tracking is happening properly. In the following frame, two instances are predicted on the same individual, while the first one lost its instance. In the following frame (not shown), the instance is recovered but the identities have been switched. What I am suspecting is that in the problematic frame, initially there was 2+ instances detected, but somehow the one on the left fish got culled. This is surprising to me because the model is very well-trained (1000+ frames) and the spurious instance has a low-score, while the one that should have been kept had a high score on the previous frame. Would you have any tips to fix that ?

Cheers,
Olivier

[lilyava873]

"Hi Olivier, the track switch might be due to the track_window size (25 frames) or the instance similarity metric, which can confuse identities if tracks momentarily align. Reducing the window (e.g., to 15) or experimenting with centroid similarity might help.

For culling, post-culling alone should suffice if you consistently have two instances, as pre-culling might exclude valid candidates. Visualizing similarity scores, much like in systems for tforce final mile tracking, could also shed light on the issue. Hope this helps!"