How to perform "segment everything" and "salient instance segmentation"??

[xiankgx]

Dear author, thank for releasing your model and Colab notebook on example usage. But in your Colab notebook, you demonstrated how to perform inference with box prompt and point prompt inputs. What about "segment everything" and "salient instance segmentation"? How do we do that? Can you please provide some examples.

[yformer]

Hi @xiankgx, thanks for your interest in this work. For segment everything, we are cleaning our code and adding it in the Colab notebook. For salient instance segmentation, we used our internal infra for the task and it may take a while to add it in the Colab notebook. When the code is ready, we will message you here.

[xiankgx]

@yformer Thank you so much! Looking forward for that release. We're finding great use case for this.

[skylning]

Looking forward "segment everything" and "salient instance segmentation" release

[exhyy]

I tried to perform "segment everything" like the original SAM and also changed some parameters (e.g. pred_iou_thresh), but they did not work well. Looking forward for the release.

[bxl1234]

I am looking forward too "segment everything"

[Horatio9702]

I'm interested in segmenting salient objects, can this be done directly by using text prompts like "salient objects"?

[yformer]

@xiankgx, @skylning, @exhyy, @bxl1234, @Horatio9702, The segment everything example can be seen at notebooks/EfficientSAM_segment_everything_example.ipynb. We will also add salient segmentation example.

[bxl1234]

@yformer I tried "segment everything" and its current execution speed is very slow, much slower than "sam". What's going on?

[yformer]

@bxl1234 It should not be the case. Can you share more information?

[bxl1234]

@yformer
I used "EfficientSAM_segment_everything_example.ipynb" to execute it without making any changes. I looked at the code and the calculation is done on the CPU. How can I use the GPU?

[UyeYwb]

@yformer I used "EfficientSAM_segment_everything_example.ipynb" to execute it without making any changes. I looked at the code and the calculation is done on the CPU. How can I use the GPU?

you can change .cpu()2,cuda(), its work for me！

in def run_everything_ours（）：
model = model.cuda()
img_tensor.cuda(),
points.reshape(1, num_pts, 1, 2).cuda(),
point_labels.reshape(1, num_pts, 1).cuda(),
model.cuda(),

[UyeYwb]

The speed of full segmentation inference is too slow.I cant find reason.

[ibrandiay]

@UyeYwb What is the input size of your images? how many fps did you get? Don't forget that images that are too large on input degrade the performance of your network.

[UyeYwb]

@UyeYwb What is the input size of your images? how many fps did you get? Don't forget that images that are too large on input degrade the performance of your network.

The input image I use is the dog image in the example.

[xiankgx]

@yformer Thank you author for your release. Great job!

[yformer]

@xiankgx @skylning @UyeYwb @exhyy @bxl1234 @Horatio9702, we released efficient track anything for unified image and video segmentation, https://github.com/yformer/EfficientTAM, which also supports segment everything