Ensembling multiple semantic segmentation models

[ruipimentelfigueiredo]

Hello,
I was wondering if this library provides any way of averaging / voting the output of different models. If so could anyone provide an example?
Thanks

[cagey-squirrel]

Do you have a binary segmentation or multiple classes?
There's a simple way to implement binary segmentation ensembling by taking mean of segmentation probabilities from all models. Although this is not exactly the same as voting, it can sometimes give even better results. Multiclass segmentation can be implemented in a similar way as well.

[ruipimentelfigueiredo]

Could you provide some sample code with how to do it? i.e. how to extract the probabilities of multiple networks and use them for voting A quinta, 14/12/2023, 12:50, cagey-squirrel ***@***.***> escreveu:

…

Do you have a binary segmentation or multiple classes? There's a simple way to implement binary segmentation ensembling by taking mean of segmentation probabilities from all models. Although this is not exactly the same as voting, it can sometimes give even better results. Multiclass segmentation can be implemented in a similar way as well. — Reply to this email directly, view it on GitHub <#813 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVNAN5HECF4THYUBVALUGDYJLYXPAVCNFSM6AAAAAA5EIWVMWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TQNJTGI4DE> . You are receiving this because you authored the thread.Message ID: <qubvel/segmentation_models.pytorch/repo-discussions/813/comments/7853282@ github.com>

[cagey-squirrel]

Sure thing.
For example, for binary classification you can train multiple unet models with sigmoid activation function:
model = smp.Unet(encoder_name='resnet34', encoder_depth=5, encoder_weights='imagenet', activation='sigmoid')
When you finish training each of these models you save their weights so you can load them later.

Now, to make an ensemble
You can load you models into a list:
ensemble = [load_model(model_path) for model_path in list_of_model_paths]
Then for each batch you can call each model once and aggregate the results:

for data_features in data_loader:
    ensemble_prediction = torch.zeros((BATCH_SIZE, CHANNEL_SIZE, WIDTH, HEIGHT))
    for model in ensemble:
        prediction_probabilities = model(data_features)
        ensemble_prediction += prediction_probabilities

     ensemble_prediction /= len(ensemble)
     ensemble_prediction = ensemble_prediction > 0.5 # Getting 0/1 values from probabilities with 0.5 treshold