ModelCheckpoint callback saves the encoder weights even though they are frozen

[nkaenzig]

During evals, we currently use lightning.pytorch.callbacks.ModelCheckpoint to save the best model checkpoints during fit. While we only fit the decoder / head on the downstream tasks, the encoder usually remains frozen, so it's not necessary to include it into the saved checkpoints.

Furthermore, for big encoders (e.g. ViT-G), this becomes a major bottleneck in terms of runtime, taking up to 30 seconds to save the checkpoint, while the GPU remains idle: