with torch.enable_grad()

[gk966988]

Hi,
in forward( ) function, in gain.py
if remove the code " with torch.enable_grad():" what will it happen? Does it influence the final result?

[paganpasta]

@gk966988 I am confused as well. To my understanding with torch.enable_grad() should be removed and any other calls to change the mode to train or otherwise. If you are calling forward in eval or train, it still allows you to compute gradients. Moreover, to preserve consistency for validation inference, the mode should not be switched from eval to train. Only thing to make sure is not using with torch.no_grad() header during testing.

But would love to know from @ngxbac the reasoning behind his choice.