[Fix] Move attention mask to the model device type (databrickslabs#180)

The attention mask needs to be on the same device as the rest of the model and inputs, or else there will be a device mismatch.

training/generate.py

@@ -136,7 +136,7 @@ def _forward(self, model_inputs, **generate_kwargs):
 
         generated_sequence = self.model.generate(
             input_ids=input_ids.to(self.model.device),
-            attention_mask=attention_mask,
+            attention_mask=attention_mask.to(self.model.device),
             pad_token_id=self.tokenizer.pad_token_id,
             **generate_kwargs,
         )