Skip to content

Commit

Permalink
Update transformers version in examples to 4.30.0 (facebookresearch#430)
Browse files Browse the repository at this point in the history
Summary:
Context in huggingface/transformers@80ca924

Pull Request resolved: facebookresearch#430

Test Plan:
Build conda env with transformers==4.30.0 then run some of the example scripts.

FLAVA:
```
python -m flava.train config=flava/configs/pretraining/debug.yaml training.lightning.accelerator=cpu training.lightning.gpus=0 training.lightning.strategy=null
...
Epoch 0: : 9it [00:30,  3.36s/it, loss=11.3, v_num=0, train/losses/mlm_loss=10.50, train/losses/mim_loss=9.200, train/losses/mmm_text_loss=10.80, train/losses/mmm
Epoch 0: : 10it [00:33,  3.32s/it, loss=11.3, v_num=0, train/losses/mlm_loss=10.50, train/losses/mim_loss=9.200, train/losses/mmm_text_loss=10.80, train/losses/mm
...
```

ALBEF:
```
torchrun --nproc_per_node=4 finetune_retrieval.py --config ./configs/retrieval.yaml
...
time 0:00:59, epoch 0/5, batch 0/4427, loss 2.0888588428497314
time 0:01:10, epoch 0/5, batch 5/4427, loss 1.331891655921936
time 0:01:22, epoch 0/5, batch 10/4427, loss 1.749828577041626
time 0:01:34, epoch 0/5, batch 15/4427, loss 1.4500877857208252
time 0:01:47, epoch 0/5, batch 20/4427, loss 1.3513760566711426
...
```

MDETR:
```
CUBLAS_WORKSPACE_CONFIG=:4096:8 torchrun --nproc_per_node=2 phrase_grounding.py \
--resume https://pytorch.s3.amazonaws.com/models/multimodal/mdetr/pretrained_resnet101_checkpoint.pth --ema --eval --dataset_config phrase_grounding.json
...[   0/1247]  eta: 2:27:31    time: 7.0986  data: 0.5251  max mem: 2649[  10/1247]  eta: 0:19:15    time: 0.9340  data: 0.1489  max mem: 2979[  20/1247]  eta: 0:12:31    time: 0.2882  data: 0.1060  max mem: 2979[  30/1247]  eta: 0:10:02    time: 0.2537  data: 0.0998  max mem: 2979[  40/1247]  eta: 0:08:39    time: 0.2396  data: 0.0956  max mem: 2979
...
```

Reviewed By: pbontrager

Differential Revision: D46668375

Pulled By: ebsmothers

fbshipit-source-id: 05b066ea0394bc180e1a5f0a5cc2d7898da9fcfa
  • Loading branch information
ebsmothers authored and facebook-github-bot committed Jun 13, 2023
1 parent f76aa2d commit 0bf3e07
Show file tree
Hide file tree
Showing 4 changed files with 4 additions and 4 deletions.
2 changes: 1 addition & 1 deletion examples/albef/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ opencv-python==4.6.0.66
pytorch-lightning==1.6.0
Pillow==9.3.0
ruamel_yaml==0.17.21
transformers==4.24.0
transformers==4.30.0
2 changes: 1 addition & 1 deletion examples/flava/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,5 +5,5 @@ requests==2.27.1
DALL-E==0.1
omegaconf==2.1.2
hydra-core==1.1.2
transformers==4.16.0
transformers==4.30.0
pycocotools==2.0.4
2 changes: 1 addition & 1 deletion examples/mdetr/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ prettytable==3.3.0
pycocotools==2.0.4
pytorch-lightning==1.6.0
scipy==1.8.1
transformers==4.16.0
transformers==4.30.0
2 changes: 1 addition & 1 deletion examples/mugen/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
transformers==4.11.3
transformers==4.30.0
av==8.0.3
omegaconf==2.1.2
hydra-core==1.1.2

0 comments on commit 0bf3e07

Please sign in to comment.