-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
54 additions
and
13 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,24 +1,65 @@ | ||
|
||
|
||
|
||
|
||
|
||
|
||
SPOILER CLASSIFICATION | ||
|
||
|
||
Model (multi-class classification - Spoiler Type )) F1 Score | ||
BERT (base uncased) 61.16ROBERTA ( base ) 70.38DistilBERT 65.01Longformer 67.56GPT-3 ( Ada - Prompt 1) 68.50GPT-3 ( Ada - Prompt 2) 67.22GPT-3 ( Curie - Prompt 3) 70.02 | ||
|
||
Model (multi-class classification - Spoiler Type )) F1 Score | ||
|
||
BERT (base uncased) 61.16 | ||
ROBERTA ( base ) 70.38 | ||
DistilBERT 65.01 | ||
Longformer 67.56 | ||
GPT-3 ( Ada - Prompt 1) 68.50 | ||
GPT-3 ( Ada - Prompt 2) 67.22 | ||
GPT-3 ( Curie - Prompt 3) 70.02 | ||
|
||
|
||
|
||
|
||
SPOILER GENERATION (PHRASE) | ||
|
||
|
||
PHRASEModel BLEU Meteor BERT Score | ||
BERT* 0.21 0.48 60.34ROBERTA* 0.31 0.59 71.51Long T5* 0.5473 0.5991 96.68LED* 0.553 0.5764 96.24T5(T5-small) 0.64 0.55Bart 0.20 0.27SPOILER GENERATION (PASSAGE) | ||
PHRASE | ||
Model BLEU Meteor BERT Score | ||
|
||
BERT* 0.25 0.48 60.34 | ||
ROBERTA* 0.36 0.59 71.51 | ||
Long T5* 0.5473 0.5991 96.68 | ||
LED* 0.553 0.5764 96.24 | ||
T5(T5-small) 0.64 0.55 | ||
Bart 0.20 0.27 | ||
|
||
|
||
|
||
SPOILER GENERATION (PASSAGE) | ||
|
||
|
||
PASSAGE | ||
Model BLEU Meteor BERT Score | ||
|
||
BERT* 0.13 0.27 20.63 | ||
ROBERTA* 0.17 0.27 24.25 | ||
Long T5* 0.891 0.9069 98.16 | ||
LED* 0.853 0.8857 97.70 | ||
T5(T5-small) 0.49 0.62 | ||
Bart 0.18 0.25 | ||
|
||
|
||
PASSAGEModel BLEU Meteor BERT ScoreBERT* 0.13 0.27 20.63ROBERTA* 0.17 0.27 24.25Long T5* 0.891 0.9069 98.16LED* 0.853 0.8857 97.70T5(T5-small) 0.49 0.62Bart 0.18 0.25 | ||
|
||
SPOILER GENERATION (MULTI) | ||
|
||
|
||
MULTIModel BLEU Meteor BERT ScoreBERT* 0.032 0.12 14.05ROBERTA* 0.025 0.12 13.86Long T5* 0.8185 0.8620 96.57LED* 0.7546 0.8254 95.79T5(T5-small) 0.5 0.7Bart 0.17 0.22 | ||
MULTI | ||
Model BLEU Meteor BERT Score | ||
|
||
BERT* 0.032 0.12 14.05 | ||
ROBERTA* 0.025 0.12 13.86 | ||
Long T5* 0.8185 0.8620 96.57 | ||
LED* 0.7546 0.8254 95.79 | ||
T5(T5-small) 0.5 0.7 | ||
Bart 0.17 0.22 | ||
|