Skip to content

Commit

Permalink
xlm attn readme added
Browse files Browse the repository at this point in the history
  • Loading branch information
graviraja committed Aug 18, 2020
1 parent dd59be9 commit f58039b
Show file tree
Hide file tree
Showing 4 changed files with 36 additions and 0 deletions.
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -305,6 +305,15 @@ To alleviate this problem XLM-RoBERTa model (which has been pre-trained on 100 l

An F1 score of `0.59` was achieved. Methods to improve this will be explored later on.

### Day 94: Sentimix with XLM-RoBERTa-LSTM-Attention

The final output from XLM-RoBERTa model were used as input embeddings to bi-directional LSTM model. An attention layer, which takes the outputs from LSTM layer, produces a weighted representation of the input, which is then passed through a classifier for predicting the sentiment of the sentence.

![codemix](assets/images/applications/sentiment/codemix_xlm_attn.png)

An F1 score of `0.64` was achieved.


### Day 21: Document Classification with Hierarchical Attention Network

A Hierarchical Attention Network (HAN) considers the hierarchical structure of documents (document - sentences - words) and includes an attention mechanism that is able to find the most important words and sentences in a document while taking the context into consideration.
Expand Down
7 changes: 7 additions & 0 deletions applications/classification/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,13 @@ To alleviate this problem XLM-RoBERTa model (which has been pre-trained on 100 l

An F1 score of `0.59` was achieved. Methods to improve this will be explored later on.


### Sentimix with XLM-RoBERTa-LSTM-Attention.ipynb

The final output from XLM-RoBERTa model were used as input embeddings to bi-directional LSTM model. An attention layer, which takes the outputs from LSTM layer, produces a weighted representation of the input, which is then passed through a classifier for predicting the sentiment of the sentence.

An F1 score of `0.64` was achieved.

# Document Classification

## Document Classification with Hierarchical Attention Network
Expand Down
20 changes: 20 additions & 0 deletions applications/classification/sentiment_classification/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,23 @@ Following metrics were achieved on test data.
- `F1-Score`: 0.59

Methods to improve this will be explored later on.


### Sentimix with XLM-RoBERTa-LSTM-Attention.ipynb

The final output from XLM-RoBERTa model were used as input embeddings to bi-directional LSTM model. An attention layer, which takes the outputs from LSTM layer, produces a weighted representation of the input, which is then passed through a classifier for predicting the sentiment of the sentence.

![codemix](../../../assets/images/applications/sentiment/codemix_xlm_attn.png)


Following metrics were achieved on test data.

- `Precision`: 0.64

- `Recall`: 0.64

- `F1-Score`: 0.64

#### References

- [Google transliterated data](https://github.com/keshav22bansal/BAKSA_IITK)
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit f58039b

Please sign in to comment.