nlp

Repo that holds the code produced for the course Natural Language Processing, taught by the professor Thiago Faleiros.

Author

Matheus Stauffer Viana de Oliveira

Instructions

To ensure that you are going to execute this repo code correctly, please create a virtual environment and install the requirements:

python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

Summary

1. Byte-Pair Encoding tokenization
2. Bigram Model
3. Classification Experiment