GPT-3-Encoder Javascript BPE Encoder Decoder for GPT-2 / GPT-3

About

GPT-2 and GPT-3 use byte pair encoding to turn text into a serires of integers to feed into the model. This is a javascript implementation of OpenAI's original python encoder/decoder which can be found here

How to Use

1. Install Node
2. clone https://github.com/AIDungeon/GPT-3-Encoder.git
3. cd GPT-3-Encoder.git
4. npm install
5. node example.js