GPT-2 and GPT-3 use byte pair encoding to turn text into a series of integers to feed into the model. latitudegames implemented this in Node.js. This is an implementation of their encoder for use in the browser, re-written in Typescript.
npm install @beskar-labs/gpt-encoder
import { encode } from '@beskar-labs/gpt-encoder';
const tokens = encode(
'This is an example sentence to try encoding out on!'
).length;