Skip to content

darknoon/tiktokenizer

This branch is 3 commits behind dqbd/tiktokenizer:master.

Folders and files

NameName
Last commit message
Last commit date

Latest commit

071fb6e · May 14, 2024

History

54 Commits
Mar 2, 2023
May 14, 2024
Apr 22, 2024
Mar 2, 2023
Apr 25, 2024
Mar 3, 2023
Mar 3, 2023
Mar 17, 2023
May 14, 2024
Mar 2, 2023
Mar 2, 2023
Mar 2, 2023
Mar 2, 2023
May 14, 2024

Repository files navigation

Tiktokenizer


Tiktokenizer

Online playground for openai/tiktoken, calculating the correct number of tokens for a given prompt.

Special thanks to Diagram for sponsorship and guidance.

CleanShot.2023-03-02.at.22.58.11.mp4

Acknowledgments

About

Online playground for OpenAPI tokenizers

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • TypeScript 91.6%
  • JavaScript 8.3%
  • CSS 0.1%