Tokens, Words and Characters Calculator for LLMs

Tokens can be thought of as pieces of words. Calculate precise token counts using the GPT-2 tokenizer.