Each token could either be an entire word or chunks of characters within a word. For example, longer words such as “hamburger” get broken down into three tokens: “ham”, “bur” and “ger”. Shorter and smaller words, such as “pear”, is counted as a single token. Generally speaking, a token is usually 4 characters long, an average sentence would be around 25 tokens, an average paragraph would be around 100 tokens and 1,500 words would be around 2,000 tokens.