Skip to content

What is a token in AI?

Glossary entry. Checked on 19 November 2026.

Token, in plain words

A token is the smallest unit of text a language model works with: a word, part of a word, a character or a byte. Models read and write tokens, so context limits and API prices are counted in tokens rather than words.

Tokens and words

How many characters make a token depends on the model and the language. Anthropic's glossary gives about 3.5 English characters per token for its earlier models, and says its newer tokenizer (Claude 4.7 and later) produces about 30% more tokens for the same text. For exact numbers, providers offer token counting for the model you use.

Why you meet them

  • Limits: the context window is a number of tokens.
  • Prices: API usage is billed per input and output token.
  • Speed: longer outputs mean more tokens to generate.
Share this definition

Sources

Official documentation, checked on 19 November 2026. Products change: if a step differs, the source has the current version.