What is a token in an LLM?
A token is the unit of text a language model reads and writes, usually a word, part of a word or a punctuation mark, and the unit in which context windows, rate limits and API prices are measured.
Language models do not see letters or words. A tokenizer splits text into pieces from a fixed vocabulary, common words as single tokens and rarer words as several fragments, and the model works on those. Every limit that matters in production is counted in them: the context window, maximum output, rate limits and the price per request.
Token counts are not portable between models. Anthropic's token counting docs state that Claude Opus 4.7 and later models use a newer tokenizer that produces roughly 30% more tokens for the same text than earlier Claude models, and tell teams to recount prompts against the model they plan to use.
Kinds of tokens on a bill
Input tokens
Everything sent to the model: system prompt, tool definitions, history, retrieved documents and tool results.
Cached input tokens
Input that matches a cached prefix from an earlier request, billed at a fraction of the normal input rate.
Output tokens
Everything the model writes. Output usually costs several times more than input: Anthropic lists Claude Opus 5.5 at $4 per million input tokens and $20 per million output tokens.
Thinking tokens
The reasoning a model produces before answering. Anthropic bills them as output tokens, and they count toward the maximum output and the context window.
Why token counts drive product decisions
In an agent, one user action can trigger dozens of model calls, and each call resends the growing conversation. Cost per completed task, not cost per call, is the number that decides whether a feature pays for itself.
The levers are well known: keep stable content at the start of the prompt so it can be cached, trim tool results before they enter the context, use a smaller model for simple steps, move non-urgent work to a batch API, which Anthropic prices at 50% off, and cap output length where a short answer will do.
Common confusions
A token is not a word
Short common words are often one token; long, rare or non-English words often take several. Code and structured data can tokenise very differently from prose.
Counts are estimates before sending
Anthropic's token counting endpoint is free but returns an estimate; the billed figure comes from the actual request.
Caching does not shrink the window
Cached tokens cost less but still occupy the context window.
How Motiontic's engineers apply it
Motiontic's engineers log input, cached, output and thinking tokens per step in traces, and report cost per completed task alongside quality in every eval run. A prompt or model change that improves quality but doubles tokens per task is a decision, not a free win.
How tokens are billed on Claude Opus 5.5 (Anthropic list prices, November 2026)
| Token type | Price per million | Note |
|---|---|---|
| Input | $4 | System prompt, tools, history, documents, tool results |
| Cache read | $0.20 | Input that hits a cached prefix |
| Output | $20 | The visible response |
| Thinking | $20 | Billed as output; counts toward max_tokens and the window |
| Batch API | 50% off | For work that can wait |
Common questions
What is a token in AI?
A small unit of text, usually a word or part of a word, that a language model reads and writes. Context limits and API prices are measured in tokens.
Why do output tokens cost more than input tokens?
Output is generated one token at a time, while input is processed in parallel, so output takes more compute per token. Vendors price it accordingly: Claude Opus 5.5 output costs five times its input rate.
Do different models count tokens differently?
Yes. Each model family has its own tokenizer. Anthropic reports that its newer tokenizer, used from Claude Opus 4.7 onward, produces roughly 30% more tokens for the same text than earlier Claude models.
How can a team reduce token costs in an agent?
Cache stable prompt prefixes, trim tool results, route simple steps to smaller models, batch work that can wait, and measure cost per completed task rather than per call.
Related
Context window
The model's working memory for one request, measured in tokens, and finite.
Prompt caching
Paying less for the part of the prompt that does not change between calls.
Large language model (LLM)
A next-token predictor trained at scale, and the engine inside every AI agent.
Extended thinking
Claude reasoning in thinking blocks before it answers, now steered by effort.
AI product development
New products and major AI features, built from scratch alongside the client's team.
Running agents that have to hold up?
Describe the agent, the systems it touches and what it must never get wrong. The first call is with a lab engineer, who will say plainly whether it is ready for production, and whether Motiontic is the right team.