What Is an AI Token?
Short answer: a token is the piece of text a model actually reads — and the unit every provider bills. It is smaller than a word: roughly 1 token ≈ 4 characters ≈ 0.75 English words. That ratio is a planning estimate, not a constant. Send code, JSON or raw HTML instead of prose and you pay 1.6x to 2.2x more tokens for the same information. Below: what a token really is, the conversion by content type, and why output tokens cost 5–8x input.
Not a word, not a character — a sub-word
Tokenizers split text into frequent fragments. Common English words survive whole; everything else is assembled from pieces.
the, and, price — one token each, which is why ordinary prose meters cheaply.unbelievable becomes un + believ + able: one word, twelve characters, three tokens.1234567 is typically 123 + 456 + 7. IDs, SKUs and UUIDs are far more expensive than their character count suggests. hello and hello are different tokens. Indentation in code is therefore billed, not free.How much text one token really is
Characters per token assumes 1,000 English words ≈ 5,600 characters including spaces. Tokens-per-word ranges match the content-type table in our 1,000-word guide.
| Content type | Characters per token | Tokens per 1,000 words | 1,000 tokens is about | Why it meters that way |
|---|---|---|---|---|
Plain English prose Baseline | ~4.2 | 1,250 – 1,400 | 750 words · 1.5 pages | Short common words map to one token each. |
Business / technical English Reports, specs, docs | ~3.7 | 1,400 – 1,600 | 670 words | Jargon, numbers and abbreviations split into sub-words. |
Transcribed speech Calls, dictation | ~3.9 – 4.7 | 1,200 – 1,450 | 760 words | Fillers and short words tokenize cheaply; punctuation is sparse. |
Source code Python, JS, TypeScript | ~2.6 | 1,900 – 2,400 | 465 words | Indentation, operators, camelCase boundaries and symbols all split. |
JSON / structured data API payloads, configs | ~2.2 | 2,300 – 2,700 | 400 words | Every key repeats, and each quote, brace and colon is its own token. |
HTML markup Scraped pages, emails | ~2.0 | 2,500 – 3,000 | 364 words | Tags, attributes and class names carry no content but are billed. |
CJK text Per 1,000 characters | ~1.0 per character | — | 950 – 1,050 characters | Roughly one token per character; word segmentation is not meaningful. |
Planning estimates, not tokenizer output. The same text meters several percent differently across providers, and the spread inside a content type is real — a dense JSON payload lands at the top of its range, a pretty-printed one higher still. Measure your own text with the Token Counter before budgeting.
The same 20 records, four token bills
Identical product data — 20 records, 6 fields each — serialized four ways and measured with this site's own estimator. Same facts, different invoices.
| Serialization | Characters | Tokens | Multiple vs prose | Cost per 100,000 records on Claude Sonnet 5 |
|---|---|---|---|---|
Plain sentences "SKU AP3-BLK-0001 is the Acoustic Pro 3 …, priced at $249 …" | 2,299 | 532Lowest | 1.00x | $5.32 |
Key–value lines sku: … / name: … / price: … one per line | 2,239 | 527 | 0.99x | $5.27 |
JSON, compact JSON.stringify(items) | 2,521 | 727 | 1.37x | $7.27 |
JSON, pretty-printed JSON.stringify(items, null, 2) | 3,362 | 794 | 1.49x | $7.94 |
Measured with the same heuristic estimator used by the tools on this site, priced at Claude Sonnet 5's $2.00 per 1M input tokens. Two findings generalise: key–value lines cost the same as prose (the labels replace the words), and pretty-printing costs 9% more tokens than compact JSON for zero extra information. Our estimator weights punctuation lightly, so treat 1.37–1.49x as a floor — production tokenizers usually meter JSON braces, quotes and colons closer to the 1.8–1.9x implied by the content-type table above.
Why reading is cheap and writing is not
Every tracked provider charges more for generated tokens than for prompt tokens — but not by the same multiple.
| Model | Input / 1M | Output / 1M | Output ÷ input | Output share of a 1:1 request |
|---|---|---|---|---|
GPT-5.6 Sol OpenAI · flagship (promo) | $4.00 | $20.00 | 5.00x | 83.3% |
GPT-5.6 Terra OpenAI · mid | $2.00 | $12.00 | 6.00x | 85.7% |
GPT-5.6 Luna OpenAI · budget | $0.20 | $1.20 | 6.00x | 85.7% |
Claude Opus 5 Anthropic · flagship | $5.00 | $25.00 | 5.00x | 83.3% |
Claude Sonnet 5 Anthropic · mid | $2.00 | $10.00 | 5.00x | 83.3% |
Claude Haiku 4.5 Anthropic · budget | $1.00 | $5.00 | 5.00x | 83.3% |
Gemini 3.6 Flash Google · mid (promo) | $0.75 | $3.75 | 5.00x | 83.3% |
Gemini 3.5 Flash-Lite Google · budget · widest gap | $0.30 | $2.50 | 8.33x | 89.3% |
Computed from published per-1M rates (OpenAI verified 2026-08-23, Anthropic and Google 2026-08-09 to 2026-08-16). "Output share" assumes a request with equal input and output token counts. OpenAI's long-context surcharge above 272,000 input tokens raises input by 2x and output by 1.5x, which narrows the ratio to roughly 3.75x on GPT-5.6 models. Caching, batch and taxes excluded.
Tokens are the budget. Everything else is a proxy.
Teams plan in words, pages, documents or API calls. Providers bill in tokens, and the translation between the two depends entirely on what kind of text you are sending. A 1,000-word article and a 1,000-word JSON payload differ by roughly 2x in tokens before a single output token is generated.
The practical consequence is that the cheapest optimization is usually in the input, not the model choice. Sending the same information with less scaffolding — converted HTML instead of raw markup, compact records instead of pretty-printed JSON, 8 retrieved chunks instead of 25 — cuts tokens on every request forever, and it compounds with whatever model you route to.
The second consequence is that output dominates conversational bills. Because output is priced 5–8x input, a chat endpoint that reads 2,000 tokens and writes 500 spends more on the answer than the prompt. Capping max_tokens and asking for shorter answers is a direct cost lever — see what a 1,000-word exchange costs on every model.
AI token questions
What is an AI token in simple terms?
A token is the chunk of text a language model actually reads and that providers bill for. It is usually smaller than a word: common English words are one token, longer words split into pieces such as un + believ + able. A practical rule of thumb is 1 token ≈ 4 characters ≈ 0.75 English words, or about 1.33 tokens per word.
How many tokens is one word?
For plain English prose, roughly 1.25 to 1.40 tokens per word — about 1.33 on average, which is the 0.75 words-per-token rule used across this site. Technical English runs 1.4 to 1.6, source code 1.9 to 2.4, JSON 2.3 to 2.7 and raw HTML 2.5 to 3.0 tokens per word. CJK text is better measured per character: about 0.95 to 1.05 tokens per Chinese character.
Why are output tokens more expensive than input tokens?
Reading a prompt can be processed in parallel, while generating a reply happens one token at a time and each new token depends on the ones before it. Providers price that serial work higher: on the eight models tracked here, output costs 5.00x input on GPT-5.6 Sol, Claude Opus 5, Sonnet 5, Haiku 4.5 and Gemini 3.6 Flash, 6.00x on GPT-5.6 Terra and Luna, and 8.33x on Gemini 3.5 Flash-Lite. At equal input and output volumes, output is 83% to 89% of the bill.
Do all models count tokens the same way?
No. Each provider trains its own tokenizer, so the same text can meter several percent differently across OpenAI, Anthropic and Google — and tokenizers also change between model generations. That is why the calculators on this site label results as estimates. If you are budgeting against an invoice, reconcile with the usage object the API returns.
How can I use fewer tokens?
Four levers cover most workloads: strip markup before sending web content, send fewer and smaller retrieved chunks, shorten the system prompt and move repeated instructions into a cached prefix, and route easy requests to a cheaper model. Converting scraped HTML to Markdown is usually the largest single win on input-heavy pipelines.
Where these numbers come from.
Per-million input and output rates are provider-published values, re-verified weekly against official pricing documentation; the output-to-input multiples above are computed directly from those rates. Conversion figures — characters per token, tokens per 1,000 words — are planning estimates derived from the content-type ranges published in our 1,000-word guide, assuming 1,000 English words ≈ 5,600 characters including spaces. They are not tokenizer output, and real counts vary by provider, model generation and the specific text. The token-counter estimator used across this site weights CJK characters, emoji, punctuation, whitespace and Latin characters separately; it is a cross-model approximation, not an exact count. Caching, batch, tool use and taxes are excluded throughout.
Official sources: OpenAI model docs, Anthropic pricing, Google Gemini API pricing. Always confirm the invoice before making purchasing decisions.