A short question can produce a surprisingly large AI request. The model may also receive instructions, earlier messages, tool results and pieces of uploaded files.
AI systems measure much of that work in tokens. Understanding the count helps explain why a long conversation behaves differently from a fresh chat, and why an agent can use more than a simple question.
There are three separate ideas to keep straight: the tokens processed, the context that fits in one request, and the way a product bills or limits your usage.
What is an AI token?
For text, a token is a chunk produced by a model’s tokenizer. It might represent a word, part of a word or punctuation. Tokens are not a fixed word count.
OpenAI’s key concepts guide gives a rough English estimate of about four characters, or three-quarters of a word, per token. That makes 300 English words roughly 400 tokens. Treat this as an estimate, not an exact conversion for every language or model.
If you are building with an API, count with tools intended for the model you will use. Anthropic’s token-counting documentation also warns that an estimate can differ slightly from actual request usage.
Input and output are two sides of the request
Input is what the model receives for a request. Output is what it generates. Your typed question is only one possible part of the input.
Suppose you ask for help answering an email. For a simplified example, assume the email and instructions total 400 input tokens, and the reply uses 200 output tokens.
That request processes 600 tokens across the two sides. It does not necessarily cost the same as 600 input tokens because API providers can price input and output differently.
Some reasoning models also account for thinking tokens that are not displayed as the final answer. Check the selected model’s usage and pricing documentation before treating visible reply length as the whole bill.
A simple cost example without a price table
Use separate rates for the two sides. Imagine a hypothetical API charging $2 per million input tokens and $10 per million output tokens. These are example rates, not a quote for a current plan or model.
- Input: 400 ÷ 1,000,000 × $2 = $0.0008.
- Output: 200 ÷ 1,000,000 × $10 = $0.002.
- Total for this simplified request: $0.0028, or 0.28 cents.
This calculation leaves out reasoning, tools, cache writes and other possible charges. Use actual usage records and the applicable rates for a real bill.
The separate AI token cost guide goes deeper into the cost side. Here, the important habit is to keep input and output separate instead of multiplying one total by one price.
A context window is capacity, not a monthly allowance
The context window is the amount a model can work with in a request. Conversation history, instructions, documents and tool results can take up space alongside the generated response.
Anthropic’s context window guide explains how these pieces fit together. A large window also does not guarantee perfect recall of every detail.
Picture a desk rather than a filing cabinet. The window describes the working material available now. It does not mean the model has permanently learned your files.
If an app summarizes or drops older material to keep working, the effective context can change. Save important decisions somewhere you can inspect. Our article on chat history and backups explains why the conversation itself is not enough.
Why a follow-up can process old information again
A tiny follow-up does not always mean a tiny request. An application may include earlier messages or reference material again so the model can understand what you mean.
For a hypothetical report workflow, suppose the same 40,000-token report is included in each of five follow-up requests. The report alone contributes 200,000 input tokens across those requests. It does not need to fit into a 200,000-token window all at once.
Actual applications differ in how they retain, retrieve, summarize or cache context. The example explains cumulative processing, not a guaranteed bill for every chat product.
Prompt caching can reduce repeated processing costs for eligible content. It does not make that content disappear from the model’s context. Anthropic’s caching guide describes its requirements and separate cache pricing.
Subscription limits are another measurement
A context-window limit tells you what fits in a request. An API bill tracks chargeable usage. A chat subscription has its own product limits, which may depend on the model, tools, workload and plan.
Do not convert a monthly subscription fee into a promised number of tokens unless the provider explicitly gives that allowance. A model’s context size is not such a promise.
Five ways to keep the work focused
- Start a new chat when the job changes. Bring a short, checked handoff containing the facts still needed.
- Send the relevant material. Include enough context to answer correctly, without unrelated folders or repeated documents.
- Specify the useful output. Ask for a brief answer when a brief answer will do.
- Check the usage record. Builders should measure actual requests instead of estimating every job from word count.
- Set limits for agent work. Bound the scope and retries, and ask for a progress report when the task cannot be completed within them.
The goal is useful context. Removing a critical requirement to save tokens can make the result more expensive to fix. Keep the evidence the job needs, and cut what no longer helps.
