Context Window: Token Limits, Recall and What Fits

A context window is the total token space a large language model (LLM) can use for a request: the input it reads plus the output it generates. In a 1M-token window, your instructions, documents and conversation must share that space with the response, including thinking where the model uses it. Filling the window is not the same as getting the model to use every detail correctly.

What Is a Context Window in AI?

Think of the context window as a workbench. Documents, instructions and unfinished work occupy its surface. A larger bench lets you lay out more material, but you still need room to work, and spreading out every document does not guarantee you notice the relevant clause.

A token is a unit the model processes, often a word fragment rather than a whole word. Text, code and other inputs are converted into tokens under the model's own rules. Count the actual request instead of estimating capacity from pages or file size. Google's token guide explains this distinction.

The input can include material you never typed into the chat box:

• System instructions that define the assistant's behavior.
• Your current message and retained conversation history.
• Tool definitions, which describe the actions available to the model.
• Tool results, retrieved passages, images and documents.

Anthropic explicitly includes these components in its context accounting. A short user question can therefore sit inside a very large request.

Context Window vs Token Limit

The context window limits the whole request; the output limit caps the generated portion. Reasoning models may use part of that generated budget for thinking before producing the visible answer. OpenAI documents input, output and reasoning as consumers of the same window.

For a shared window, the planning rule is:

Input tokens + reserved generated tokens must fit within the context window.

Include reasoning in that reservation when the model counts it there. Also respect the model's separate maximum output. Setting a large output limit does not create extra space.

What Does the Context Window in an AI Tool Determine?

It determines how much material the model can consider together for that request. It does not tell you how accurately the model will use it, what the application stores between sessions, or which stored material the application sends back next time.

For a support operator, the difference is practical: a ticket can remain saved in your help desk while being absent from the assistant's current input. For a builder, saving a transcript and assembling the next model request are separate responsibilities.

Current Context Window Size: Claude, GPT-6 and Gemini 4

The main API models below offer roughly million-token windows, but their output rules matter as much as that headline capacity. Verified against the makers' live model pages on 16 October 2026. Each row links directly to its maker; these are API specifications, not promises about a consumer chat plan or an application's own limits.

Claude's current model page lists the same window and output ceiling across the models shown. That makes window size a poor way to choose between them: compare their answers on your task instead.

GPT-6 Astra, GPT-6.1 Sol and GPT-6 Luna also share the listed capacity. A model with the same window can still behave differently on extraction, planning or coding. The specification establishes what fits, not a quality ranking.

Gemini 4 Argon's long decode allowance is part of its shared window. Google says the API caps requested output to the space remaining after input; thinking also consumes that budget. A large output setting cannot be added on top of a full input window. Gemini token accounting gives the precise rule.

A Worked Example: The Prompt Fits, the Response Does Not

A million-token prompt leaves less room for a response than the model's maximum output suggests.

Take GPT-6 Astra's documented limits. Suppose the assembled request contains 1,000,000 input tokens, including instructions, tools, history and source documents. This is an illustrative budget, not a measured workload. You want to reserve the full 128,000 generated tokens allowed by the model.

The arithmetic is:

• Window: 1,050,000 tokens.
• Space remaining after input: 1,050,000 − 1,000,000 = 50,000 tokens.
• Planned input plus output: 1,000,000 + 128,000 = 1,128,000 tokens.
• Shortfall: 78,000 tokens.

The input fits. The planned response budget does not. The remaining space must cover reasoning and the visible answer together.

To reserve that full output allowance, input must be at most 922,000 tokens, calculated as 1,050,000 − 128,000, before any additional margin your workflow needs. You do not have to reserve the maximum for every task; choose the reservation from the output the task requires.

Why a Bigger Window Does Not Guarantee Better Recall

A request can fit and still produce an answer that misses evidence inside it. That is a quality failure, even when the API accepts the prompt.

Anthropic describes declining recall as context grows in its context engineering guidance. The failure does not have to appear as a dramatic refusal. An assistant might overlook an exception, mix an old policy with its replacement, or answer from a nearby passage that looks relevant.

The research paper Lost in the Middle, by Nelson F. Liu and colleagues, found that performance on document question answering and key-value retrieval depended on where relevant information appeared. Evidence in the middle could be harder to use than evidence near the ends. That is historical research, not a benchmark of the current models in the table.

Treat long-context quality as something to measure on your workload. There is no universal token count here at which every model suddenly becomes unreliable.

For a support assistant, build a small evaluation set around policy exceptions whose correct answers you already know. Compare a focused request containing the applicable policy with a longer request containing unrelated policies as well. Move the decisive passage earlier, into the middle and later. Check whether the answer uses the correct exception and cites the right source.

For a coding assistant, ask a question that depends on an implementation file and its caller. Add unrelated files only after establishing the focused answer. If the longer version gets worse, buying more capacity does not diagnose the mistake.

A successful search for a distinctive sentence is useful evidence about retrieval. It does not establish that the model can reconcile conflicting documents or carry every constraint through a long task.

What to Do When You Hit Context Too Long

Builders should inspect the assembled request. Operators should preserve the current task state before continuing the session. Both need to distinguish an input rejection from an answer that stopped because it ran out of room.

For current Claude models, Anthropic documents a prompt-too-long error when input alone exceeds the window. If input fits but generation reaches the boundary, the response can instead stop with model_context_window_exceeded. Check the actual error or stop reason rather than assuming the system quietly discarded old messages. Claude overflow behavior.

1. 1

   Count what the application sends

   Inspect instructions, conversation messages, tool definitions, retrieved passages and tool results together. Use the provider's token-counting facility where it supports your request shape, then compare its estimate with response usage. Anthropic's counting endpoint accepts structured messages and many input types, but has restrictions for server tools and some file sources.

2. 2

   Reserve generated space

   Record the model's window, its output ceiling and the output reservation you need. Include thinking where applicable. Reduce the input budget accordingly; do not treat every remaining token as room for another document.

3. 3

   Find the material that keeps growing

   Look for repeated search results, entire logs, duplicate documents and abandoned branches of the conversation. A coding agent may need the failing test and relevant function; it rarely needs every previous successful command's full output in every turn. Keep the evidence needed to verify the next action.

4. 4

   Keep a continuation note

   Before restarting or compacting, preserve the objective, confirmed decisions, unresolved issues, relevant source locations and next action. For an operator handling a customer case, keep the current request and applicable policy exception. Reopen the original source whenever exact wording matters.

5. 5

   Check correctness after the change

   Repeat the failing task with the smaller request. Confirm that the required evidence is present and the answer is correct. If the prompt now fits but still misses the decisive detail, investigate selection and recall instead of continuing to trim blindly.

If a short, stable workflow already answers correctly and stays within its limits, leave its model choice alone. A larger advertised window is not by itself a reason to migrate.

Choose Retrieval, Summaries, Compaction or Caching by the Problem

Choose the intervention according to what is taking up space. These techniques solve different problems, and they can work together.

Use retrieval when the source collection is much larger than the question's evidence needs. Retrieval-augmented generation (RAG) supplies selected source material before the model answers. A support assistant should fetch the applicable return policy and order record instead of reading the entire knowledge base. The agentic RAG guide covers how an agent can search again when the first evidence is insufficient.

The limit is selection: if retrieval misses the decisive exception, the model cannot use it. Keep a bounded set of complete documents in context when the task genuinely requires comparing their details throughout, and evaluate whether narrowing the set drops needed evidence.

Use a summary when old detail can become durable task state. A summary should preserve what was decided, what remains uncertain and where the supporting sources live. For a code change, that might mean the chosen approach, current failing behavior and relevant file paths. Do not replace an exact contractual clause or an error message with a loose paraphrase when the next decision depends on its wording.

Use compaction when the conversation itself has become the bulk. Compaction replaces accumulated history with a smaller continuation context. Anthropic warns that overly aggressive compression can lose important details, so check what survives rather than treating a compacted conversation as a lossless archive. Anthropic's compaction discussion explains the tradeoff.

The broader context engineering guide explains how to choose and maintain those contents across an agent's work.

Use prompt caching when useful input repeats. It reuses processing of a matching prompt prefix, the unchanged beginning of the request. Put stable instructions and shared material before changing content, and follow the model's cache rules; OpenAI's current guide includes explicit breakpoint behavior, while Gemini 4 Argon's model page lists implicit caching only. OpenAI caching, Gemini caching support.

Long prompts add billable input tokens; cache hits can reduce the charge for repeated input, subject to cache-write rules, but those tokens still occupy the window. Caching mechanics are covered separately.

Compacting or rewriting the beginning of a prompt can reduce cache reuse. Preserve a stable prefix when it remains useful; remove irrelevant history when it compromises the request. A high cache-hit rate does not certify a good answer.

What Is Overhyped About Long Context?

The capacity number is often asked to prove things it does not measure. It does not establish perfect recall, permanent memory, or the ability to replace every retrieval workflow with a larger prompt.

The useful case for a bigger window is concrete: your task needs more relevant evidence simultaneously, the smaller request loses essential relationships, and the larger one produces correct results in your evaluation. That is a reason to use the capacity.

Act now when logs, history or source collections are crowding out the answer. First improve what enters the request. Consider a larger model window when the relevant material still cannot fit. Wait when the existing workflow is reliable and your only motivation is a larger specification.

Questions About Context Windows

What is a context window in an AI language model?

It is the model's per-request token capacity for the input it considers and the output it generates. For reasoning models, account for thinking under the maker's rules as well.

How long is a 200K context window?

200K means 200,000 tokens. It is not a fixed number of words or pages, and a shared window must also accommodate generated output. Count your material with the selected model's tokenizer or supported counting endpoint.

What does context mean in programming?

Context is the surrounding information available to an operation. In an LLM application, it means the instructions, messages and supporting material assembled for the current model request.

What is a context engineering example?

A support assistant receives the current ticket, the applicable policy and a concise record of prior decisions. It retrieves additional evidence when needed instead of receiving the full ticket archive.

How much do context engineers make?

A context-window specification cannot establish a salary. For hiring, define whether the work involves retrieval, application engineering, evaluations or infrastructure, then compare roles with that scope in your location.

How can I learn context engineering?

Inspect the complete requests from a small application. Change the instructions, retrieved evidence or retained history separately, and compare answers against known correct results. Learn to explain why a change improved correctness, not just why it shortened the prompt.

What's the difference between context engineering and prompt engineering?

Prompt engineering focuses on the instructions and examples you give the model. Context engineering also covers selecting evidence, providing tools and retaining the right state as work continues.

What are the best practices for context engineering?

Budget the full request, reserve output space, retain relevant evidence and keep source pointers when summarizing. Check that the answer remains correct after trimming or compaction.

What are the best skills for agents in context engineering?

Give the agent focused ways to find sources, inspect relevant passages and preserve task state. Judge those tools by whether their results help the next decision without flooding the request with unnecessary material.

The Monday Move

Take a request that failed or missed an important detail. Count the complete input, reserve room for the expected output, and identify the largest block that does not help answer the question. Replace that block with selected evidence or a continuation note, then repeat the task against a known answer. Keep the smaller request only if it preserves the evidence and the result.

For practical guides to building reliable AI workflows, join the newsletter.