Context Engineering: How to Keep Agents on Task

When an agent works in a demo but drifts during real work, inspect what reaches the model before adding more instructions. Context engineering puts you in charge of 5 inputs at each step: instructions, tools, retrieved facts, memory, and history. The aim is enough evidence for the next correct action, with less irrelevant material carried along.

What Is Context Engineering?

Context engineering is the job of deciding what a model sees when it makes its next decision. That includes the instruction you wrote, the tools it can choose, the evidence those tools return, the memories you load, and the conversation you keep.

A model's context window is its working space for a request. A token is a unit of text or other content processed by the model. A document saved somewhere in your system is available information; its contents become active context only when your application or agent loads them.

That distinction explains a familiar production failure. Your agent can have access to the right policy and still answer from an old conversation, because the current policy never reached the model at the decision point. Equally, loading every policy can introduce unrelated exceptions and obsolete versions.

In its September 29, 2025 engineering post, Anthropic describes the work as “curating and maintaining the optimal set of tokens (information) during LLM inference.” Its distinction is between writing instructions and managing the broader information supplied during execution. Anthropic's context engineering post.

LangChain's July 2, 2025 post describes “filling the context window with just the right information at each step of an agent’s trajectory.” It groups the work into write, select, compress, and isolate: save useful state, fetch relevant material, shorten what remains, and separate independent investigations. LangChain's context engineering post.

The practical change is the unit you design. A single prompt can stay fixed. An agent's next request changes whenever it reads a file, receives a tool result, completes a subtask, or learns that an earlier assumption was wrong. You need a policy for assembling that request throughout the task.

Documentation verified October 13, 2026: the tool behavior below was checked against current Anthropic and OpenAI documentation. The worked example is an illustrative design, not a production benchmark.

Context Engineering vs Prompt Engineering

Prompt engineering improves the instruction; context engineering improves the entire request around it. You still need both.

For a coding agent, prompt engineering might replace “fix signup” with “find why retrying signup sends duplicate welcome emails; propose a minimal fix and a regression test.” Context engineering then decides which signup files, retry logs, project rules, and earlier findings the agent receives before proposing that fix.

A beautifully written instruction cannot compensate for loading the wrong environment's logs. A carefully selected log excerpt cannot compensate for an instruction that never says whether the agent should diagnose, edit, or deploy.

The Context Engineering Skill That Matters: Choosing Boundaries

For each piece of information, decide when it should enter context and when it can leave. Permanent project rules, temporary task state, and raw evidence need different lifetimes.

A coding convention belongs in persistent instructions. “The retry branch remains untested” belongs in the current task's handoff. A full provider response belongs in an accessible evidence file, with the relevant fields loaded when they affect the decision.

This is also why retrieval-augmented generation, or RAG, is only part of the job. RAG retrieves supporting material for a model. You still have to choose the instructions, tools, retained history, and output of that retrieval.

Context Engineering for AI Agents: The Main Moves

Start with the largest source of irrelevant context in the failing step. Do not introduce every mechanism at once. A log-heavy debugger needs a different first repair from an assistant that keeps forgetting project constraints.

Compact History Around Decisions, Not Just Topics

Claude Code, Anthropic's coding agent, provides /context to inspect context usage and /compact to shorten a conversation. Its automatic management clears older tool outputs and then summarizes the conversation if needed. The documentation explicitly warns that detailed instructions from earlier in the conversation can be lost. Claude Code context management.

Give compaction a purpose. In the welcome-email investigation, use /compact Preserve the current goal, change boundaries, relevant file paths, observed evidence, failed hypotheses, and the next unresolved question. This is a command with task-specific guidance, not a promise that every detail survives.

Before continuing, check that the summary still distinguishes observed, inferred, and not checked. “Email retries investigated” is a topic label. “Provider acceptance is known; the local acknowledgement path still needs inspection” is actionable state.

Compaction is lossy. Keep original evidence accessible and keep standing rules in project instructions, so a later step can recover the detail it needs.

Retrieve Evidence Just in Time

Fetch the source that answers the next question, then widen the search only if the result is insufficient. Anthropic's engineering post describes this as retaining lightweight references and using tools to load information when needed. Just-in-time context.

In Claude Code, file-reading and shell tools support that workflow. Ask it to locate the retry handler, read that handler and its caller, and inspect the matching error sequence. Where the dedicated Grep tool is enabled, it can return file paths, matching content, or counts instead of whole files. Claude Code tool reference.

For your own agent, return a source handle, a relevant excerpt, and enough metadata to establish scope: environment, record identity, version, or effective date. A later tool call can open the surrounding material.

Retrieval can miss relevant evidence. Keep a widening rule. If a narrow search finds nothing, change the scope or query before concluding the fact does not exist. If the excerpt cannot establish whether an exception applies, load the surrounding section.

Trim Tool Results Before They Enter the Conversation

Return the result needed for the next decision, with a way to recover omitted detail. Anthropic recommends filtering, pagination, range selection, and truncation for verbose tools, with helpful guidance when results are shortened. Anthropic's tool-design guidance.

Claude Code applies a concrete version of this pattern to large successful shell-command output: above the inline ceiling, it supplies a saved-file path and a preview, allowing the agent to search or read the file when necessary. The documented handling of failed commands differs, so do not assume every truncated error includes a recoverable file path. Shell output handling.

For the signup agent, a useful log result contains the matching event, the preceding operation, the error, and the source location. Repeated healthy requests can remain in the log file. Keep the error code and event identity exact; a prose description such as “something timed out” may erase the clue that decides the fix.

Blind truncation can remove the decisive exception. Make omission visible and provide a follow-up query or file reference. Trim repeated noise before trimming contradictory evidence.

Use Non-Fork Subagents for Clean Investigations

Claude Code subagents can perform a bounded investigation in a separate context window and return a result. A non-fork subagent starts fresh from its definition and supplied task; a conversation fork inherits the parent's history. Both separate subsequent work, but only the former gives you a clean starting context. Subagents and forks.

Delegate something like: “Inspect the email retry path. Do not edit files. Return the relevant paths, the evidence for duplicate delivery, competing explanations, and the next check.” Include the constraints the worker needs instead of assuming it knows the parent conversation.

Keep the return narrow enough to use: findings, evidence references, and uncertainty. A worker that returns its entire reading history simply moves the bloat back into the main conversation.

Workers make their own requests and consume usage. They may repeat work when their brief omits a decision already established by the parent. Use them when the investigation is separable and its intermediate output is disposable. Keep tightly coupled reasoning together. The setup details belong in Claude Code subagents.

Give Memory Files a Small, Stable Job

CLAUDE.md is a project instruction file, not a place to paste every finished conversation. Claude Code also has auto memory for learned notes. Its docs distinguish both from enforced configuration: putting a prohibition in a memory file does not make an action technically impossible. Claude Code memory.

For a small product team, keep repository-specific rules, genuine build and test commands, change boundaries, and pointers to deeper references in the standing file. Keep temporary investigation state in a separate task note. See CLAUDE.md team memory for the file structure.

A subtle limitation: importing a long document into CLAUDE.md loads that content too. Moving text into imported files can improve organization without reducing active context. For instructions needed only in part of a repository, consider the documented path-scoped rules. For reusable workflows, Claude Code skills normally expose their descriptions first and load their full instructions when invoked. Memory imports and scoped rules, skill loading.

Persistent files can preserve obsolete assumptions just as effectively as useful knowledge. Remove resolved incidents and superseded commands. Record a source or revision for facts that change. For the wider distinction between a session, working context, and durable storage, read agent memory explained.

How the Claude Agent SDK and Codex Apply These Moves

The mechanisms exist in named tools; your application still chooses the evidence and the task boundaries. Automatic context management cannot decide which business fact your workflow forgot to retrieve.

Claude Agent SDK: Put the Loop in Your Application

Claude Agent SDK provides the tools, agent loop, and context management behind Claude Code in Python and TypeScript applications. Use it when you are embedding an agent in a product or automation you operate. Agent SDK overview.

The SDK automatically compacts older history as the window approaches its limit and emits a compact_boundary system event. Log that event beside task outcomes so you can identify failures that appear after summarization. Non-fork SDK subagents keep their intermediate work in their own conversations and return their final message. SDK agent loop, SDK subagents.

Be explicit about project instruction loading. Current default query() options enable user and project sources. If you supply settingSources in TypeScript or setting_sources in Python, include project when you need the project's CLAUDE.md; an empty array prevents it from loading. This is controlled by setting sources, not by choosing the claude_code system-prompt preset. SDK instruction loading.

A capable loop still needs a domain-specific retrieval policy. A support agent still needs your definition of the current customer, applicable policy, and evidence required before it acts.

Codex: Separate Repository Instructions From the Running Chat

OpenAI Codex reads AGENTS.md instructions before work. It combines global guidance with project files from the repository root toward the working directory; later guidance takes precedence, and AGENTS.override.md takes priority at its directory. Codex project instructions.

Use that hierarchy to keep general build rules at the root and specialized rules near the relevant work. Codex CLI also provides /compact to summarize the visible chat and /init to create an instruction-file scaffold. Review the scaffold against your repository before treating it as authoritative. Codex commands.

A repository instruction file does not automatically capture the running task. Save the current goal, confirmed findings, pending work, and evidence locations separately. After a restart or compaction, the next request should make that task state available without replaying the whole investigation.

Context Engineering Examples: One Agent, Bloated to Lean

Cut the material around the decision while keeping the decision's dependencies. Consider a hypothetical coding agent asked to investigate duplicate welcome emails after signup retries. The file names and evidence below illustrate the design; they are not results from a deployed system.

Its bloated starting bundle contains the whole engineering handbook, broad repository dumps, logs from unrelated services, every earlier debugging exchange, and tools for unrelated administrative work. Its actual job is narrower: find the duplicate-send path, propose the smallest change, and identify a regression test. Schema changes are outside scope.

1. 1

   Keep the active task and standing constraints

   Replace the transcript's competing requests with the current goal and acceptance criteria. Keep the schema restriction visible. Expose the repository tools needed for this job and leave unrelated administrative tool definitions out of this run. Move stable repository conventions to the appropriate project instruction file; leave temporary observations in the task note. Do not put the entire handbook into an automatically loaded import.

2. 2

   Replace bulk files with an evidence map

   Retain pointers to signup-handler.ts, welcome-job.ts, the retry test, and the relevant log artifact. Those names are illustrative. Read the handler and worker when they answer the next question, and inspect their dependencies when required. The map makes evidence reachable; it does not substitute for reading it.

3. 3

   Filter the tool result around the failure

   Request the affected signup event and its retry sequence. Suppose the hypothetical record shows provider acceptance followed by a missing local acknowledgement and another queued attempt. Keep that sequence, event identity, and source location. Leave unrelated successful requests in the source log. Preserve uncertainty about whether the second attempt actually delivered.

4. 4

   Isolate a bounded lookup

   Give a non-fork worker the job of finding where duplicate-send prevention is implemented. Supply the task scope and relevant paths. Ask for locations and evidence, including an explicit statement if the search was incomplete. Return that finding to the main agent; keep the worker's exploratory reads outside it.

5. 5

   Compact the investigation into a usable handoff

   Keep the current goal, known evidence, rejected explanations, unresolved question, and next action. Preserve the distinction between a queued retry and a confirmed duplicate delivery. Archive the original transcript so the agent can recover a detail instead of guessing.

6. 6

   Build the next request from what remains

   Load the task, standing rules, relevant handler excerpt, filtered event sequence, and worker finding. The next action is to inspect the acknowledgement and duplicate-prevention path and propose the regression test. Load more evidence if that action requires it; do not reattach the whole archive by habit.

The resulting task note could read:

Goal: Explain duplicate welcome emails after signup retries and propose a minimal fix with a regression test.
Constraint: Keep schema changes out of scope.
Evidence: The selected log sequence shows provider acceptance, a missing local acknowledgement, and another queued attempt. Retain the event reference and original log location.
Unknown: Whether the queued retry delivered and where duplicate-send prevention applies.
Next check: Inspect the acknowledgement and duplicate-prevention paths in the referenced handler and worker.

This hypothetical note leaves the uncertainty visible. The model receives the supporting excerpt alongside it when making the diagnosis.

The result is a smaller content inventory, not a claimed token reduction. Exact token counts depend on the actual request and tokenizer. Lower spending and better completion rates require measurement; this example establishes what to remove and what to preserve.

A Context Engineering Template for the Handoff

Use a task note with these fields, filled with real evidence from the run:

• Goal and done condition: the current request and what would count as completion.
• Constraints: boundaries the next step must retain.
• Established facts: each finding paired with its source location and relevant revision or scope.
• Uncertainty: what remains inferred, contradictory, or uninspected.
• Next action: the next check and where its evidence can be retrieved.

Keep this note current when a finding changes. Appending every new thought without revising the old one recreates the original problem in a smaller file.

The Numbers: Context Size and Cost Need Different Checks

Optimize the cost of an accepted result, not the length of one request. A smaller main conversation can coexist with more worker calls, retrieval requests, or failed attempts.

Two documented instruction budgets show why units matter. Codex's default combined project-document cap is 32 KiB, which is 32 × 1,024 = 32,768 bytes. That is a byte limit, not a token budget. Claude Code recommends keeping each CLAUDE.md under 200 lines; that is writing guidance, not a hard context-window limit. Neither number predicts savings. Codex instruction limit, Claude Code writing guidance.

For a production workflow, record the cost of the main agent, workers, compaction, retrieval, and retries. Then calculate:

Cost per accepted task = total workflow cost across all attempts ÷ tasks that meet the acceptance criteria.

For a monthly budget, add model usage charges to tool, retrieval, storage, and hosting charges. There is no single monthly price for context engineering, and no defensible dollar saving without your workload and the applicable provider rates.

Caching complicates the comparison. OpenAI's prompt cache reuses matching rendered prefixes. Compaction can change that prefix and reduce cache reuse, even though the request becomes shorter. Keep reusable instructions stable where possible, and compare total cost before and after the change. A lower cache-hit rate alone does not prove an optimization failed. OpenAI prompt caching.

Use GPT-6 prompt caching diagnostics to separate problems caused by changing prefixes from problems caused by unnecessary content. A cache makes repeated input cheaper to process; it does not decide whether that input belongs in the task.

Context Engineering Best Practices: What to Change First

Act when the trace shows a context failure; choose the smallest intervention that addresses it. A trace is the recorded sequence of requests, tool calls, and results, including what the model saw before each decision.

For builders, inspect the step immediately before the wrong action. If the evidence is missing, fix retrieval. If it is buried in repeated output, filter the tool result. If earlier discussion overrides the current task, repair task state and the handoff. If a separable investigation floods the main conversation, isolate it.

For operators, define an accepted outcome and preserve representative failures for evaluation. Compare completion quality, unsupported claims, retries, latency, and total cost on the same tasks. Change one context policy at a time so the result remains interpretable. Use existing traces to design offline evaluation cases.

For buyers, ask whether the product exposes the context and tool results behind an answer, preserves source references after summarization, and lets you control persistent instructions. A “memory” feature without visibility into what gets loaded is difficult to diagnose.

Wait on additional machinery when a short, bounded workflow already behaves reliably. You may need a clearer instruction or a better tool error, without a memory service or worker network. If a failure comes from incorrect permissions, broken application logic, or unavailable source data, address that cause directly; context curation alone cannot repair it.

What's Overhyped About Context Engineering?

A larger context window, more memory, and more agents are capacities, not evidence of better decisions. The useful question is whether they help this workflow carry the right information to the next step.

“Put everything in the prompt” avoids choosing, but introduces competing material. “Summarize everything” chooses too aggressively if it erases the exception that matters. “Let a subagent handle it” only helps when the task can cross that boundary without losing essential decisions.

The most expensive mistake is confusing a shorter request with a validated system. An agent that cheaply answers from the wrong policy is still wrong. A good context policy should make the answer easier to verify: the governing rule is visible, the supporting evidence has a source, and unresolved uncertainty remains explicit.

Questions From a Builder Whose Agent Drifts

Will context engineering fix an agent that works in a demo but fails in production?

It can address missing evidence, stale instructions, excessive tool output, and lost task state. First inspect the failing request. A broken tool, incorrect application logic, or a task beyond the model's capability needs its own fix. Judge the change by accepted outcomes on representative tasks.

Do I need a vector database or a multi-agent framework to start?

No. Begin with explicit project instructions, focused file or document retrieval, bounded tool results, and a task handoff. Add semantic search when finding relevant material requires it, or a subagent when a self-contained investigation benefits from separate context.

How do I know I trimmed too much?

Check whether the next decision still has its governing rule, supporting evidence, applicable scope, and important exceptions. If the agent must guess a removed fact, the context is too lean. Keep a retrievable source and widen the read when the excerpt does not settle the question.

The Monday move: take a failed run you already have and inspect the request before its first wrong action. Label its contents as standing instructions, active task state, evidence needed now, or material that can remain in storage. Make the smallest corresponding change, then compare the outcome and total workflow cost on the same evaluation cases.

Join the newsletter for practical guides to building and operating useful agents.