Context Engineering for AI Agents: Stop the Forgetting

TL;DR

Context engineering is the practice of controlling what an AI agent sees at each step, not just what you tell it once. Most agent failures come from context pollution, not weak models. Three techniques fix the bulk of them: compaction, structured note-taking, and multi-agent splitting. You can apply all three this week without changing your model or your framework.

Why Your Agent Forgets What Matters

You give an agent a clear task. It starts strong, then drifts. By step twelve it ignores instructions you wrote at step one. The model did not get dumber. The context window filled up with noise, and the important instructions got pushed out or buried under tool output.

This is the core problem context engineering solves. The term describes how you shape, filter, and route the information an agent receives at every turn. Prompt engineering asks “what do I say once?” Context engineering asks “what does the agent need to see right now, and what should it never see again?”

Anthropic’s engineering team published their internal findings on this in 2025, noting that agents working across long horizons fail mostly from context pollution, not reasoning limits. The fix is structural, not a better model. The techniques below come from that work and from teams running agents in production at scale.

Also Read:

Why Long Context Breaks AI Coding Agents

The cost of ignoring this is real. A competent agent that loses its own instructions mid-run looks incompetent to the user, and the usual response is to blame the model. Swapping models rarely helps. The window management does.

The Three Failure Modes

Before the fixes, name the enemies. Most agent breakdowns trace to one of three patterns, and each needs a different countermeasure.

Loss. Early instructions scroll out of the window as the conversation grows. The agent literally cannot see the rule anymore. This hits long tasks hardest, where the setup happens in step one and the consequence appears in step thirty.

Also Read:

AI Tools That Clean Messy Spreadsheets

Distraction. The window fills with tool outputs, error logs, and irrelevant retrieved chunks. The signal drowns in noise, and the agent starts answering from the loudest context rather than the right one.

Contamination. One bad step writes wrong conclusions into shared memory, and every later step inherits the mistake. Sourcegraph’s team describes agents as especially vulnerable here because a single polluted retrieval can cascade through an entire run and corrupt everything downstream.

Technique 1: Compaction

Compaction means summarizing the conversation on a schedule so the agent keeps the gist without holding every token. Instead of feeding the full transcript back each turn, you compress older exchanges into a tight summary and prepend it.

Also Read:

7 Free AI Apps That Replace Paid Subscriptions

The practical version: every N steps, ask the model to write a short recap of what changed, what decisions were made, and what is still open. Discard the raw history beyond that point. The agent keeps momentum without paying the full token cost of the whole session.

Compaction trades some detail for stability. The trick is to keep the recap structured: status, decisions, open questions. A flat blob of prose loses the structure you need later, and the agent re-derives context it already resolved.

Also Read:

AI Voice Transcription Tools That Actually Save Time

Most frameworks expose this through a context manager or a summarizer hook. If yours does not, a scheduled prompt that reads the last K messages and returns a structured summary is enough to start. You can read the broader case for this approach in Anthropic’s writeup on effective context engineering for AI agents.

Technique 2: Structured Note-Taking

Rather than cramming everything into one rolling transcript, give the agent a scratchpad. A dedicated notes file or memory object holds the facts that matter: the user’s stated goal, constraints discovered mid-run, and intermediate conclusions.

This separates two kinds of context. The transcript is volatile and noisy. The notes file is curated and stable. The agent reads the notes at each step and writes to them when something important changes. This is the same split humans use when we keep meeting notes instead of replaying the whole meeting from memory.

Packmind’s guidance for dev teams makes the point concrete: use brief prose to explain the why behind a convention, because rationale helps the agent apply rules in edge cases. Use code blocks for the what. A “Preferred versus Avoid” block with real code beats a paragraph of explanation every time.

The notes file also gives you an audit trail. When the agent makes a wrong call, you can read the note it was working from and see exactly which assumption broke. That visibility is impossible when everything lives in an opaque transcript.

Technique 3: Multi-Agent Splitting

Some tasks should never share one context. When you run a research step and a writing step in the same window, the research noise contaminates the writing, and vice versa.

The fix is to spawn sub-agents with their own isolated contexts. A research agent gathers and filters sources, then hands a clean summary to a writer agent that never saw the raw dump. Each agent stays focused because its window holds only what its job requires.

Anthropic’s own agentic system uses this pattern: a lead agent orchestrates, and specialized sub-agents handle retrieval or analysis in isolation. The lead only sees refined output, not the scraped pages behind it. LangGraph and similar frameworks make this stateful, auditable split easier to build, and the pattern shows up across most production agent stacks.

Isolation has a cost. Handoffs can lose nuance, and a sub-agent that lacks the full picture may make a locally safe choice with globally bad effects. Keep the handoff contract explicit: define exactly what the sub-agent receives and what it must return.

Context Engineering vs Prompt Engineering

The two are not rivals. Prompt engineering is a subset. You still write good prompts. Context engineering asks what those prompts sit next to, and what gets stripped before the agent reads them.

The comparison matters because teams often hire for prompt skills and stop there. The harder discipline is the system that feeds the prompt, and that is where most reliability gains now live. For background on the framing, see the context engineering guide from Anthropic and the practical lab notes from Sourcegraph’s engineering team.

Practical Setup You Can Ship This Week

You do not need a new framework to start. Three changes cover most cases, and each takes an afternoon to wire in.

First, add a compaction step to any agent that runs more than ten turns. Summarize every five to eight steps. Keep the recap short and structured, and prepend it rather than appending it where it competes with fresh instructions.

Second, give the agent a notes file it reads at the start of every step and writes to when facts change. Treat it as the source of truth, not the transcript. Version it if you can, so a bad write is recoverable.

Third, split any workflow where one stage produces heavy output the next stage does not need to see raw. Run the heavy stage in a sub-agent, pass only the cleaned result forward, and keep the contract tight between them.

If you already use MCP to connect AI tools to your data, context engineering is the next layer. MCP feeds the agent; context engineering decides what stays visible. Teams that combine clean tool access with disciplined context routing see fewer mid-run failures. For multi-step reliability, prompt chaining pairs well with compaction, since each chain link gets a fresh, trimmed window instead of an ever-growing one.

Common Mistakes to Avoid

Dumping everything into the prompt feels safe but backfires. Retrieving fifty chunks and hoping the model finds the right one loses to retrieving ten and re-ranking to five. Sourcegraph’s testing showed a two-stage retrieve-then-rank pipeline beats a single wide dump almost every time, because the second stage removes the noise that would otherwise bury the answer.

Another mistake is treating the transcript as memory. The transcript is evidence, not truth. Write conclusions to the notes file explicitly, or the agent re-derives them from noisy history on every step and drifts toward the loudest context.

Security matters too. A polluted or poisoned context can steer an agent into bad actions, especially when context comes from untrusted sources. Review your agent security risks before wiring external context into a live system. When an agent gets stuck in a loop, debugging the loop often means checking what context it actually sees, not the code around it. Memory platforms like Mem0’s context engineering guide go deeper on long-term memory patterns if your agents need to recall across sessions.

FAQ

Is context engineering only for coding agents? No. Any agent that runs multiple steps benefits. Coding agents are just the easiest lab because failures show up fast in test suites, but the same techniques apply to research, support, and ops agents.

Do I need a vector database? Not to start. A notes file and a compaction step cover most single-agent cases. Add retrieval when your agent needs knowledge it cannot hold in the window, and keep the retrieve-then-rank pattern to avoid noise.

How often should I compact? Every five to eight steps for long runs. Shorter agents may not need it at all. Watch for quality dropping after long stretches, then add compaction at the point where the drop begins.

Final Thoughts

Your agent is only as good as what it can see at the moment it acts. Context engineering is the discipline of keeping that view clean, focused, and honest. Start with compaction and a notes file, split heavy stages into sub-agents when contamination appears, and your agents will stop forgetting the rules you set at the start.

Want a repeatable workflow to build on? Read our guide to spec-driven development with AI agents to see how structured context turns into shipped results, and check LangChain’s engineering blog for production agent patterns that put these techniques into practice.