Context Engineering: The Art of Managing AI's Working Memory

Learn how to substantially cut context token usage without losing quality. This guide covers the fundamentals of context management in AI-powered development: why LLMs are stateless, how context grows naturally over time, and four battle-tested strategies to keep your AI workflows efficient. Master conversation management, sub-agents, external knowledge systems, and context layering to build AI systems that scale.

It's 3 PM and your agent failed again — after reading irrelevant files, burning 180k tokens, and producing broken code. More context made results worse because the model drowned in noise. This is the context crisis: developers assume more context = better performance, but that's wrong.

In our last article on zero‑config Claude Code plugins in Devcontainers, we showed how we built a language-agnostic plugin system. The key insight? Convention over configuration through context separation. We separated generic plugin instructions from project-specific conventions, loading only what was needed when it was needed. This wasn't just a design preference — it was a fundamental requirement driven by how AI context actually works.

But what exactly is context, and why does it matter so much? Let's dive deep into the mechanics.

This is critical to understand: Every single message you send includes the ENTIRE context from scratch.

The LLM doesn't "remember" your previous conversation. It doesn't have persistent memory. Each request includes the full context assembled by the client: system prompt, tools, memory files, message history, and your new input. The model processes that request and returns only a response. Then it forgets everything.

Think of it like calling a stateless API endpoint. Every request is independent. When you send your next message, you're not continuing a conversation — you're starting fresh, but with the previous exchange included in the new request. 

What This Means:

Context naturally grows over time. Every message adds to the history. Every tool result adds more data. Every file you reference expands the context window.

For Sonnet 4/4.5, the usual context window is ~200k tokens — and that's a hard ceiling for those models. Think of it like RAM for a computer. Other models/versions may differ.

Here's a real example from Claude Code's /context command (Sonnet 4.5 with a ~200k-token window; values vary by model/version):

This is a healthy context profile. 61% free space means there's plenty of room for the AI to work.

Context grows through four primary mechanisms:

Every action adds to context and accumulates across turns - what you send is exactly what the LLM sees.

Context pollution isn't just an annoyance - it's a performance killer. When your context window fills with irrelevant information, everything degrades.

Context pollution is expensive. Every irrelevant token costs you speed, accuracy, reliability, and money. Optimize ruthlessly.

You can't optimize what you can't measure. Context observability is critical for understanding where your tokens are going and identifying optimization opportunities.

Claude Code provides the /context command for real-time visibility into token distribution. This is your context dashboard - use it frequently to monitor system prompt, tools, memory files, messages, and free space.

Focus on context quality over quantity. You could be at 90% free space with irrelevant content and get poor results, or at 40% with highly relevant context and get excellent results. 

Proactive Context Management:

Don't let context pressure build until it becomes a problem. Manage proactively, not reactively.

Now that we understand what context is and why it matters, let's explore practical strategies for managing it effectively. These aren't theoretical concepts - they're battle-tested patterns from building production AI workflows at Zenity.

Pattern: Actively remove irrelevant history. 

Long conversations accumulate cruft. Early messages become irrelevant as the conversation evolves. Prune aggressively.

Implementation:

Use /clear for fresh starts when switching tasks, or /compact to compress while preserving key context (you can add optional instructions like /compact focus on the API changes). Design workflows that complete in fewer conversational turns to minimize accumulation.

Understanding Autocompact:

Claude Code automatically compresses context at ~155k tokens (~77.5% usage) for Sonnet 4/4.5, leaving a 45k safety buffer. You can disable this with /config set autocompact false but must then manually manage context with /clear or /compact before hitting the hard limit.

Pattern: Delegate specific operations to context-efficient sub-agents.

Monolithic agents carry everything - all tools, all context, all capabilities. This is inefficient. Instead, create focused sub-agents that specialize in specific tasks, then delegate work to them. For example, a Utils Agent handles Git/JIRA/GitHub operations with specialized tools, completes the task, and returns only a brief summary to your main agent.

Critical Advantage: Each sub-agent runs in isolated context that's cleared on exit. For example, Utils Agent loads specialized Git/JIRA/Python tools (15-40k tokens), executes commands, and returns only essential results (~50-200 tokens) to your main conversation. Your main agent never carries that tool overhead.

Pattern: Store large reference datasets outside the context window, query on-demand.

Not everything belongs in context - even dynamically loaded. For massive documentation sets, historical decisions, or organizational knowledge that spans thousands of pages, use external systems with semantic search or structured queries. 

When to Use External Systems vs Dynamic Loading:

Use MCP servers like Context7 to fetch library documentation on-demand, or connect to Confluence/ClickUp/etc for organizational knowledge. Instead of loading 50-100k tokens of comprehensive docs, query semantically and bring in only the 2-3k tokens relevant to your current task.

Pattern: Organize context into tiers - keep essentials small, load details on demand.

The temptation is to frontload all possible information into system prompts or memory files. Resist this urge. Instead, use a three-tier architecture that keeps always-loaded context minimal while preserving access to comprehensive information.

The Three-Tier Architecture:

Implementation example:

Instead of embedding everything in your system prompt (which could be 53k+ tokens), use a three-tier approach: keep your system prompt generic (~500 tokens), store essential project conventions in CLAUDE.md (~4-6k tokens), and fetch detailed documentation only when implementing specific features. This reduces your always-present context from 53k to under 7k tokens. 

Real-World Application:

Our Planner agent demonstrates this approach: it always loads core conventions from CLAUDE.md (~4k tokens) but fetches language-specific documentation only when implementing features in that language. Same agent, same plugin, but only relevant detailed documentation is loaded per task.

Key Benefits:

Context engineering is a core competency for AI-powered development. Just as we optimize algorithms for complexity, we must optimize AI workflows for context efficiency.

Context engineering is the difference between AI tools that feel magical and AI tools that feel frustrating. Between agents that consistently deliver and agents that randomly fail. Between workflows that scale and workflows that collapse.