Context Engineering for Agents: Building the Context Layer

This is where context engineering becomes useful.

Anthropic describes context engineering as managing the full set of information available to the model during inference, including instructions, tools, external data, and message history. For agents, this curation happens repeatedly as the agent runs rather than only once when we write the original prompt.

In this article, I will use a slightly more practical definition:

Context engineering is the process of deciding what information, state, and capabilities the model should see at each step of an agent run.

The important part is at each step.

Agent context is not something we can create at the beginning of the task and forget about it. It changes while the agent searches, reads, uses tools, loads skills, interacts with integrations, and learns more about the task it is trying to complete.

Many sources, one context window

Consider a small example. We are building an agent that researches a company and prepares a sales brief.

At the beginning, the agent only has a goal:

Research Trugen AI and prepare a short sales brief.

It searches the web and gets several results. It opens the company's website. It checks the CRM and discovers that the company already exists there. It then reads the previous sales notes.

Within only a few steps, we have accumulated the goal, search results, website content, CRM data, previous account history, tool results, and whatever conversation has happened so far.

If our implementation simply keeps appending everything, context grows every time the agent does something.

This is the first important problem: every capability we add to an agent creates another potential source of context, but all of those sources eventually compete for the same working spaceHaving a larger model context window helps, but it does not remove the need to decide what deserves to be there. Context is limited, and adding more tokens can produce bad results if the useful content gets buried inside irrelevant bloat.

Available information is not current context

I find it useful to separate two things:

what the agent can access and what the model can currently see.

An enterprise agent might technically be able to search years of Slack messages, thousands of documents, customer CRM records, an entire codebase, and long-term user memory. A general-purpose agent might also have access to the open web, a browser, a terminal, and hundreds of reusable skills.

This larger collection is the agent's memory environment.

The model's context is only a temporary working set selected from that environment.

you can put it like storage versus working memory. A computer does not copy everything on its disk into RAM whenever an application needs one file. Similarly, making more information accessible to an agent should not automatically mean making more information visible to the model.

The goal is not to limit what the agent can eventually know. The goal is to keep the model focused on what it needs right now.

Context changes as the agent works

The next important point is that context is not built once.

Let’s continue with the Trugen example we talked before. At first, the task goal is the important piece of context. After the web search, the search results matter.

Once the agent finds and reads the official website, most of those original search snippets may not be useful anymore. Then the agent checks the CRM. Now the previous opportunity and sales notes become important. Later it searches recent news and discovers that Acme launched a new product last week. Some of the older assumptions may now be less useful.

The working context changes as the task moves forward.

simply put, Some information should enter the context. Some should leave. Some should be replaced by newer information. Some should only stay around for a few steps.

A browser page is a good example.

The agent observes a page, clicks a button, and reaches a new page.

The old browser observation may now be useless.

Memory, state, skills, and tools are inputs to context

This is where a lot of agent architecture gets confusing because several different systems are often grouped together.

I find it easier to give each one a simple job.Memory is information we keep so the agent can use it later, State tells us where the current task is now, History tells us what happened previously, Skills contain reusable instructions for how to perform a type of task and Tools give the model capabilities it can call.

And context is the part of all of this that the model gets to see for the current step.

All of those facts can remain in memory.

If the current question is:

What CRM does Trugen use?

we probably retrieve only:

Trugen uses Salesforce.

Later, if the agent needs to run a demo, another memory may become useful.The same idea applies to skills, agent may have hundreds of skills available:

Research
Product Demo
Salesforce
Deployment
PDF Analysis
Spreadsheet
Incident Debugging
...

There is no reason to load all of their instructions into every prompt.

The model may initially see only a small skill index. When the task requires a product demo, the harness loads the full Product Demo skill.

Tools have a similar problem. An agent may have 100 tools, but a research task may only need web search, URL reading, and CRM lookup.

A browser task may need observe, click, type, and navigate.

A scheduling task may need calendar and CRM tools.

So the question is not only:

What should the model know?

It is also:

What should the model be able to do right now?

Retrieval gives candidates, not final context

A lot of context engineering discussions quickly become RAG discussions. Retrieval is important, but it is only one part of the problem.

Suppose someone asks:

Does the product support Salesforce?

Search returns several documents.

One is an old roadmap. One is yesterday's integration guide. One is an internal discussion. Another is a marketing blog post. All of them may be semantically related.But they should not all be treated equally.

The context layer may need to consider whether the information is fresh, which source is more trusted, whether the user is allowed to see it, and whether it matters to the current task. Sometimes one current and trusted paragraph is better context than five complete documents.

When context grows: clean first, compact second

Eventually, an agent will collect too much context. The obvious solution is to summarize it.

But before doing that, there is a cheaper question, What can we simply remove?

A lot of agent context is easy to recreate. The agent may have an old browser snapshot from three pages ago. It may still have thousands of lines of terminal logs even though it already found the useful error. It may have read the same file twice, or kept search results after opening the page it actually needed.

So the first step should usually be simple cleanup.

We can remove stale observations, duplicate reads, oversized tool output, or search results that have already served their purpose.

I like to think of this as context hygiene.

Hermes-agent is a great example here. When its context starts getting large, it does not immediately ask another model to summarize everything. It first removes large old tool outputs that are no longer useful to the recent working context.

This is also why tool design matters. If a file-reading tool returns 40,000 tokens every time it runs, the context layer is already fighting an unnecessary problem.

A good harness controls context at the source, not only after it reaches the prompt. "Compaction should happen before the context breaks", If cleanup is not enough, then we need compaction.

The harness should ideally track how full the context is instead of waiting for a context window exceeded error. 

Conceptually:

                   current input size
Context pressure = ──────────────────
                    context capacity

When pressure is low, the agent can keep more detail. As it rises, we can remove stale data and duplicates. At higher pressure, we compact older history. The important point is that compaction should be normal runtime behavior, not an emergency recovery step. "Keep the beginning and the recent end" Not every part of the run has the same value.

The beginning often contains the original goal and important details .The recent end contains the latest actions, observations, and unfinished work. The long middle is usually the safest place to compress.

Compact for continuation, not for retelling

There is a big difference between:

Summarize what happened.

and:

What does the agent need to know to continue correctly?

Suppose an incident-response agent learned that the database is healthy, Redis connections are saturated, restarting a staging worker improved latency, production is untouched, and approval is still pending.

A weak summary would be:

The agent found a Redis issue.

A better compact state would preserve:

Goal:
Fix payment failures.

Evidence:
Redis connections are saturated.
Database looks healthy.

Actions:
Restarted one staging worker.

Result:
Staging improved.

Constraint:
Production untouched.

Next:
Get production approval.

That gives us a useful rule: Compress narrative, preserve the state.

If the context grows again later, we can update this compact state with the new work instead of creating a completely fresh summary every time. Re-acquire detail when needed. Compaction will always lose some detail. That is okay if the original information is still available.

If the compact state says Redis caused the issue and the agent later needs the exact error message, it can read the logs again.

So the full lifecycle becomes:

Acquire
   ↓
Use
   ↓
Clean
   ↓
Compact
   ↓
Re-acquire if needed

This is much more useful than trying to keep every token forever.

Cache discipline matters too

This gives us a simple rule:

Put stable information earlier and frequently changing information later.

Small implementation details matter here.

If the tool list did not change, keep it in the same order. but with dynamic tool selection this is a drawback there is two way you can handle this one is do a hybrid loading where you load all the few core tools like web search and stuff like that and load remaining of them selectively other approach is go for profiles like Research Profile (Web Search, Read, URL, CRM, Lookup) Browser Profile (Observe Click, Type, Navigate) Scheduling Profile (Calendar, CRM, Contact Lookup) depending on your need you go for one of these approaches to get max cache hit rate.

Putting it together

Product Question
      ↓
Product Context

Qualification / Demo
      ↓
CRM + Browser + Skills

Meeting
      ↓
Calendar + CRM + Summary