The anatomy of an agent
In short An agent is a model in a loop plus seven things around it. The loop, memory and toolbox are the spine; the rest is where I rebuilt the most.Full verdict ↓
I set out to build a multi-agent system that runs on its own. This series takes it apart one component at a time, in the order I built them, with what I tried, what I deleted and what already exists. This post is the map.
What an agent is
A language model only produces text. An agent is that model in a loop where it can also act:
messages = [user_request]
while True:
reply = model(messages, tools) # decide the next step
if not reply.tool_calls: # no tool asked for: done
return reply.text
for call in reply.tool_calls: # the harness does the work
messages.append(run(call)) # the result goes back in
Everything else in this series is what it takes to make that loop remember, reach the world, survive failure, run safely, act unprompted and be trusted. The picture below is the design as a concept. The posts that follow go into how each piece is actually built.
The design
What each component answers
| Need | Without it | Component | Post |
|---|---|---|---|
| Remember across sessions | It forgets me every morning | Memory | 02 |
| Reach the world | It can only talk | Tools | 03 |
| Survive failure and interruption | A crash loses the task | Harness | 04 |
| Know what to read and how to work | The prompt overflows, or it improvises every time | Context and skills | 05 |
| Run code without trusting it | One bad program owns the machine | Runtime and sandbox | 06 |
| Act without being asked, and stay in bounds | It only reacts, or runs away | Triggers and governance | 07 |
| Be trusted | No way to check it | Evals | 08 |
The order I built it in
- Memory. Started here, because an agent that forgets is a chatbot.
- Tools. One hub for tools, skills and logins, so no agent holds a credential.
- Harness. A durable control loop around the model, extended from an existing open-source agent.
- Context, skills, triggers. What the model reads, procedures it follows, and how it comes back unprompted.
- Evals. A suite that checks the one decision everything depends on.
- Runtime and sandbox. The isolation I had just built was replaced by something lighter, the same day.
- Journeys. The 9/24 ceiling was the instrument, not the model. A fake world now checks what the agent does, not just its first decision.
Six rules I kept
| Rule | What it means | Where it shows |
|---|---|---|
| The model steers, the harness supports | No classifier, router or planner decides how a request is handled. Code guards only irreversible actions, spend, context and recall | Harness |
| Leverage, don't rebuild | Delegate to tools and engines that already do a thing well | Harness, tools |
| Everything durable | State lives in a database, never in a process | Harness, memory |
| Every belief cites its source | A summary can be traced to the words it came from, and forgotten with them | Memory |
| The agent never holds the keys | Credentials live with the tools, not in the agent or its sandbox | Tools, sandbox |
| Measure, then delete | If a component does not earn its keep in a test, it goes | Evals, the finale |
What already exists
I surveyed each area before building. Each post gives the detail; this is the map.
| Component | What exists | What I did |
|---|---|---|
| Harness | Agent frameworks (LangGraph, OpenAI Agents SDK) and durable runtimes (Temporal, Restate, Inngest) | A durable runtime for the loop, my own control logic on top |
| Memory | MemGPT/Letta, Mem0, Zep, A-MEM, Hindsight | Followed Hindsight's operations, added layers and provenance |
| Tools | MCP servers, gateways and registries, tool search, code mode | One hub with search, connections and typed code |
| Runtime and sandbox | Containers, microVMs, user-space kernels, WebAssembly | One execution interface, after two rewrites |
| Coding work | Claude Code, Codex and other coding agents | Hosted, not rebuilt |
Where it stands, and my verdict
| Component | Status | |
|---|---|---|
| Memory | Pipeline decommissioned once, rebuilt in layers | rebuilt |
| Tools | Running, with one built-then-removed feature | stable |
| Harness | Running; router and planner removed | stable |
| Context and skills | Built; skills rewritten once | stable |
| Runtime and sandbox | Replaced within a day of the first build | churning |
| Triggers and governance | Waking redesigned three times, now small | churning |
| Evals | Four suites; the 9/24 ceiling was a measurement fault, fixed; journeys now measure whole tasks | rebuilt |
Verdict: the spine held and the edges moved. The harness, memory and tools kept their shape through every rewrite. Runtime, sandbox and waking were redesigned repeatedly. Evals was the thinnest part and is now the most changed: its first ceiling was a measurement fault, and the journey suite is the first test that checks what the agent actually does. Next: memory, the first thing I built.