The toolbox: one door to every tool
In short One door that holds every login, a typed catalog the model searches, and code instead of one tool call per turn. The agent never holds a secret.Full verdict ↓
A model that can only talk is not an agent. Reaching the world means tools, and the more of them you connect, the worse it gets: definitions fill the prompt and the model picks badly. The toolbox is my answer, built second, right after memory.
The design
The rule underneath is the one I trust most: the agent asks, the toolbox holds the keys. Credentials, OAuth tokens and per-service headers live in the toolbox and are attached when it calls a service. The agent, and the sandbox its code runs in, never see one.
The journey
- A searchable registry. Each service described by a manifest, every tool stored with an embedding so the agent finds one by meaning.
- Skills joined it. Procedures are stored and searched the same way as tools.
- OAuth upgrade. Connections sign in once and refresh, so a login belongs to the toolbox and not to an agent.
- More connections. Mail, search and others, each with its own sign-in. A tool named in a query is always returned first.
- A cleaner idea. The toolbox is the agent's device: find a capability, use it, let the device hold the sign-in.
- Built and removed in a day. A second door for code to call tools directly, with signed tokens. Removed because the executor has no network at all.
- New connections and usage. A cluster connection, a browser connection, and a record of every proxied call.
Decisions that held
| Decision | Why |
|---|---|
| A catalog, not a dispatcher | Every operation is a real typed entry with its own schema and hints (read-only, destructive, open-world). Nothing hides behind a generic call |
| The model writes code against the catalog | One model turn per call is slow. A short program loops, filters and joins, and returns only what it needs |
| The toolbox owns credentials | The agent and its sandbox never hold a secret, so a leaked prompt or program cannot leak one |
| Search belongs to the harness | The toolbox serves the catalog at three detail levels. The harness decides what the model is shown |
| A tool's definition is hashed | What the agent sees cannot change silently. The hash exists today; holding a changed tool until I approve it is not built yet |
What already exists
The idea that tools should not all sit in a prompt is now widely shared. Four approaches, and where mine fits:
| Approach | What it solves | What it leaves |
|---|---|---|
| MCP servers | A standard way to expose tools | You still need to find, secure and combine many of them |
| Gateways and registries (ContextForge, Docker, Microsoft, Agentic Community registry) | Aggregate many servers behind one endpoint, with auth and governance | Mostly about routing and control, not about what the model sees |
| Tool search | Loads tool definitions on demand | Intermediate results still pass through the model |
| Code mode (Cloudflare, Sep 2025; Anthropic, Nov 2025) | The model writes code against a typed API, so intermediate data stays out of its context | Needs a sandbox to run the code |
| Mine | A catalog with detail levels, logins held in one place, code run in a sandbox with no network | Code mode is built but not yet evaluated on real requests |
The research behind it is consistent: the CodeAct paper reports up to 20% higher success and 30% fewer actions when a model acts by writing code. I have not reproduced that on my own workload.
Where it stands, and my verdict
| Component | Status | |
|---|---|---|
| Memory (post 2) | Built, traceable; two memory journeys pass | thin |
| Catalog, search and logins | Running, with usage and audit | stable |
| Tool discovery and chaining | Journey suite: 27/27 on the 35B, 6/27 on the 8B; a flaky tool, a down tool and a missing tool are all in the set | measured |
| Code against the catalog | Built behind a switch, not evaluated on real requests | unproven |
| A second door for code | Built and removed in one day | deleted |
| Holding a changed tool definition for approval | Hashing is built, the hold is not | later |
Verdict: the door and the key-holding are right; the code idea is still a bet. Keeping secrets out of the agent has held up under every redesign. The journey suite now measures the door itself: the 35B finds hidden tools, chains them and retries a flaky one; the 8B mostly cannot. The claim that code beats one-call-per-turn still rests on other people's results until I measure it on my own requests. Next: the harness, the loop that decides when to call any of this.