Agent Harness Engineering

The previous section explained what AI agents are: systems where a language model runs in a loop, calling tools and reasoning over results until a goal is achieved. But that description glosses over a critical question — who runs the loop?

The language model does not run itself. It cannot call its own tools, manage its own memory, enforce its own safety rules, or decide when to stop. Something else must do all of that. That something is the agent harness: the infrastructure code that wraps the language model and turns it from a text-completion engine into an autonomous agent.

If the agent is the pilot, the harness is the cockpit — the instruments, the controls, the autopilot logic, and the emergency overrides. A skilled pilot in a bad cockpit crashes. The same model behind a well-engineered harness versus a poorly-engineered one produces dramatically different results.

This chapter covers how to design and build that harness: the execution loop, tool dispatch, context management, guardrails, observability, and the production concerns that separate a working demo from a reliable system.

Why This Chapter Matters: When the Harness Breaks, the Model Looks Broken#

In April 2026, users across the internet reported that Claude Code had become noticeably "dumber." Responses were shallower, the agent forgot context mid-task, and tool choices felt random. Many assumed Anthropic had degraded the model. They hadn't. The postmortem revealed three harness-level bugs — none involving the model itself:

1. A thinking cache bug (deployed March 26, fixed April 10): A caching optimization malfunctioned, causing the system to clear the model's chain-of-thought reasoning on every turn instead of only once on session resumption. The agent was effectively lobotomized — forced to reason from scratch each iteration while appearing to have full context. Users experienced forgetfulness, repetition, and bizarre tool choices.

2. A reasoning effort downgrade (March 4 – April 7): The default reasoning effort parameter was silently shifted from high to medium as a latency optimization. Internal benchmarks showed marginal quality difference; real users noticed immediately. The lesson: latency gains that trade away intelligence are invisible to automated evals but obvious to humans doing real work.

3. A system prompt constraint (April 16 – April 20): A single instruction — "keep text between tool calls to ≤25 words" — caused a measurable 3% drop in coding evaluations. A few words in the prompt, added to reduce verbosity, inadvertently constrained the model's ability to reason through complex code changes.

These incidents illustrate the central thesis of this chapter: the harness matters as much as the model. A frontier model behind a buggy harness — one that silently drops reasoning, misconfigures parameters, or routes requests to wrong infrastructure — will underperform a smaller model with a well-engineered harness. The model provides intelligence; the harness provides the conditions under which that intelligence can actually express itself. When the harness breaks, it does not throw an error. It makes the model look broken. And that is far harder to debug.

The Execution Loop#

Every agent harness has a central loop at its core. The model generates output; the harness inspects it; if the output contains tool calls, the harness executes them and feeds the results back; if the output is a final answer, the loop terminates. This is the heartbeat of every agent system, regardless of framework.

Here is the loop in pseudocode — the pattern is identical across OpenAI Agents SDK, Vercel AI SDK, CrewAI, and Claude Code:

messages = [system_prompt, user_goal]
step = 0

while step < max_steps:
    response = call_llm(messages, tools)

    if response.has_final_text():
        return response.text            # Done

    if response.has_handoff():
        active_agent = response.target   # Switch agent
        messages.append(handoff_context)
        step += 1
        continue

    for tool_call in response.tool_calls:
        result = execute(tool_call.name, tool_call.args)
        messages.append(tool_result(tool_call.id, result))

    step += 1

raise MaxStepsExceeded()

The step limit is the most important safety mechanism in the entire harness. Without it, a confused or hallucinating model will loop until you hit an API rate limit or run out of budget. Every production harness must have one.

Tool Dispatch#

The execution loop calls tools, but how tools are dispatched, validated, and sandboxed is a design problem in its own right. The harness is responsible for three things: translating the model's tool call into an actual function execution, validating the inputs before running anything, and returning the result in a format the model can understand.

Tool Design Principles for Harness Engineers#

The AI Agents chapter covered tool design from the model's perspective — how to name and scope tools so the model uses them correctly. From the harness engineer's perspective, there are additional concerns:

CLI Is All You Need#

Before diving into the remaining harness components — context management, guardrails, observability, error handling — it is worth stepping back to ask a more fundamental question: what interface should developers use to interact with an agent? The answer, for coding agents specifically, is overwhelmingly the command line.

LLMs are text-native. They consume text and produce text. The terminal is the purest text interface available — there is no impedance mismatch between what the model generates and what the developer sees. A GUI adds a translation layer that must render, re-render, and interpret model output; a CLI simply prints it. This directness is not a limitation — it is a feature.

The convergence is striking: Claude Code, Codex CLI, and Gemini CLI were all built by different organizations with different models, yet they all arrived at the same CLI-first architecture. This is not coincidence — it is the natural consequence of building agents on top of text-native models for a developer audience that already lives in the terminal.

The practical implication for harness engineering: design your agent as a CLI tool first, then add GUI layers on top. The CLI forces you to get the core loop, tool dispatch, and context management right without hiding problems behind a visual interface. If the agent works well in a terminal, adding a web UI or IDE extension later is a presentation concern — not an architecture change.

Context Management#

Context is the scarcest resource in an agent system. Every tool result, every reasoning step, and every piece of conversation history consumes tokens from a finite window. When the window fills up, the model either loses access to earlier information, or the harness must intervene to compress or evict content. How the harness manages this resource determines how long and how complex a task the agent can handle.

Practical Context Management Patterns#

Persistent instructions belong outside the conversation. System prompts, project conventions, and tool descriptions should be loaded from configuration files (like CLAUDE.md or system prompt templates) rather than injected as user messages. This ensures they survive compaction and are always visible to the model regardless of how many conversation turns have elapsed.

Tool results are the biggest context consumers. A single file read or grep result can return thousands of tokens. Effective harnesses truncate tool results at the source — return the first 200 lines of a file rather than the full 5,000 — and let the model request more if needed. This is a form of lazy loading for context.

Pre-step hooks (called prepareStep in Vercel AI SDK, call_model_input_filter in OpenAI Agents SDK) allow the harness to modify the message array immediately before each LLM call. This is the right place to implement dynamic context pruning: drop tool results from steps that are no longer relevant, summarize long outputs, or inject fresh retrieval results.

Guardrails#

An agent that can call tools and take actions in the world needs safety boundaries. Guardrails are the harness components that validate inputs and outputs, enforce policies, and halt execution when something goes wrong. Without guardrails, a model that hallucinates a dangerous command or generates inappropriate content will execute it — the model has no built-in concept of "I should not do this."

Permission Models#

Production agent harnesses use tiered permission systems rather than a binary allow/deny. Claude Code's model is representative:

The permission model is a spectrum between safety and speed. Moving toward more autonomy requires compensating controls: good test suites, sandboxed environments, version control, and the ability to roll back.

Observability#

Agent systems are inherently non-deterministic. The same input can produce different tool call sequences, different reasoning paths, and different final answers across runs. Traditional logging — recording inputs and outputs — is necessary but insufficient. Agent harnesses need structured tracing that captures the full execution tree: every LLM call, every tool invocation, every decision point, with timing and token usage at each step.

Key Metrics for Agent Systems#

Error Handling and Recovery#

Agents fail in ways that traditional software does not. A web server either processes a request or returns an error. An agent can partially succeed — completing 8 of 10 steps correctly — and then fail in a way that corrupts the work done so far. The harness must handle not just tool-level errors but also agent-level failure modes: the model going off track, context overflow, and cascading mistakes.

Checkpointing and Durable Execution#

For long-running agent tasks (minutes to hours), the harness should save progress at meaningful checkpoints — typically after each successful tool execution or at task boundaries. If the process crashes, the agent can resume from the last checkpoint rather than starting over.

Memory System#

Every agent conversation is stateless by default — when a session ends, everything the model learned about the user, the project, and past decisions vanishes. The next session starts from zero. The user must re-explain their role, re-state their preferences, and re-correct the same mistakes. This is not a minor UX annoyance; it is a fundamental limit on how useful an agent can be over time.

A memory system solves this by persisting key facts across conversations. It is not a chat history replay — it is a curated knowledge base that stores what matters and discards what can be derived from code, git history, or documentation.

What Gets Stored (and What Doesn't)#

The key insight in memory system design is that memory should only store what cannot be derived from the current state of the codebase. Code patterns, file structure, and git history are always available by reading the project — storing them in memory creates stale duplicates that conflict with reality.

What is explicitly not stored: code conventions (read the code), architecture decisions (read ADRs or CLAUDE.md), git history (run git log), debugging solutions (the fix is in the code), or ephemeral task state (use a task list instead). These exclusions prevent memory from becoming a stale mirror of information that has a better source of truth.

Putting It All Together#

The components covered in this chapter — execution loop, tool dispatch, context management, guardrails, observability, error handling, and memory — are not optional features. They are the minimum viable harness for a production agent. Missing any one of them creates a specific class of failure:

Summary#

The most important insight in agent harness engineering is that the harness matters as much as the model. A frontier model with no step limit, no context management, and no error handling will fail on tasks that a smaller model with a well-engineered harness handles reliably. The model provides the intelligence; the harness provides the reliability. You need both — invest in them equally.

Sources:

• Building Effective Agents — Anthropic Research
• How Claude Code Works — Anthropic
• OpenAI Agents SDK Documentation
• Vercel AI SDK — Building Agents
• CrewAI Documentation
• AutoGen Architecture — Microsoft
• LLM Powered Autonomous Agents — Lilian Weng
• LangChain: What Is an Agentic System
• Amazon Bedrock: How Agents Work