Building an agent harness: loop, memory, gates

Most "build an AI agent" guides stop at the demo: a loop that calls a model, runs a tool, and prints an answer. That is the easy third of the job. The part that determines whether your agent is useful in month two is everything around the loop: what it remembers, what it is allowed to do unsupervised, and how you know when it is wrong.

I run my operation on a harness I built called NxtOS, with a memory layer called NLM. This post is the guide I wish existed before I built them: first the minimal harness from scratch, then the same harness on NxtOS, then the memory layer that makes either version compound. Mixed audience on purpose. If you are non-technical, the shape of the thing is in the first two sections. If you are senior, skip to the hardening checklist and argue with me.

For background on the loop pattern itself, see my earlier post on the local coding agent.

What a harness actually is

An agent is a model. A harness is everything that makes the model reliable: four pieces.

1. The loop. Take a task, gather context, act, observe the result, repeat until done or stuck. The loop is the only piece most tutorials cover.
2. The tools. The actions the agent can take: files, shell, APIs, databases. Each tool needs a schema, a timeout, and an error contract. An agent with vague tools produces vague outcomes.
3. The memory. What the agent knows across sessions without you re-explaining. This is the piece everyone skips, and it is the piece that matters most after week one.
4. The gates. What the agent may do unsupervised versus what needs a human. Per function, per rung. No gate, no unattended runs.

Loop without memory means re-solving the same problems every session. Tools without gates means an agent writing files outside your project folder at 3am. (I measured this. More below.)

Path A: the minimal harness from scratch

You need less than you think. A single script around an agent CLI, a workspace folder, and a short markdown file describing the operation.

The workspace. One folder holds everything the agent reads at session start: what the business is, how decisions flow, which tools exist, and per-function notes. One page of context beats fifty pages nobody reads. If the agent needs it every session, it goes in this folder. If not, it does not.

The loop script. Pseudocode, but barely:

# harness.sh (the whole v1)
CONTEXT=$(cat ./operation.md)
while true; do
  OUTCOME=$(agent run --context "$CONTEXT" --task "$TASK")
  echo "$OUTCOME" >> ./log.md
  echo "$OUTCOME" | grep -q "DONE\|STUCK" && break
done

That plus a log file is a harness. Everything after this is hardening.

Three rules that make v1 work:

• Append every outcome to log.md. Your log is your cheapest memory and your first audit trail.
• Give each tool a timeout and a JSON error shape. Agents handle structured failure well and silent failure badly.
• Define DONE before you start. "Looks good to me" is not a completion condition. A file exists, a test passes, a row appears in the database. Something checkable.

This gets you a working harness in an afternoon. It will also teach you, within a month, exactly where your time leaks and which workstreams deserve real automation first. That diagnosis is the point of v1, not the automation itself.

A concrete starter: the DuckDB-Wasm data bot

Here is the approach applied to a starter project worth building: a local, in-browser data analysis bot with DuckDB-Wasm holding the data and an LLM turning conversation into charts, tables, pivots, and exports. Treat it as a learning project where "good enough" is the goal. That is exactly the right shape for a Path A harness, so walk it through the four pieces.

Map the project onto the four pieces:

• Loop: question in plain words, generate SQL, run it against DuckDB-Wasm, render the result, and on error feed the database error back to the model to fix forward. The error-feedback step is the whole ballgame for SQL generation.
• Tools (three, no more): run_sql with a timeout and structured errors, render_table_or_chart that takes rows plus a chart spec, export_csv for downloads. Resist adding a fourth tool until v1 embarrasses you.
• Memory: append every working question-plus-query pair to the log. Within weeks that log becomes a library of query exemplars the agent reuses instead of regenerating from scratch.
• Gates: open the DuckDB connection read-only. The strongest gate is architectural: the tool physically cannot write, so unsupervised runs cannot corrupt data.

ask -> generate SQL -> run against DuckDB-Wasm
  -> error? feed the error back, fix forward
  -> rows? render table/chart, or export CSV
  -> log the working pair, DONE

DONE per artifact, defined up front: a table renders N rows, a chart spec validates, a CSV downloads. Starting vague is fine. Vagueness is what v1 is for: build the loop, watch where it struggles, and let the log tell you what to harden next. That feedback cycle, loop first and hardening second, is the approach. The data bot is just one place to run it.

Path B: the same harness on NxtOS

NxtOS (my agentic-OS harness, currently in a private repo) is what Path A grows into when the workspace folder needs structure. Same four pieces, stricter contracts.

The six-lens onboard. Instead of writing operation.md freeform, an interview walks through six questions: where decisions route today, which functional units already produce approvable output, what the canonical record is per workstream, which processes are documented versus tribal, whether the operation compounds learning or re-solves problems, and per function what must stay gated versus run unattended. Answers land in nxtos/intake.yaml. Edit the answers, re-run compose, the harness regenerates.

The bundle. Compose turns intake into a nxtos/bundle/ directory: operation.md, per-function docs, subproject docs. The agent reads the bundle at session start. Because the bundle is generated, it cannot drift from the intake the way hand-maintained docs always do.

The four commands that matter:

nxtos-validate ./nxtos/bundle     # strict profile check, fails loud
nxtos-render ./nxtos/bundle --html map.html   # one-page Action Organizer
nxtos-reconcile ...               # scaffold agents/skills the bundle declares but the workspace lacks
nxtos-drift ./nxtos/bundle        # temporal drift: what changed since the last snapshot

Validate, render, reconcile, drift-detect. If your harness cannot do all four, you have a loop with docs, not an operating system.

Engines over ACP. NxtOS talks to coding agents over the Agent Client Protocol, so the engine (Claude, Codex, anything ACP-compatible) is a config entry, not a code change. This is the correct layering, and it contains a trap: ACP lets each agent decide for itself which tool calls it asks you about first. Nothing in the protocol forces a check-in before writing a file. I ran a mediation probe against my two engines and found one writes files to disk, including outside the project folder, without asking. The other asks before every write. You cannot tell from docs. Measure before unattended runs:

python3 scripts/acp-mediation-probe.py <engine-id>

The memory that makes it compound: NLM

Here is the interesting part. A harness stores state. Memory compounds it. NLM (Non-Linear Memory, pbmagnet4/nlm-memory) is the memory daemon my harness reads: it indexes every agent session across runtimes and serves session recall, fact history, code exemplars, and citation tracking over an MCP endpoint on localhost.

Why this matters more than the loop: after roughly a month of use, the value of my setup stopped being "the agent does tasks" and became "the agent remembers." Decisions, dead ends, naming choices, which approach failed last time. Without memory, every session pays the re-derivation tax. With it, the harness gets cheaper to run the longer you use it.

Three memory practices worth stealing even without NLM:

1. Recall at session start, not mid-task. pull prior context on the task's subject before doing substantive work. Mid-task recall is how agents contradict prior decisions.
2. Facts over transcripts. Store the decision ("we chose X because Y"), not the conversation. Transcripts are for audit; facts are for work.
3. Verify memory against the bundle. NxtOS has a verification stage that grounds bundle claims against memory with tracing. Memory lies by omission: it remembers what you told it, including the things you later changed. Reconciliation between memory and the declared operation is a scheduled job, not a hope.

The failure mode to watch: memory that is never pruned becomes confident and stale. If a recalled fact conflicts with your current config, the config wins, and the fact gets updated. Every time.

The hardening checklist

Do these before any agent runs unattended. All of them. In this order.

• Completion conditions. Every task type has a checkable DONE. No DONE, no run.
• Mediation probe. Measured what your engine does unsupervised, per engine. Re-probe on every engine version bump.
• Eval gates. Scoreable assertions on outputs that gate promotion: an agent output that fails the gate does not ship, and the failure is logged. Ungated autonomy is a demo.
• Secrets as references. {env:VAR} resolved at runtime, values in .env, never in the repo. Scan staged files for leaks in a pre-commit hook.
• CI green. Tests plus lint on every push. My NxtOS checkout currently runs 3,763 tests; the suite caught a real packaging bug (an undeclared module) during the exact audit that produced this post.
• Log everything. Session outcomes appended to a log with dates. The log is your drift detector, your audit trail, and your training data.
• One human-readable artifact. A rendered page (mine is the Action Organizer) that a non-technical person can pin above their desk. If the harness cannot explain the operation on one page, it does not understand the operation.

When to graduate from A to B

Stay on Path A while the workspace folder fits in your head. Graduate when one of these goes wrong: the docs drift from reality, two agents contradict each other across sessions, or you cannot answer "what changed since last month." That is the moment a generated bundle, a memory daemon, and drift detection stop being overhead and start being the product.

The harness is not the model. The model is interchangeable. The harness, the memory, and the gates are the system you actually own. Build those, and the next model upgrade is a config change instead of a rewrite.