Building a Domain Harness for AI Agents

Some time ago, I started thinking about how to make agents execute tasks reliably in complex virtual environments. I came across an open-source project called mindcraft, which uses a harness to let an LLM-powered bot enter a Minecraft world. You can tell it something like “build a house,” and it will start placing blocks.

The demo is interesting, but when I tried it myself the bot was not very stable. It would get stuck, and even a small building could stall halfway through. The causes might have been perception, action execution, pathfinding, or context maintenance, but the common problem was clear: real-time closed-loop control forces the model to maintain 3D state, process environmental feedback, and keep doing precise coordinate reasoning. Small errors accumulate quickly.

So I tried a different direction. If building inside the game loop was too fragile, could the model generate a blueprint outside the game and let a mod load it later? That led me to WorldEdit and the schematic toolchain. The problem became: how could an LLM produce a .schem blueprint that WorldEdit could load?

That became the starting point for MinePilot.

But .schem is still a low-level representation. Every position eventually maps to a concrete block state. Asking a model to generate raw voxel data still leads to gaps in walls, misaligned roofs, and broken stairs.

At that point I realized I had already met the same problem in a completely different project: MyInvestPilot. As I wrote in How I Built an AI-Native Quantitative Investment System, I stopped asking the model to write unconstrained Python trading logic because it was too easy to introduce look-ahead bias. Instead, I built Strategy Primitives and a DAG-based engine so the agent could express a strategy only inside a constrained domain space.

Minecraft construction and quantitative investing look unrelated, yet both projects converged on a similar architecture. Looking again at the idea of harness engineering in coding agents gave me a better name for what I had been doing: building a Domain Harness for each domain.

Why Coding Works So Well for Agents

Coding agents have one important advantage: software engineering already provides a machine-friendly feedback environment.

Files
Git
Source code and ASTs
Compiler / Type Checker
Linter
Tests
CI
Issues / PRs / Review
Shell / Runtime / Logs

An agent can edit a file and compile immediately. A compiler points to a file and line. A failing test returns an assertion. CI runs checks again. Git records each change. Pull requests and review add another feedback layer.

A coding agent is therefore not working in a completely open world. The model matters, but compilers, tests, version control, and runtime diagnostics make repeated edit → validate → repair possible.

Many professional domains are missing an equivalent machine environment. We often connect a general agent to APIs or MCP tools and expect it to understand domain semantics, maintain state, obey constraints, detect errors, and repair them. Much of the hard part has not yet been turned into software.

I use Domain Harness as a name for that missing layer. It is not an established industry standard with a fixed definition; it is an abstraction that emerged from building MyInvestPilot and MinePilot.

What I Mean by a Domain Harness

A Domain Harness is an agent-facing work environment for a specific domain. It turns relatively stable domain knowledge, actions, state, constraints, and validation rules into structures that machines can operate on.

A simple shape looks like this:

User Intent
   ↓
Domain Contract / DSL / Tools
   ↓
Structured Plan ← Repair Feedback
   ↓
Domain Engine + Validation
   ↓
Versioned Artifact / Result

The Domain Engine sits inside the Domain Harness and owns the deterministic core. Wherever a rule can be calculated or checked mechanically, I prefer to move it into the engine: coordinates, data alignment, indicator calculation, bounds, budgets, references, schemas, and similar constraints. The model focuses on interpreting intent, planning, and responding to feedback.

This gives the agent a clearer action space, makes important state machine-readable, and lets failures point to specific objects or fields. Plans and outputs can be versioned, and deterministic stages can be replayed when the plan and engine version are fixed.

DSLs and DAGs happen to work well in both MyInvestPilot and MinePilot, but they are not requirements of the idea. Some domains may need only a small set of tools, state, and validation rules. The useful part is moving work that the agent would otherwise have to remember, calculate, or guess into the environment itself.

How MyInvestPilot and CraftDAG Apply It

Move the action space up to domain semantics

When developers discover that unconstrained code generation is unreliable, one response is to expose very granular APIs. In Minecraft, for example, the agent might get only placeBlock(x, y, z, type). That limits what the model can do, but it does not remove the complexity: coordinates, state, and low-level calculations still belong to the agent.

MyInvestPilot’s Strategy Primitives take a different approach. The model composes primitives such as EMA, GreaterThan/LessThan, Lag, and Streak instead of writing Pandas logic. The engine handles data alignment, indicator semantics, and protections against look-ahead bias.

MinePilot’s CraftDAG does the same for construction. The agent writes a ComponentPlan using concepts such as RoomShell, Door, GableRoof, anchor, wall, offset, and overhang. It does not enumerate tens of thousands of voxel coordinates.

The model can say “put the door on the front wall with an offset of 3” instead of calculating every absolute coordinate involved in the wall and doorway.

One design rule has remained useful throughout this work: Agent is the author. Engine is the compiler.

Keep deterministic work in the engine

CraftDAG now roughly follows this path:

1. BuildIntent: natural-language user intent;
2. ComponentPlan: an agent-authored component-level plan;
3. CraftDAG IR: dependency, attachment, and geometry relationships;
4. VoxelPlan: final block coordinates, states, and materials for preview or schematic export.

MyInvestPilot follows a similar pattern. The agent composes Strategy Primitives, the engine builds an indicator and signal DAG, evaluates it topologically, and produces a signal series that can be backtested and recalculated consistently over time.

The model may propose “use MA200 as a trend filter,” but today’s MA200 value, signal timing, and look-ahead checks should follow fixed engine rules. A model may decide that a castle needs a tower in one corner, but bounds, material budgets, and reference validity are better handled by deterministic code.

Machine-Readable Constraints and Repair

Putting an agent behind a DSL is not enough. A Domain Harness becomes much more useful when a failure explains what actually went wrong.

CraftDAG can return an error like this:

{
  "stage": "component-validation",
  "code": "ASSEMBLY_INSTANCE_OUT_OF_BOUNDS",
  "path": "instances[0].anchor",
  "componentId": "northwest_tower",
  "repairHint": "Move the instance inward or increase global bounds."
}

That is very different from returning only Invalid plan. The agent can see the stage, object, field, and a possible repair direction, then change only the faulty part and validate again.

Pre-authoring constraints matter just as much. CraftDAG has an LLM_AUTHORING_CONTRACT.md. Over time I have found it more useful to write explicit ABSOLUTE PROHIBITIONS than to keep expanding a prompt that tries to describe every correct behavior.

A concrete example: every ComponentPlan component must have a unique id; references must use { "ref": "existing_component_id" }; and components must never be inlined directly inside inputs. This does not exist to suppress model creativity. It makes the dependency structure inspectable, validation possible, and errors localizable and repairable.

MyInvestPilot exposes similar machine-facing material through its AI-assisted development documentation, llm-quickstart.txt, and JSON Schema.

At that point the agent has a practical repair path: submit a plan, receive a structured error, patch the local problem, and validate again. That is close to the edit-test-fix loop that coding agents already benefit from.

A rough comparison looks like this:

This distinction is important: CraftDAG is the core engine inside MinePilot’s Domain Harness, not the entire Domain Harness by itself. The same is true of MyInvestPilot’s DAG evaluator. The full Domain Harness also includes the contracts, schemas, tools, state, and error feedback that the agent works with.

Above the Harness Is the Production System

Once the problem of “how can an agent work reliably in this domain?” is reasonably solved, another set of questions appears: why produce this artifact now, should it be delivered, and what happens after delivery?

I now separate those layers like this:

Level 1 — Agent Runtime Harness
model + context + tools + sandbox + loop + memory

Level 2 — Domain Harness
domain contract + tools + state + engine + validation + repair

Level 3 — Domain Production System
opportunity / intent → Domain Harness → artifact → review → delivery → feedback

MinePilot makes the distinction concrete. CraftDAG handles how a building plan is expanded, validated, and compiled. MinePilot still has to decide what to build, how to review it, when it becomes public, whether users download it, and whether recurring, mechanically checkable review issues should become new validators.

I call that broader layer an Agent-Native Domain Production System. It solves a different problem from the Domain Harness: the Domain Harness is about reliable work inside a domain; the Production System also includes opportunity selection, delivery, and real-world feedback.

Where This Pattern Stops Being Useful

Not every AI product needs a Domain Harness. It is most useful when the domain has relatively stable structure, important intermediate state can be observed, results can be calculated or validated, and common errors can be localized and repaired.

Even then, the smallest useful structure is usually the right place to start. Not every domain needs a DSL, and certainly not every domain needs a DAG. Forcing an open-ended creative task into a large schema and compiler can remove capabilities rather than add reliability.

Validation also does not make the outcome “correct.” A strategy can pass schema checks and backtests without being profitable in the future. A building can pass bounds and geometry checks without looking good. Mechanically verifiable parts belong in code where possible; value judgments, risk decisions, and aesthetics still belong to people.

I used to describe this architecture as an “agent-friendly workflow” or a new kind of software interface for the agent era. I now find it more useful to think of it as an infrastructure layer a domain may need before general-purpose agents can participate in its core workflows reliably.

If that view holds, then as models and general agent runtimes become more standardized and interchangeable, the durable work in a vertical domain may be less about a particular prompt or agent and more about the contracts, primitives, state, validation rules, and feedback mechanisms around it. That is what I mean here by a Domain Harness.

Related Explorations & Resources:

• i365.tech: My AI-Native Systems and Product Laboratory
• Deep dive into the quantitative system: How I Built an AI-Native Quantitative Investment System
• MinePilot Official Website
• CraftDAG Source Code (GitHub)