Introduction

A limit order is the intent to buy or sell a stock with a restriction on the maximum price to be paid. The limit order book is a data structure that tracks outstanding buy and sell orders at pre-set prices. Since limit orders represent an intent by a single actor and the LOB represents a set of intents across a price range, placement of the limit order is key to ensure execution. Placement of limit orders adds liquidity to the limit order book (LOB) since they are only executed if the constraint price is agreed upon with another party. The act of placing a limit order is a balance of risk management, capital allocation, and hypothesis invalidation to ensure orders have a higher probability of filling. Due to the challenge of this task, traders utilize a myriad of strategies and tools to ensure their orders are filled, especially operating on lower-time horizons, which is why this task is computationally tractable and fit for an agent.

Harnesses are executable programs agents rely upon in order to perform long-horizon tasks efficiently. They provide a concrete way to augment and surpass an agent’s base capabilities. Although harnesses yield productive agents, harnesses remain fixed regardless of the task the agent is performing. A fixed harness in an environment runs counter to tasks that demand adaptations. For example, imagine you are taking an exam proctored by a teacher. The teacher observes your attempt, then grades the final submission. Instead of asking you to retake the exam with only a pencil, the teacher gives you a calculator. On the next iteration, the teacher provides a formula sheet. Over the course of many iterations under the same task, the student’s environment adapted to the needs of the student, which is the power of this idea. Treating the harness as an adaptive program shaped by the iterative performance of the agent and feedback from the environment allows for more robust extraction of model capabilities in different environments. Therefore, this project is an application of self-improving harnesses to limit order placement.

Task Design - Finance

Due to compute and time constraints, I chose a task that has a quick feedback loop with decision-making under changing conditions. To build the environment, I needed a source of market data, specifically different levels of the LOB. For simplicity’s sake, there are three levels of market data:

LOBSTER is an online limit order book data tool to provide easy-to-use, high-quality limit order book data. I used their sample datasets covering Amazon, Apple, Google, Intel, and Microsoft, which formed the basis of the episodes to simulate trading conditions for the agent. If I had more resources, I would have utilized the LOB from Hyperliquid (see An Open Book: Level 4 Order Book Data from the Hyperliquid Exchange).

The objective of the agent is to use the harness to inspect a snapshot of market conditions (LOB), then propose a limit order placement within the queue. A conservative queue that treats the same-price volume as ahead of the hypothetical order and credits execution against the queue was used. The state of the order (partial or complete fill) is used to measure the agent’s performance in a given episode. The snapshot of the LOB provides a nice task formulation for the harness design problem since the harness can be adapted to allow the model to inspect the market data and derive better insights while operating through the same interfaces across sequential market snapshots.

Self-Adapting Harness Design

Developing a meta-level system for design (aka a Harness Factory) involved figuring out a way to distill telemetry into a concrete specification that could be used to programmatically build a harness. While sketching out a potential implementation, I was reminded of learning about context-free grammars within my Automata & Complexity, specifically the idea of structured generation using constrained decoding. The idea is that you could programmatically constrain the output of a model to a specific set of words. Taking this one step further, I decided to create a harness language that would restrict the universe of possible harnesses that could be created, while still allowing for the free-form insights from an LLM on the episode’s trajectory and performance to be utilized to improve the design of the next generation of harnesses.

The harness language allowed for each of the harnesses to share the same structure, while enabling me to trace the lineage and change of the harness over the course of multiple episodes. Essentially, the first harness generation could be treated as an interface that is iteratively implemented based on feedback by an LLM. The agent still uses the same high-level functions, but the implementations change depending on what the Factory LLM deems beneficial for the performance of the agent. I did keep the number of functions for this project fixed for simplicity’s sake.

For clarity, The Factory does not create Python code. It returns a mutation from a fixed harness language e.g. increasing visible book depth. The mutation is applied, compiled, validated, and prompted.

Harness_current = Compile(Harness_initial)

for episode in MarketEpisodes:

    Observation = Environment.Observe(episode)

    Action, Trajectory = DecisionAgent.Act(
        Harness_current,
        Observation
    )

    Outcome = Environment.Simulate(
        episode,
        Action
    )

    Experience = BuildExperience(
        Trajectory,
        Action,
        Outcome
    )

    Proposal = Factory.Propose(
        Harness_current.Spec,
        Experience,
        HarnessLanguage
    )

    if ValidateProposal(Proposal):

        HarnessSpec_next = ApplyMutation(
            Harness_current.Spec,
            Proposal.Mutation
        )

        if OperationallyValid(HarnessSpec_next):
            Harness_next = Compile(HarnessSpec_next)
            Status = PROMOTED
        else:
            Harness_next = Harness_current
            Status = INVALID

    else:
        Harness_next = Harness_current
        Status = INVALID

    Record(
        episode,
        Harness_current,
        Experience,
        Proposal,
        Harness_next,
        Status
    )

    Harness_current = Harness_next

Harness Evolution Process

1. Agent uses the harness to analyze the LOB snapshot and place an order.
2. The agent’s order is simulated using a conservative queue model processing limit orders. Once executed, the performance of the order is calculated.
3. Factory LLM receives trajectory, performance, harness, and legal DSL of harnesses.
4. Factory LLM returns a typed proposal.
5. A deterministic function turns the typed proposal into a harness specification.

Example Harness Language DSL

HARNESS ::= OBSERVE AUGMENT DECIDE CONSTRAIN FEEDBACK

OBSERVE ::= BOOK_LEVELS LOOKBACK
          | BOOK_LEVELS LOOKBACK RAW_EVENTS

AUGMENT ::= FEATURE*

FEATURE ::= SPREAD
          | IMBALANCE
          | MICROPRICE
          | ORDER_FLOW
          | CANCEL_RATE
          | VOLATILITY

DECIDE ::= DIRECT
         | ANALYZE_THEN_CHOOSE
         | COMPARE_ACTIONS
         | PROPOSE_CRITIQUE_REVISE

CONSTRAIN ::= OFFSETS FALLBACK

OFFSETS ::= [0, 1, 2]

FALLBACK ::= 0 | 1 | 2

FEEDBACK ::= NONE
           | LAST_OUTCOME
           | HISTORY WINDOW METRICS

WINDOW ::= 10 | 25 | 50

METRICS ::= REWARD
          | FILL_RATIO
          | TIME_TO_FILL
          | SIGNED_MARKOUT

Flowchart

flowchart TD
    A["HarnessSpec Hₜ"] --> B["Compile harness"]
    B --> C["Decision agent acts"]
    C --> D["Environment produces objective outcome"]
    D --> E["Trajectory + outcome"]
    E --> F["Factory LLM"]
    A --> F
    F --> G["One typed DSL mutation"]
    G --> H{"Operationally valid?"}
    H -- Yes --> I["Apply mutation → HarnessSpec Hₜ₊₁"]
    H -- No --> J["Retain HarnessSpec Hₜ"]
    I --> B
    J --> B

Results

For the primary full-session AAPL run:

• 26 chronological episodes spanning approximately 09:30 –15:45

• 26 decision-agent logical calls and 26 factory logical calls

• 23 operationally valid mutations promoted

• 3 invalid factory proposals

• 20 distinct harnesses were actually active

• 18 of 26 episodes had identical rewards for all three actions

• Only 8 episodes were outcome-differentiated

• The chosen action filled in 3 episodes:

  • 1 full fill
  • 2 partial fills

• Only 1 chosen action produced a positive reward

For the experiment, I choose AAPL markets and ran the project across 26 chronological episodes spanning 9:30 am through 3:45 pm. Each loop contained one decision agent and one Factory call. The factory produced 23 valid successor harnesses and 3 invalid proposals.

The main limitation became visible in the telemetry. In 18 of the 26 episodes, all three legal placements produced exactly the same reward. Only 8 episodes contained any decision signal. The agent’s selected order filled in three episodes—one full fill and two partial fills—and only one selected action produced a positive reward.

The evolving harness achieved a mean reward of −11,953.85. Therefore, the experiment demonstrates that the harness can modify itself while preserving a constrained interface and auditable lineage, but it does not demonstrate performance self-improvement.

Conclusion

Wrapping things up, this was a fun exercise blending together multiple parts of my past (NLP, ML, Systems, and Automata & Complexity) into one. Although the system did produce a lineage of harnesses, model performance did not concretely change over time. The telemetry shows one direct limitation: 18 of 26 episodes contained no action differentiation, leaving the Factory with a sparse performance signal. The conservative queue model also produces a lower-bound estimate of fills, and the dataset covers only one trading session. These limitations mean the results cannot establish trading profitability or generalization. Nevertheless, the project demonstrates a concrete method for allowing an agentic system to modify its harness without generating arbitrary code or changing the environment. It also produced enough counterfactual evidence to show that the evolution did not outperform a simple static policy. The mechanism works; making the evolution practically useful remains an open problem