Research

Most of what an agent does, it has already done

Open notes on one idea and its consequences: the parts of an AI system that repeat should not be paid for twice, and the parts that never needed a model should not have one. Each note says plainly whether it is shipped or still a direction.

READ THE DOCS
The thesis

Determinism is the cheapest intelligence

An agent that looks up the same fact twice pays twice. An agent that rebuilds its context after every compaction pays for what it already knew. A build step that produces identical output every time does not need a model in it at all.

None of that is a reasoning problem, and reasoning is the most expensive tool available. The work below is about finding the boundary: which parts of an AI system are genuinely novel each time, and which parts only look that way because nobody measured them.

Where a claim needs a number, it gets one from a benchmark you can reproduce, or it does not appear here yet.

Notes

What we are working on

SHIPPEDWATCHMANAGENTSCOST

How much of an agent's work is secretly repeatable?

Most of what an agent does, it has done before. Watchman records every tool call with its result and replays the ones whose answer has never changed.

THE QUESTION

If you recorded every tool call an agent made and compared the results, what fraction would turn out to be deterministic enough to stop paying for?

WHAT WE FOUND

  • A tool call is only replayed when the same input has returned the same output every time it has been observed. The term for that is witnessed-deterministic, and the caution in it is deliberate: a result that has been identical so far might still change, so the replay is a bet, not a proof.
  • The interesting artifact is not the cache, it is the census. Reporting what share of a workflow is repeatable tells you something nobody currently measures about their own agents.
  • A cached answer served after the world moved is a worse failure than a slow one. Freshness-sensitive calls have to be excluded by policy rather than discovered by incident.
SHIPPEDMEMORYISOLATIONRETRIEVAL

Retrieval that stays fast as you add projects

Memory partitioned by project, company, user and agent, so an agent working in one folder searches that folder's graph rather than everything you have ever stored.

THE QUESTION

Does partitioning a memory graph by scope keep retrieval quality flat as the corpus grows, instead of degrading the more you store?

WHAT WE FOUND

  • An unpartitioned graph gets slower and noisier with every project added to it. Scope is what keeps the tenth project from making the first one worse.
  • The same boundary doubles as an isolation guarantee: the agent working on one repo cannot read another's graph. That is a security property, not just a tidiness one, and it needs to be documented as such — where it is enforced, and what happens when a query crosses it.
IN PROGRESSLANGUAGEDETERMINISMCOMPILERS

A language where the machine never reasons

A compiler that turns a handful of words into a finished, production-grade site. No model in the build path, no network call, same words in means the same code out, forever.

THE QUESTION

For the work that is ceremony rather than taste, is a deterministic compiler simply better than a model — cheaper, faster, and reviewable in a way generated output is not?

WHAT WE FOUND

  • Building a high-end site is mostly ceremony: scene setup, camera, resize handling, animation loop, lights, scroll-linked motion, reduced-motion fallback, mobile degradation, preloading, theme consistency. It is close to identical every time, and it is where the weeks go.
  • Generation and compilation fail differently. A generator produces something for any input; a compiler errors when a word does not exist. That is worse coverage and better trust, and for repeated work the trade favours the compiler.
  • Determinism is what makes output diffable and reviewable across many projects. The same source compiles the same way next year, which is not a property a model can offer.
  • Any model layer belongs outside the compiler, emitting words a human reads and edits before anything is built. The moment the model is in the build path, the guarantee is gone.
IN PROGRESSAGENTSSAFETYAPPROVAL

The middle between watching every step and blind trust

Guards the agent cannot cross, a proxy that answers routine decisions from your PRD, and your phone for anything irreversible.

THE QUESTION

What is the smallest set of controls that makes leaving an agent running unattended a reasonable thing to do?

WHAT WE FOUND

  • The blocker on autonomous agents is not capability, it is that you cannot predict step forty. Sitting at the keyboard caps the agent at the speed of your attention; trusting it fully works until the first force-push.
  • The escalation list has to be an allowlist. A blocklist of dangerous operations is never complete, and the first operation nobody thought to add is the incident. Unknown operations should be treated as high severity by default.
  • Approval has to fail closed. An escalation that times out because a phone was unreachable must block, never proceed. The entire promise survives exactly until the first counterexample.
Directions

Agent shapes we are studying

Each of these is a different stress test of the same layer. None of them is a shipped product, and nothing here should be read as a capability claim.

Loyalty recovery

EXPLORATION

Reaching a customer who has quietly stopped buying, before the churn is old enough to be permanent.

The signal is in behaviour that already happened, so the hard part is noticing the change in time rather than composing the message.

Clinical literature

EXPLORATION

Keeping up with a corpus that moves faster than any individual can read, and citing it accurately.

Provenance matters more here than anywhere else. An answer without a source is not merely unhelpful, it is unusable — which makes it a strict test of cited retrieval, not a medical product.

Social publishing

EXPLORATION

Posting on a schedule with an awareness of what is currently happening, rather than from a queue written last month.

A good test of freshness: the difference between a timely post and an embarrassing one is usually a few hours of world state.

Executive briefing

EXPLORATION

Assembling the state of a company, its market and its competitors into something readable each morning.

Synthesis across many sources where the value is in what gets left out, which is the hardest thing to evaluate.

Market analysis

EXPLORATION

Reasoning over prices, filings and news together, on data fresh enough to still be worth something.

The strictest latency and validation requirements of any of these, and the one where a single bad source has the clearest cost.

Related
WatchmanCoding agentsResearch & analyticsBlog
Build

Try the Parts That Already Work

The shipped notes above are available today. Start free.

START FREE TRIAL