Blog

Dispatch

You already know this architecture. It shipped in 2007.

Familiar plumbing, novel use-case.

The best analogy to describe a Lloyal harness is: MVC with a live LLM as the Model — your product surfaces are the Views, the harness — ordinary TypeScript — is the Controller, and a resident model holds the live generative state. That analogy keeps paying, so we followed it down a layer. It turns out the serving stack maps too — almost noun for noun — onto the stack that carried Ruby on Rails for two decades.

This isn't a coincidence, and it isn't branding. The same forces that shaped Rails' stack — many frontends, many ways to serve, one application that must not care — act on any platform that puts a program between users and a resource. When the forces are the same, the load-bearing shapes converge. What follows is the map, and for each mapping, the why: the force that produced the Rails piece, and how the same force — sometimes amplified — produces ours.

If you ever deployed a Rails app, you already hold Lloyal's architecture in your head. You just know the pieces by their old names.

The map

You knew it as In Lloyal it's What it is
Rack — one tiny interface between every framework and every server @lloyal-labs/binding one tiny interface between every harness and every surface
Puma — many requests multiplexed in one process @lloyal-labs/host many sessions multiplexed over one resident model
Unicorn — process isolation; each worker handles one request at a time @lloyal-labs/relay isolation by OS process, one harness per connection
config.ru — five generated lines wiring app to server the driver a few generated lines wiring harness to host
rails newrails server harness.dev new → your harness's own bin scaffold your application, boot it locally in one command (scaffold in development)
Basecamp — the application Rails was extracted from reasoning.run — the application the harness contract was extracted from the working product that came first; the framework is its generalization
Gems AgentApps installable capabilities — ours arrive signed

Rack, but for intelligence

Rack's insight was almost embarrassingly small. Every Ruby web framework and every Ruby web server agreed on one method:

app.call(env)  # => [status, headers, body]

Why it had to exist. Before Rack, Ruby had an N×M problem: every framework needed bespoke glue for every server. Rails had its adapters, Merb had its own, Camping had its own — and every new server meant every framework wrote another one. Rack collapsed the matrix: N frameworks and M servers each conform to one contract, and N×M integrations become N+M adapters. That single method is why Rails never cared whether it ran under WEBrick, Unicorn, Puma, or a Heroku dyno. The server became a deployment decision instead of an application decision.

We hit the same matrix from the other side. A harness needs to reach a terminal, a desktop app, a browser, and a script — and it needs to run in-process, across a fork boundary, and across a network. Four surfaces, three process topologies: without a seam, that's every surface knowing every placement. With one, it's four adapters against one contract:

Diagram showing four surfaces and three process topologies tangled in an N-by-M integration matrix on the left, collapsing to a single binding seam with N plus M adapters on the right.
N surfaces × M topologies collapses to N + M adapters against one contract — Rack's move, replayed.
type Binding<E, C> = (
  bus: EventBus<E>,           // events flow down: harness → surface
  dispatch: (c: C) => void,   // commands flow up: surface → harness
  bootstrap: E[],
) => () => void;              // dispose

Why the shape differs from Rack's. Rack is request/response because a web app is silent until asked. A harness isn't. A resident model runs: agents spawn and finish, tokens stream, memory pressure rises and recovers — the application has things to say when nobody asked. So the contract is a stream, not a cycle: events flow down continuously, commands interject upward. The two extra pieces are there because transports are physical: bootstrap exists so a surface's first paint already reflects reality (no flash of empty state while events catch up), and the returned dispose exists because sockets die and processes get killed — every adapter must tear down cleanly when its transport dies ungracefully, and the contract makes that non-optional.

Four adapters conform today: render (the terminal UI), ndjson (a stdout stream for scripting), ipc (across a process fork), and wss (across the network — the transport ships today; the browser product built on it is in development). And the consequence is measurable rather than rhetorical: the same reasoning.run — unchanged, same package — is currently the CLI and the engine inside the Artifact desktop app. When our desktop client adopted the published package, it deleted a twelve-thousand-line vendored fork and replaced it with a dependency line. Twelve thousand lines is what the N×M matrix costs when you pay it by hand. One interface is what it costs when you don't.

Puma and Unicorn, but for models

Rails deployment eventually settled into two philosophies, and it's worth remembering they coexisted — neither ever won, because they answer different questions.

Unicorn's doctrine: isolate by OS process. Each worker is its own world; the kernel is your safety boundary; a crashed worker takes nothing with it. Puma's doctrine: multiplex. Many requests share one process and its memory; you accept a shared fate in exchange for density.

Why the same fork appears here — with the dial turned up. The tradeoff behind Puma-vs-Unicorn is isolation versus the cost of duplicating the shared resource. For a web server, that shared resource is modest: a boot-time framework heap, some connection pools — duplicating it per worker is affordable, which is why Unicorn was ever viable. For a resident model, the shared resource is the weights — for a frontier-class model, hundreds of gigabytes of loaded parameters. Duplicating that per user isn't a performance preference; it's arithmetic that decides whether multi-user serving exists at all. A model that fills most of a GPU box fits exactly once. So the two philosophies return, but the stakes invert which one is default:

@lloyal-labs/relay is the Unicorn answer: fork one harness process per connection, each with its own residency. Maximum isolation, honest economics when the box serves one user — and the right shape when the user is the tenant.

@lloyal-labs/host is the Puma answer: load the model once, and run N sessions as structured children over that single residency — each session with its own context, its own KV state, its own agent population. The weights that would cost hundreds of gigabytes per user cost them once per box.

Diagram comparing the relay shape — one harness process per connection, each with its own model residency — with the host shape: one resident model with N sessions multiplexed over a single residency.
Same fork, higher stakes: the relay isolates by OS process; the host runs N sessions over one residency.

Why there's a queue in front of it. A web request occupies bounded worker capacity for a short interval, and a server can trade latency for throughput across thousands of them. A resident-model session is different: its KV cache reserves a physical slice of GPU memory for as long as the session lives. Capacity is a hard integer, not a soft target. So the host runs a small FIFO with explicit admission: a session is queued, then warming, then live — and the client can see which. It's the honest version of what a web load balancer hides: when the box is full, you know you're waiting and why.

Why both ship rather than one winning. Same reason Unicorn survived Puma: isolation is sometimes the requirement, not the compromise. A regulated deployment that wants kernel-enforced boundaries between sessions picks the relay shape. A team serving its own users from one box picks the host shape. The choice is an operator's line in a config — the harness, per the contract above, cannot tell which one it's living in.

(Two honest labels, in the spirit of our status badges: the binding layer ships today and powers both of our surfaces. The host core is built and published at 0.1.0; the web surface it exists to serve is in development.)

config.ru, but for harnesses

Nobody thinks about config.ru. It's five generated lines that wire your application to whatever serves it:

require_relative "config/environment"
run Rails.application

Why a file this small has a name. Because of what it lets the two sides not know. Puma doesn't import Rails; Rails doesn't import Puma; config.ru is the only place both names appear. That non-knowledge is load-bearing: it's why Puma serves every framework and Rails runs under every server.

Our equivalent is the driver — the short generated file that hands a specific harness to the host: how to materialise a session, how to run one. The host imports no harness and no SDK; the harness imports no host; the driver is the only file that knows both. The payoff is the same as Rails', plus one that matters more here: because the host only ever sees the driver's little interface, the entire serving lifecycle — admission, queueing, teardown, failure containment — is tested against a fake harness, with no model and no GPU in the loop. The most dangerous layer of the stack is the one that never needs the expensive hardware to be verified. Like config.ru, you will read the driver once out of curiosity and never edit it.

What you actually write

Here's the part that matters if you're building on this. A Rails developer in 2007 wrote models, controllers, and views. They did not write Rack adapters, did not configure Unicorn's preforking, did not think about the server's accept loop. The stack beneath them was deep — and invisible.

Same deal here. A harness author works with three concepts: the runtime, AgentApps, and their harness — one generator function:

export function* harness(ctx, events, commands) {
  // enable your apps, run your pipeline,
  // emit your events, handle your commands
}

Why the center is code, not configuration. Rails' deepest doctrine wasn't MVC — it was convention over configuration: decide the boring things once, in the framework, so the author's file contains only what's genuinely theirs. We hold that line at every boundary — surfaces, wiring, serving are conventions and generated files — and deliberately not at the center, because for an intelligent application, the center is the product. How your agents collaborate, when a human must approve, what happens when memory runs short, which evidence is trusted — that's behaviour, and behaviour is what programming languages are for. Frameworks that reduce the center to prompt files and config make the easy things easy by making the essential things inexpressible. So the harness is a program — a generator, because structured concurrency is the honest shape for a thing that spawns agents, waits on humans, and must clean up whatever happens: an agent's lifetime is a scope; cancel the scope and everything inside it — including its slice of the model's memory — is released, guaranteed, by construction.

Everything in the table above sits below that generator. You never import the binding. You never meet the relay or the host. The driver is generated. Where your harness runs — a laptop terminal, a desktop app, a shared GPU box — is a deployment decision, not an application decision. Rack's promise, kept for a different Model.

Extracted, not designed

One more parallel, and it's the one that explains why the others hold. Rails was not designed as a framework. It was extracted from Basecamp — a working application shipped first, and the framework was pulled out of what that application had already proven it needed. That's why Rails' conventions fit real work: every abstraction had carried production weight before it had a name.

The harness contract has the same lineage. reasoning.run — our deep-research application — existed first, running in terminals and inside our desktop app. The harness(ctx, events, commands) entrypoint, the binding contract, the runner seam: each was extracted from that working application, not sketched in advance of one. Where we haven't yet earned a generalization, we say so — the full scaffold ships once a second harness has proved which parts are genuinely conventional, for the same reason Rails waited for patterns to repeat inside Basecamp before naming them conventions.

Where the analogy bends

Every good analogy earns trust by admitting where it stops, so: Rails' Model was an Active Record object backed by rows in a database — durable state you query. Ours is a live language model, and that difference is not cosmetic. Three things exist here that have no Rails noun:

The Model thinks. ActiveRecord waits for a query. A resident model runs — which is why the binding contract is a stream rather than a request cycle, all the way down.

The Model forks. When a session spawns ten agents, they aren't ten connections re-reading the conversation. They fork the live attention state: each new agent inherits the entire decoded context in O(1), the way a process inherits memory at fork() — decoded once, shared physically, diverging from there. There is no database analogue, because rows don't have a working memory to inherit.

The Model persists differently. A database persists by writing. Attention state can be snapshotted — but a snapshot is bound to the exact model, quantization, and backend that produced it; it is not a portable record the way a database row is. Sessions here persist by reconstruction: park a session as text, wake it later — on the same machine or a different one — and rebuild the live state from what was kept. The research behind that mechanism is published; the short version is that bounded reconstruction holds quality within a few percent of never having parked at all.

No web framework ever had a Model with a mind. That's the one genuinely new thing here — and it's precisely because it's new that everything around it is deliberately old. Familiarity is a budget. We spent none of it on the plumbing, so all of it is available for the part that's never existed before.

Rails proved you don't make a platform legible by making its core trivial — Rack, Puma, and Unicorn were never trivial. You make it legible by making the author's surface small and the topology someone else's job.

rails new for harnesses is coming. Our Basecamp you can run today, in one command:

npx reasoning.run

reasoning.run is MIT; the runtime and SDK are Fair Source; the harness.dev CLI is Apache-2.0. The docs are at docs.lloyal.ai, the code at github.com/lloyal-ai.