Routing a request to another machine should be a config change, not a rewrite

That sentence is the whole thesis of Aurora's distributed architecture. The moment "run this on my desktop's GPU instead of this Raspberry Pi" turns into a refactor, you have lost. Everything below is how I kept it a config value.

The first piece in this series made the case that a single device is a ceiling for private AI, and the way past it is a mesh of your own machines rather than the cloud. This article covers the engineering underneath that claim: how Aurora went from a single-process local monolith to a contract-based microservices system that can route a call to a peer without the caller ever knowing.

Where it started: a monolith built to come apart

Aurora v1.0 was one process. Wake word feeds speech-to-text, which feeds a LangGraph orchestrator, which feeds text-to-speech and the UI, with a SQLite and vector store for RAG and a plugin/MCP tooling layer alongside. Every capability ran as a thread inside that process and talked through in-memory calls. It was private, it was local, and it worked.

It also carried the limitations you would expect. One process meant one machine, no fault isolation, no way for a second device to join, and no notion of identity or access control between nodes. Those limitations are what motivated the distributed work.

One early decision made the rest possible. Even as a monolith, Aurora's capabilities ran as separate services that communicated through a message bus with explicit, typed contracts. Not function calls reaching into each other's internals, but messages. That discipline turned "make it distributed" into a routing problem instead of a rewrite.

One bus, two topologies

The first abstraction is the message bus. A single MessageBus interface sits in front of two implementations:

• LocalBus, in-memory asyncio queues. This is "Threads Mode": everything in one process, zero infrastructure, ideal for development and single-node use.
• BullMQBus, Redis-backed queues via BullMQ workers. This is "Processes Mode": each service in its own container, ready to run across machines.

The services do not know which one they sit on. The same code that runs as threads on a laptop runs as separate containers across a network. There is no "Lite" build and "Pro" build and no forked codebase, just a different bus implementation selected by an environment variable (AURORA_ARCHITECTURE_MODE).

The bus carries three message shapes, which covers everything the system does:

• Commands: point-to-point, with retry and a dead-letter queue for failures.
• Events: pub/sub, best-effort, fire-and-forget.
• Queries: request/response with a timeout.

A small priority scheme sits on top. Interactive (10) beats System (50) beats External (80), so a live "Hey Aurora" request jumps ahead of background work like ambient transcription. Concurrent delivery on the LocalBus uses asyncio.gather(..., return_exceptions=True) so one failing handler cannot take down the others, and the distributed path leans on Redis for reliability. One implementation detail bit me and is worth stealing: in the BullMQ path I use ephemeral per-request reply.* queues and tear them down after the response, since a long-running deployment otherwise leaks workers and queues until it hits the OS file-descriptor limit.

Contracts are the lingua franca

A message bus only gets you so far if every service speaks its own dialect. What makes a local call and a remote call identical is that the same contract describes both.

I declare every service method with a @method_contract decorator backed by Pydantic models for input and output. A BaseService base class auto-registers those contracts at startup and auto-subscribes the service to its bus topics. The contract registry, which replaced an older and looser event registry, knows every method, its schema, its exposure (internal, external, or both), and its type, a "use" operation versus a "manage" operation.

This registry is the workhorse of the system. The same contract definition does three jobs:

1. It validates and routes messages on the bus.
2. It generates the HTTP API automatically (more below).
3. It backs a remote procedure call over the mesh, since a method that is just a typed contract calls the same way on a peer as it does locally.

Contracts carry a SHA-256 digest and a semantic version, which matters the instant two devices with different builds try to talk. They export, import, and round-trip, which lets one node learn what another node can do.

The Gateway builds itself

Because every service publishes contracts, I do not write the HTTP surface by hand. The FastAPI Gateway generates its routes from the contract registry.

A RegistryAggregator subscribes to service-announcement and service-departure events and keeps a live picture of what is available. A RouteGenerator lazily turns those Pydantic schemas into FastAPI routes with auto-generated OpenAPI docs. When a service joins, its endpoints appear. When it leaves, they disappear, so peers joining and leaving the mesh do not leave a graveyard of routes that 404. One sharp edge for anyone doing this: I had to resolve $defs inline in the generated schemas to keep the OpenAPI output valid.

Failure behavior improved too. In the old world, a call to a service that was not there would hang until a 30-second timeout. With the registry tracking presence, the Gateway fails immediately, roughly a 9-millisecond error response instead of a half-minute stall. Fast failure is a feature.

The MeshBus: where config-not-rewrite happens

Now the payoff. The MeshBus sits on top of the message bus and intercepts every message. It holds a routing table you configure, and for each message it answers one question. Does this go to a local service or to a remote peer?

Because services already communicate through typed messages and explicit contracts, sending a message to another device instead of a local handler is a lookup in a routing table, not new code in the caller. The orchestrator asking for a transcription does not know or care whether the STT service is a thread next door or a container on a Raspberry Pi across the room. The remote call is JSON-RPC over an encrypted WebRTC DataChannel, the subject of the next article, carrying the same contract schemas the service uses internally.

I keep two configuration concerns separate, because conflating them is how distributed systems go all-or-nothing:

• Sharing Policy: what this device offers to the mesh, with capacity limits. Your desktop might share its GPU-heavy orchestrator at max_concurrent: 2 and its TTS at max_concurrent: 10, while keeping its database local.
• Routing Preference: what this device consumes from the mesh. Your phone might prefer the network for orchestration but insist on local wake-word detection, where latency matters more than power.

// Sharing Policy: what I offer
{ "Orchestrator": { "share": true, "max_concurrent": 2 },
  "TTS":          { "share": true, "max_concurrent": 10 },
  "DB":           { "share": false } }

// Routing Preference: what I consume { "Orchestrator": { "prefer": "network", "fallback": "local" }, "TTS": { "prefer": "local" }, "DB": { "prefer": "local" } } ```

Those two small objects let you build whatever topology you want: one powerful box in the house serving LLM and TTS to thin clients in every room, a work machine lending its capabilities to your phone on the go, two friends pooling hardware to beat what either could do alone.

Designing for the network you actually have

Distributed systems fail. Networks drop, devices sleep, containers restart. I learned this the tedious way, so every routing rule carries a fallback:

{ "prefer": "network", "fallback": "local" }

If the desktop goes offline, the Pi keeps working by falling back to local processing. Slower, less capable, still running. When the desktop comes back, routing switches to the network again with nothing for the user to do. The monolith already gave you this local-first graceful degradation, and I kept it through the move to distribution instead of trading it away.

Services also exchange capability manifests on connect: version, supported features, capacity. If a peer's contract version is incompatible, the Gateway rejects the connection cleanly at the gate instead of failing in some confusing way three calls later. Strict compatibility checks up front beat silent corruption downstream.

The part worth taking with you

You do not get topology-agnostic deployment by writing networking code everywhere. You get it by paying down one debt early, making your components communicate through typed, versioned contracts over a message bus, then letting a thin routing layer decide per message where each one runs. The same Aurora binary runs as threads in one process, as containers across one host, or as a mesh across the internet, and the application code stays identical. The difference is configuration.

Next in the series: the transport and the trust model. WebRTC DataChannels, Bluetooth-style device pairing, bilateral trust, and the RBAC system that decides which of your devices can ask which of your other devices for what.

Aurora is open source. If you have built contract-driven or mesh systems and solved these problems differently, I want to compare notes.

Originally published on LinkedIn.