Your private AI assistant has a ceiling, and it's the device it runs on

When people tell you to "just run it locally," they skip the part where a single device is the limit.

You can run wake word detection, speech-to-text, text-to-speech, embeddings, and a usable LLM on consumer hardware today. I built all of that into Aurora's first release and I use it daily. But every time I picked up my phone, walked into another room, or asked the always-on Raspberry Pi to do real work, I hit the same wall. The machine in front of me was not the machine with the GPU.

That wall is what this article is about, and why the way past it is a mesh of your own devices rather than the cloud.

The dilemma everyone is asked to swallow

If you want a capable AI assistant, the industry offers you two doors.

Behind the first: cloud assistants. They are capable, convenient, and continuously fed your data. Your 24/7 audio, your transcripts, your conversation history, your documents get processed and often stored on someone else's servers, usually behind a monthly subscription. You rent the intelligence and you pay with sovereignty over your own data.

Behind the second: fully-local assistants. They are private by construction, nothing leaves the machine, and they are capped by whatever single box you stand next to. Run a serious model and your laptop turns into a space heater. Run it on a Pi and you wait.

That tension is the privacy-versus-capability dilemma. Most products resolve it by deciding privacy is the part you give up.

Why local-first runs out of room

Local-first hits a hard ceiling at one device.

Your phone lacks the GPU to run a decent LLM. Your desktop has one, but it sits in another room, or asleep, or off. Your Raspberry Pi makes a great always-on microphone and can barely hold the transcription model open. Cheap peripherals like a Pi or an ESP32 make ideal access points, and they sit idle because no single one of them can run the whole pipeline.

So the moment you want your assistant in your home, your office, and your pocket on the commute, local-first pushes you back toward a distributed system. And the only distributed option the market hands you is the cloud. Subscriptions. Relays. "Trust us with your data."

I wanted a third door.

The third option: distributed power, local control

The idea behind Aurora's mesh is simple. You already own enough hardware to run a capable private assistant. It just sits scattered across devices that do not talk to each other.

So let them.

Let the desktop with the GPU be the brain. Let the Raspberry Pi with the microphone be the ears. Let the phone in your pocket be a lightweight client. Each device contributes what it does well, and together they behave like one assistant, a private AI "cloud" built entirely from machines you own.

You decide what each device offers, what it consumes, and who can touch what. No central server holds your data. No corporate relay sits in the middle of a conversation. The only third party in the whole design is a STUN/TURN signaling server that helps two of your devices find each other, and it sees an IP address, never your encrypted traffic. You can host that yourself, since it does almost nothing and costs almost nothing.

The project's tagline is distributed power, local control. Your compute grows as you add hardware instead of as you upgrade a subscription tier. Your assistant gets better because you bought a second device, not because you moved up a pricing plan.

A concrete picture

Picture a normal request. You stand in the kitchen and say "Hey Aurora, summarize my unread emails."

The Pi on the counter hears the wake word and transcribes you locally. It has no business running a 12B-parameter model, so it routes the reasoning to your desktop's GPU across your own network. The desktop runs the LLM, calls the email tool, and sends the answer back. The Pi speaks it.

From where you stand it feels seamless. You talked to the cheap device, the expensive device did the thinking, the answer came back through the cheap device's speaker. The audio, the transcript, and the contents of your inbox never left your network. Distribution bought you capability without costing you privacy.

Why this matters now

We are at an inflection point. Local models keep improving at a surprising rate, quantization keeps lowering the hardware bar, and consumer devices ship dedicated AI silicon. Keep a pinch of salt nearby, since today's small local LLMs still stumble on the reliable tool-calling an agent needs, and I will say so before anyone else does. But the trajectory holds.

What has been missing is not the models or the chips. It is the architecture to tie scattered, heterogeneous, consumer-grade hardware into something coherent without surrendering control of your data along the way.

The centralized model is convenient and carries a privacy cost a lot of people no longer want to pay. The fully-local model is private and capped at one device. The peer-to-peer mesh sits in the gap between them, and that gap is where private AI gets practical.

The pattern reaches past voice assistants. Contract-based services, transparent routing between machines, trust you grant on purpose, sharing you control per capability: that shape fits almost any AI workload you want to keep private while still spreading it across more than one machine.

This is the first piece in a short series on how Aurora's distributed mesh works. Next I get into the engineering: how a local monolith became a contract-based microservices system, and how a routing layer called the MeshBus turns "run this on another device" into a configuration value.

Aurora is free and open source. If the idea of a distributed, private, user-controlled AI you actually own lands for you, I want to hear what you would build with it.

Originally published on LinkedIn.