Details
ai / prototype
Aurora
Aurora started as a local voice assistant and grew into a privacy-oriented capability mesh: typed service contracts, LocalBus/BullMQ process modes, WebRTC peer calls, recipient-specific permissions, shared React/TypeScript clients, Rust/WASM authorization, native speech, and Android lifecycle handling.
Live 3D architecture
Use the navigator to move between diagrams and related work, then scroll for more details.
The voice path is the product: wake word, transcription, orchestration, memory, tools, and spoken response.
Problem
Local assistants keep data under the user's control, but they inherit the limits of the machine in front of them. Aurora's core problem is capability sharing without collapsing privacy or control: a low-VRAM notebook can use an approved desktop model, a phone can request another device's speech capability, and each device exposes only what the owner has allowed.
Contribution
Designed and built the Aurora architecture across service contracts, bus dispatch, generated API routes, peer transport, per-peer grants, shared clients, native speech, local data boundaries, and evaluation records. The useful contribution is the integration and architecture: the project composes existing runtimes and models into a tested assistant/mesh system with explicit authority and lifecycle boundaries.
Five Architecture Axes
Aurora separates five decisions that are easy to confuse: service execution mode, client surface, mesh role, transport profile, and runtime tier. Threads mode uses LocalBus inside one Python process; process mode uses BullMQ over Redis between service processes; neither automatically means P2P. A browser, desktop shell, or Android client can act as a remote console or as an approved mesh node depending on runtime profile and grants. HTTP-only, WebRTC-only, and WebRTC-preferred are transport choices, while authorization remains an application decision.
Architecture
Aurora started as a local assistant and was separated into service contracts: STT coordinator, transcription, wake word, TTS, orchestrator, DB/RAG, tooling, scheduler, gateway, and supervisor. The same capabilities can run in thread mode through LocalBus or process mode through Redis/BullMQ, with gateway routes generated from typed contracts and mesh routing designed as configuration rather than caller rewrites.
Service Contracts And Dispatch
A method contract records module, version, method name, bus topic, exposure level, input/output models, priority, and required permissions. Those registrations drive bus dispatch, generated FastAPI validation/docs, capability announcements, reconfiguration, and removal on shutdown. Capability becomes data that can be routed, projected, filtered, and revoked rather than a direct Python object reference.
Technical decisions
Tradeoffs
Outcome
A deep AI systems project: voice UX, local privacy, tool orchestration, service contracts, RAG, model-provider flexibility, and distributed mesh architecture all visible in one codebase.
Clients, Native Runtime, And Data Boundaries
The shared React UI talks through a TypeScript SDK and platform adapters rather than direct Python services, WebRTC internals, or native commands. Desktop-local can supervise a Python sidecar; desktop-client and mobile packages run without Python. Rust/WASM covers the lightweight authorization core and selected native functions. Local-first data uses SQLite/OPFS with bounded fallback, and knowledge export/import is scoped by namespace, provenance, redaction, and explicit overwrite selection.
Native Speech And Android Lifecycle
Aurora's newer speech path uses Rust-owned sessions, task handles, generation state, cancellation, and FFI confinement around Sherpa-ONNX/ONNX Runtime for STT, VAD, keyword spotting, and TTS. Browser speech uses task-specific WASM workers so synchronous inference does not block the interface thread. On Android, a native Rust mesh session can keep a DataChannel alive, answer an authorized device-status tool while the WebView sleeps, queue bounded per-peer messages under the same identity, and defer foreground-only orchestration until resume.
Testing and results
Python suite
Authorization
Rust/WASM parity
Process mode
Browser WebRTC
screenshot · 1/6
Screenshots
Screenshots for this case study.
Dark mock reference contact sheet for Aurora UI surfaces.
Stack and domains
- Python
- FastAPI
- Pydantic
- LangGraph
- LangChain
- MCP
- SQLite
- Redis
- BullMQ
- WebRTC
- MQTT
- React
- TypeScript
- Tauri
- Rust
- WebAssembly
- Sherpa-ONNX
- ONNX Runtime
- Android