Aurora

A local-first voice assistant that combines wake word, STT, LangGraph tools, RAG memory, TTS, and private mesh architecture.

Live 3D architecture

The scene beside this panel is this system's architecture. Drag to rotate, scroll panel to read on.

Problem

Most assistants become either cloud-dependent black boxes or local demos that break down once wake word, speech, tools, memory, scheduling, model routing, and multi-device use all have to work together.

Contribution

Designed and built the assistant architecture: wake word, real-time STT, LLM orchestration, plugin/tool loading, MCP integration, local memory, TTS, UI state, and process-mode service deployment.

Architecture

Aurora started as a local assistant and was deliberately separated into service contracts: STT coordinator, transcription, wake word, TTS, orchestrator, DB/RAG, tooling, scheduler, gateway, and supervisor. The same capabilities can run in thread mode through LocalBus or process mode through Redis/BullMQ, with gateway routes generated from typed contracts and mesh routing designed as configuration rather than caller rewrites.

Technical decisions

OpenWakeWord or Porcupine for local wake-word detection, Whisper/RealtimeSTT for speech input, and Piper for local TTS

LangGraph and LangChain orchestration so the assistant can reason through tools rather than act as a prompt wrapper

RAG memory used both for long-term user context and for matching only relevant tools per request

MCP support for stdio, HTTP, SSE, and websocket tooling so existing assistant ecosystems can plug in

Typed method contracts and a message bus so services can move between in-process, containerized, and mesh topologies

Gateway generation from contract metadata so API routes reflect the live service registry

Tradeoffs

Local privacy and extensibility over a single hosted assistant path

Service boundaries over a simpler monolith

Configurable providers over one hard-coded model stack

Outcome

A deep AI systems project: voice UX, local privacy, tool orchestration, service contracts, RAG, model-provider flexibility, and distributed mesh architecture all visible in one codebase.

visual evidence · 1/6