Back to constellationBack

Details

Back to Adopt AI

Part of

Adopt AI

ai / professional

Tabby

I helped design and develop Tabby with the Adopt AI team: browser sessions for human login handoff, per-session worker pods, authenticated API execution, and browser automation.

Live 3D architecture

Use the navigator to move between diagrams and related work, then scroll for more details.

VNC login creates an authenticated BrowserContext that can support API and browser automation.

  • Role

    I helped design and develop Tabby with teammates at Adopt AI.

  • Problem

    ABCD could ingest internal web APIs, but authentication still needed human participation in some systems. Login, MFA, refresh flows and old screens could not be reduced to a simple API credential handoff, and some flows still needed real page interaction.

  • Implementation

    Tabby is a TypeScript monorepo using Nx, NestJS, Playwright workers and a Kubernetes controller. Each session is reconciled into a worker pod with tenant, app, and session context. The worker launches a browser and creates a BrowserContext that authenticated API and browser actions can reuse after login.

  • Authenticated execution

    After a user completes login, authorized fetch and browser execution paths can reuse the authenticated BrowserContext. Browser execution covers point-and-click flows when APIs are not enough. Audit events are recorded for execution paths without copying request headers, bodies or typed values into the event payload.

  • Lifecycle and recovery

    The controller reconciles desired session counts across replicas and uses database locking with FOR UPDATE SKIP LOCKED to distribute work. The state model separates session health from the human automation baton, including starting, healthy, unhealthy, login-needed, login-in-progress, failed and terminated states, plus baton recovery, retries and timeouts.

  • Captured artifacts

    The worker can export finite browser auth state: cookies, configured headers, CSRF values, browser storage and custom extractions. The worker encrypts artifacts with AES-256-GCM using a nonce, stores blobs in MinIO, records metadata and expiry in the database, and publishes a NATS event. Artifacts capture selected auth values. They do not restore the Chrome process. The configuration propagates TENANT_ENCRYPTION_KEY; it does not establish a unique key per tenant.

  • Prepared recording sessions

    A warm pool starts healthy sessions at about:blank under a system tenant. A transaction claims one session and reassigns its application, tenant and owner. This moves browser startup work ahead of the recording request. After recording, the session is terminated instead of returned to the pool; proxy reservations are managed separately.

  • CDP and VNC access

    Headless CDP sessions use a page-scoped relay that filters events and commands to a pinned target and rejects Target.* commands. Headed VNC sessions use Xvfb and a noVNC sidecar for human interaction. The two modes make browser inspection and human login available through different access paths.

Stack and domains

  • TypeScript
  • NestJS
  • Nx
  • Playwright
  • Kubernetes
  • PostgreSQL
  • Redis
  • NATS
  • MinIO

Links

GitHubgithub