Back to constellationBack

Details

Perch Insights

Fullstack AI Engineer

I moved from fullstack engineering into AI reliability for a B2B analytics product, working on natural-language queries, customer-schema retrieval, LangChain pipelines, LangSmith traces, evaluations, and ontology-guided insights.

Live 3D architecture

Use the navigator to move between diagrams and related work, then scroll for more details.

Question to query and dashboard result, with specialist agents represented as one high-level decision point.

My work

Applied AI product engineering where correctness depended on schema grounding, observability, and customer-specific evaluation.

I improved a six-agent natural-language analytics pipeline that turned business questions into structured queries over each customer's schema, separating time interpretation, measures and dimensions, intent, subqueries, sorting and filtering.
I migrated retrieval from Pinecone to PostgreSQL/pgvector and improved top-K matching with vector plus keyword search, choosing not to add reranking because dashboard latency was a product constraint.
I rewrote hand-rolled TypeScript agent classes into LangChain pipelines while preserving prompts, then added LangSmith instrumentation that exposed roughly 350 inspectable steps, phases and subphases per execution.
I integrated a custom evaluation framework with LangChain and LangSmith datasets: 300+ prompt cases at pipeline level, 400+ tests per customer environment, final and per-agent JSON checks, and expected-versus-actual diffs.
I contributed to a Python/LangGraph insight-generation service that used customer-built business ontologies to turn query results, pivots and objectives into recommendations for executives who did not know which question to ask.

Platform engineering

Query generation, retrieval, observability and evaluation.

  • Query, retrieval and evaluation

    The pipeline turns a business question into a structured query by resolving time, measures, dimensions, intent, subqueries, sorting, and filters. I migrated retrieval from Pinecone to pgvector with keyword and vector matching, keeping reranking out of the interactive path to limit latency. Customer-specific evaluation suites compare final and per-agent JSON outputs to locate regressions.

  • LangChain and LangSmith migration

    I replaced custom TypeScript agent classes and hardcoded chaining with LangChain pipelines while preserving the prompts. LangSmith exposed inputs, outputs, and context across roughly 350 execution steps and subphases, making failures easier to locate and pipeline changes easier to evaluate.

Query Retrieval Evaluation

NLP query engine; pgvector retrieval; LangChain migration; LangSmith tracing; evaluation framework

  • Problem

    Natural-language analytics needed reliable query generation across customer schemas, business context and release changes.

  • My role

    I moved from fullstack work into query reliability, retrieval, tracing, and evaluations, working with the product and data teams.

  • Implementation

    Maintain specialist-agent query generation, migrate custom TypeScript classes to LangChain while preserving prompts, add inspectable LangSmith traces and per-client evaluation sets.

  • Engineering choices

    Use Pinecone-to-pgvector hybrid retrieval; avoid reranking where latency made it unacceptable; evaluate final and per-agent JSON outputs.

Stack and domains

  • TypeScript
  • Next.js
  • Node.js
  • tRPC
  • LangChain
  • LangSmith
  • LangGraph
  • Python
  • PostgreSQL
  • PgVector
  • Pinecone
  • FastAPI
  • AWS