All work
AIDeveloper Tools2025
Lumen — AI Support Copilot
A RAG-powered support copilot grounded in 4,000 pages of docs — with eval suites proving accuracy before launch.
- -63%
- Ticket volume
- 94%
- Answer accuracy (evals)
- 11s
- Median resolution
The challenge
Lumen's support team drowned in repeat questions already answered in their docs. Off-the-shelf chatbots hallucinated API parameters — worse than no bot at all.
The solution
Custom retrieval pipeline over versioned docs with cited answers, confidence-gated escalation to humans and a 600-case eval suite run on every prompt change. Deflection hit 63% without a single hallucinated API in production.
Stack
- Claude API
- RAG
- Next.js
- pgvector
- LangSmith
“The eval-first approach sold us. We can prove the bot's accuracy to our own customers — that's a feature, not just support tooling.”