Head of AI · End to End AI Delivery: Agentic, RAG, Evals & LLMOps · DigitalHubAssist LLC
Oct 2024 · Present
Remote · Applied AI / SaaS (Anthropic partner program)
- Lead AI delivery end to end: I architected and built Agent Squad, agentic frameworks with autonomous, multi step agents in production (dev, QA, PMO, content). I design agent behavior (system prompts, tool and agent prompts via function calling, context management, guardrails), build RAG (chunking, embeddings, pgvector retrieval), and apply advanced prompt engineering (instruction design, few shot, structured outputs). Under my leadership the company reached Registered Partner status in Anthropic's partner program.
- Design evaluation frameworks (LLM as a judge, custom metrics, quality gates) and observability (Langfuse) across task completion, tool selection and failure recovery; run structured experiments across prompts, retrievers, chunking and models; and categorize failures (hallucinations, retrieval misses, instruction following). I make pragmatic feasibility calls (prompting vs RAG vs fine tuning vs classical ML) and read code (Python, TypeScript), building with Claude Code on a modern cloud (Azure, Docker, CI/CD, event driven with Inngest).