Head of AI · GenAI Quality, Evaluation & QA in Production · DigitalHubAssist LLC
Oct 2024 · Present
Remote · AI / SaaS (Anthropic partner program)
- Own the quality of the AI systems I build: coordinate autonomous QA agents (Agent Squad) in production, evaluate and trace LLM and agent outputs with Langfuse, and design checks for hallucinations, wrong outputs and regressions. Build human in the loop validation (the agent suggests, a human validates) and structured, auditable outputs (JSON, logs). MLOps: model versioning, agent monitoring, CI/CD. Under my leadership the company reached Registered Partner status in Anthropic's partner program.
- Deep on LLM behavior (prompt engineering, RAG, function calling, embeddings), so I know where GenAI breaks and how to test for it: edge cases, adversarial inputs and guardrail failures. I turn ambiguous quality criteria into repeatable checks.
Consulting Manager · Software Delivery Quality & Defect Management · NTT DATA Colombia
Apr 2021 · Oct 2024
Bogotá, Colombia · Consultancy (GSI) · Client: banking (Grupo AVAL)
- Set the delivery quality bar on the office's largest client account (up to 50 people): owned defect management and testing to production, resolving issues before they reached the client. Drove CrediCore to production on Azure, with rigorous validation against the bank's internal processes (each request checked and returned as approved or denied).
Project Manager / Digital Transformation Lead · Testing & Go Live to Production · VASS (2018·21) · GlobalHitss (2015·18) · Tecnocom (2012·15)
2012 · 2021
Bogotá, Colombia · Pensions · Health · Public Sector
- Delivered software to production across sectors with testing and go live: modernized Porvenir's channels (millions of users, LATAMDIGITAL 2019 award); delivered digital commerce (Colsanitas); led the digital transformation of Colombia's Judicial Branch under PMBOK/PMI. Recognized expert by Liferay.