Overview
The AI/ML Test Lead will own end-to-end quality assurance for AI-driven solutions across Ulta's Enterprise AI. This role is responsible for defining and executing an Enterprise AI testing strategy, ensuring functional correctness, data accuracy, AI output quality, robustness, performance, and Responsible AI compliance. The role blends hands-on testing leadership, AI/ML-specific validation, and onsite stakeholder collaboration, working closely with Product Owners, AI Engineering Leads and Data teams.
Responsibilities
- 10+ years of experience in QA / Testing roles, with leadership responsibility
- Strong experience defining test strategy, test plans, and execution ownership
- Experience testing data-heavy and analytics-driven systems
AI / ML / Data Testing Skills
- Hands-on experience testing AI/ML or LLM-based solutions
- Strong understanding of:
- Data validation and KPI testing
- AI output correctness and consistency
- Prompt-based and agent-based systems
- Familiarity with AI evaluation metrics (accuracy, error rate, hallucination rate, relevance)
#J-18808-Ljbffr