We are looking for a Senior AI QA Engineer to lead the design, development, and execution of test strategies for our AI/ML-driven systems. In this role, you will play a critical part in shaping how we ensure quality, reliability, fairness, and robustness across our AI products. You will be responsible for ensuring the quality, reliability, and performance of our AI/ML-driven products and systems. You will collaborate with cross-functional teams to test end-to-end pipelines, automate validation processes, and drive industry best practices for AI quality assurance.
What You’ll Do * Lead the QA strategy for AI/ML systems, including model evaluation, data validation, and pipeline integrity. * Design and implement robust automated and manual testing frameworks tailored to AI workflows, model validation * Validate model performance with metrics such as accuracy, F1 score, precision/recall, bias, drift, and explainability. * Drive continuous testing in ML pipelines (CI/CD for AI), enabling scalable model deployment and monitoring including regression testing * Identify and address quality risks associated with data labeling, training data, and model behavior in production. * Collaborate with engineering, DevOps, and product teams to ensure alignment on quality goals. * Test AI systems for accuracy, fairness, bias, drift, robustness, and scalability. * Analyze test results, detect anomalies, and report bugs or inconsistencies. * Assist in designing monitoring systems for model performance in production. * Document test cases, tools, and processes for internal teams and audits. * Stay updated on QA best practices for machine learning and AI systems.
Your First 90 Days
You will start by learning about and understanding our products, including from the experiences of our current team. You will be responsible for building Product mind maps to help design, lead and propose QA test strategies for our AI Product by collaborating with the product and eng teams. You will be responsible for building the first set of QA Test cases (manual or automation driven) to start building the QA pipelines.
Who You Are * 5+ years of experience in QA engineering, with at least 2+ years focused on AI/ML systems. * Strong proficiency in Python, with hands-on experience testing ML code and pipelines. * Familiarity with testing concepts such as adversarial testing, edge case discovery, and synthetic data generation. * Experience with CI/CD, Git, Docker, and at least one major cloud platform (AWS, GCP, Azure). * Strong communication skills with the ability to influence technical and non-technical stakeholders. * Understanding of test methodologies, test case coverage analysis, and effective hardening and exploration strategies
Your Differentiators * Collaborative team member * Ability to work and collaborate with others remotely * Efficiency mindset * Leads by example * Mentor to the team * Problem-solver * Strong analytical skillset * Flexible and adaptable to changing priorities and projects/applications
Our Senior AI QA Engineer will look not only to help contribute, collaborate, and develop their career but will be hungry to assist others in their careers as well.
