Testing Lead-QA

**This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Testing Lead-QA based in India.** This is an exciting opportunity for an experienced quality engineering professional to lead testing initiatives for cutting-edge AI products, including Deep Learning, Large Language Models (LLMs), and Vision-Language Models (VLMs). The role combines technical leadership, test strategy development, and documentation ownership in a fast-paced and innovation-driven environment. You will work closely with machine learning engineers, product teams, and software developers to ensure AI systems are reliable, scalable, and production-ready. This position is ideal for someone who enjoys solving complex quality challenges in non-deterministic systems and building structured processes around emerging technologies. You will play a key role in shaping quality standards, improving AI evaluation methodologies, and fostering a quality-first culture across the organization. ### Accountabilities: - Define and lead end-to-end testing strategies for Deep Learning, LLM, and VLM products and pipelines. - Establish testing frameworks covering model evaluation, acceptance criteria, release readiness, and risk assessment. - Create and maintain comprehensive documentation related to testing methodologies, model assumptions, known limitations, and quality sign-offs. - Design and execute testing strategies for prompt engineering, RAG pipelines, hallucination control, multi-turn conversations, and long-context model behavior. - Develop and manage golden datasets, regression testing suites, and benchmarking processes. - Evaluate multimodal and vision-language systems, including image-text alignment, OCR, captioning, and reasoning capabilities. - Build Python-based automation frameworks for model evaluation, validation, and regression testing. - Integrate testing processes into CI/CD and MLOps pipelines to support continuous delivery and model monitoring. - Generate quality reports, dashboards, and actionable insights for engineering and leadership teams. - Monitor production performance, identify model drift or degradation, and document behavioral changes across model versions. - Establish scalable QA standards and mentor teams on testing best practices, documentation, and quality processes. - Serve as the primary reference point for AI quality standards, testing governance, and risk management. ## Requirements - 3–4 years of experience in software testing, with significant ownership or leadership experience in AI, Deep Learning, LLM, or Generative AI testing environments. - Strong hands-on experience testing non-deterministic AI systems, machine learning models, and advanced language models. - Excellent Python programming skills with experience in test automation, data validation, and quality engineering frameworks. - Strong understanding of transformer architectures, deep learning workflows, and model evaluation methodologies. - Proven ability to create clear, structured, and maintainable technical documentation. - Experience developing testing strategies for prompt engineering, conversational AI systems, and retrieval-augmented generation (RAG) architectures. - Familiarity with CI/CD pipelines, MLOps practices, and production monitoring for AI systems. - Strong analytical and problem-solving skills with the ability to work effectively in fast-paced and ambiguous startup environments. - Excellent communication and stakeholder management skills, with the ability to collaborate across technical and non-technical teams. - Experience with Vision-Language Models, multimodal AI systems, or computer vision technologies is highly desirable. - Familiarity with tools such as LangChain, LlamaIndex, MLflow, vector databases, and embedding technologies is considered a strong advantage. - Exposure to AI governance, compliance requirements, and documentation of model risks and limitations is a plus. ## Be…