Role: QA Automation Engineer

Function: Quality Assurance

Location: Mumbai, India

Type: Full-time

Compensation: Competitive compensation and equity

Industry: AI/ML, Consumer Technology

About Company

A venture-backed AI company building the intelligence layer for India at massive scale. Backed by partnerships with global tech leaders like Meta and Google.

The team creates AI that serves the entire Indian user base across languages, contexts, and daily needs. Engineered from day one for 100M+ users early and 1B-ready constraints on latency, cost, reliability, and safety.

Position Overview

You'll build comprehensive testing frameworks for a Core Intelligence platform that powers AI workflows for millions of users across India. You'll design automation strategies that ensure quality and performance at massive scale, validating everything from product features to LLM outputs to multi-agent workflows. This role offers direct ownership of QA frameworks that will shape how AI systems operate reliably for 100M+ users.

Role & Responsibilities

• Design and implement comprehensive QA automation frameworks for AI/ML systems and large-scale distributed platforms
• Build automated testing suites using Playwright and PyTest for end-to-end validation of AI workflows
• Develop specialized testing frameworks for LLM validation, hallucination detection, and AI safety guardrails using LangSmith
• Create automated test scenarios for multi-agent coordination, task delegation, and partner integrations
• Implement CI/CD pipelines with automated quality gates for continuous deployment of AI features
• Build performance and load testing frameworks to validate 100M+ user scale constraints
• Collaborate with AI engineers to establish testing standards for model routing and orchestration systems

Must Have Criteria

• 5-10 years of hands-on QA automation experience with large-scale distributed systems
• Expert-level proficiency in Python automation frameworks, specifically PyTest
• Strong hands-on experience with Playwright for end-to-end web and mobile application testing
• Proven experience building and maintaining automated testing integration of CI/CD pipelines
• Experience with AI/ML testing frameworks and validation of machine learning model outputs
• Strong knowledge of JavaScript/TypeScript for React Native mobile app automation
• Experience testing APIs, microservices, and distributed system architectures

Nice to Have

• Experience with LangSmith, LangChain evaluation tools, or similar LLM testing frameworks
• Background in testing conversational AI, chatbots, or agent-based systems
• Experience with performance testing tools for high-scale consumer applications
• Knowledge of AI model evaluation metrics and automated quality assessment
• Previous experience in fintech, consumer tech, or AI-first product companies

What We Offer

• Opportunity to build QA frameworks for India's largest AI platform serving 100M+ users
• Work directly with cutting-edge AI technologies and multi-agent systems
• High-ownership environment with direct impact on product quality and user experience
• Competitive compensation and equity in a high-growth AI company
• Collaborative culture with top-tier AI engineers and product teams