Job Description & Requirements
ABOUT THIS FEATURED OPPORTUNITY
The QA Engineer will join the Channel Sales andOperations team to help ensure the reliability and quality of our AI/ML-poweredB2B chatbot and the foundational platforms supporting it. This role goes beyondtesting outputs you'll be working closely with engineers to ensure theend-to-end system, including cloud infrastructure and data pipelines, functionsas intended.
THE OPPORTUNITY FOR YOU
- Design and execute manual and automated test cases for GenAI platforms and chatbot systems.
- Build and maintain Python-based test automation frameworks for backend services and ML pipeline validation.
- Utilize RAGAS or similar tools to assess LLM outputs for factuality, relevance, and system performance.
- Conduct end-to-end testing from data ingestion to user-facing output.
- Validate system stability across GCP cloud components compute, storage, networking, and containers.
- Identify failures not only in chatbot answers, but also in underlying infrastructure and platform behavior.
- Collaborate with DevOps and ML engineers to triage bugs and optimize performance.
- Ensure test coverage spans across multiple deployment environments including Kubernetes clusters and cloud VMs.
Requirements
KEY SUCCESS FACTORS
- 3+ years of QA Automation Engineering experience
- Experience with Playwright for automated application testing, including sign-in and SSO authentication flows
- Ability to design tests that validate output accuracy and system behavior across different user flows
- Experience with Python API testing using requests, Pytest, and integration into CI/CD and cloud environments
- Proficiency in writing and debugging Bash scripts used in CI/CD and cloud deployment workflows
- Experience with Cloud platforms ( GCP preferred), including Kubernetes (kubectl experience is a plus), virtual machines, databases, cloud networking and storage components
- Understanding of modern cloud architecture and how distributed components interconnect in production environments
NICE TO HAVES
- Experience with LLM testing frameworks like RAGAS , and ability to interpret metrics such as factuality, relevance, and performance
- Experience with monitoring and observability tools
- Familiarity with the end-to-end architecture of GenAI solutions , including vector stores, retrievers, embedding models, and inference systems
