QA Analyst Jobs in Alpharetta, GA

SUMMARY

We are seeking a highly experienced and strategic Domain Lead / Senior AI/ML & Generative AI Quality Engineer to join our Quality Engineering organization. In this role, you will validate and ensure the quality, performance, reliability, and scalability of enterprise-grade AI/ML and Generative AI applications. The ideal candidate will possess deep expertise in Python, AI/ML pipelines, GenAI frameworks, RAG architectures, and cloud-native deployments, blending cutting-edge AI knowledge with rigorous software quality engineering best practices across the entire development lifecycle.

KEY RESPONSIBILITIES:

- Testing Strategy & Leadership: Partner with Data Scientists, AI Engineers, Product Owners, and DevOps teams to architect robust testing strategies specifically tailored for AI/ML and Generative AI solutions.
- Quality & Validation: Validate and ensure the quality, accuracy, performance, reliability, compliance, and scalability of complex AI/ML models and GenAI applications.
- Automation Frameworks: Design, code, and implement scalable, reusable automated testing frameworks in Python to streamline AI validation processes.
- Pipeline Evaluation: Evaluate RAG pipelines, prompt optimizations, embeddings, vector databases, and agent-based architectures for deterministic and non-deterministic outputs.
- Cross-Functional Collaboration: Serve as the Quality Engineering subject matter expert (SME) to ensure AI solutions meet stringent business, security, and compliance requirements.

EXPERIENCE & QUALIFICATIONS

Experience Thresholds (Strict Minimums):

- Overall IT Experience: Minimum 10 years of professional experience in software engineering/testing.
- Core Quality Engineering: Minimum 8 years of dedicated experience in software quality engineering concepts, automation frameworks, and methodologies.
- AI/ML & GenAI Domain: Minimum 5 years of hands-on experience specifically focused on AI/ML pipelines and Generative AI technologies.

Technical Skill Requirements:

- Programming & Core Tech: Strong proficiency in Python (including FastAPI, Flask, async programming, and building/testing RESTful APIs). Ability to architect and code scalable, high-performing test systems.
- AI/ML Pipelines: Proven experience validating data preprocessing, model training, fine-tuning, and deployment using frameworks like TensorFlow, PyTorch, or Scikit-learn.
- Generative AI & RAG: Hands-on experience with GenAI frameworks: LangChain, LlamaIndex, LangGraph, AWS Bedrock, etc. Deep understanding of Retrieval-Augmented Generation (RAG) pipelines, including knowledge management with vector databases, embeddings, and graphs. Working knowledge of prompt engineering, prompt optimization, and LLM fine-tuning validation.
- Graph & Agent Architectures: Familiarity with Graph databases (Neo4j) and agent-based AI architectures.
- DevOps & Cloud Infrastructure: Solid understanding of CI/CD pipelines, containerization (Docker/Kubernetes), and cloud deployment (specifically AWS).

Soft Skills:

- Excellent debugging, performance optimization, and architectural solutioning skills.
- Strong communication skills to lead quality initiatives across cross-functional teams.