Senior Quality Engineer – Test Automation, AI Testing & Java/Python
Astra-North Infoteck Inc.
Share this job:
X
Facebook
WhatsApp
Save job : Senior Quality Engineer – Test Automation, AI Testing & Java/Python
Senior Quality Engineer – Test Automation, AI Testing & Java/Python
Quality Engineer– Test Automation, Java/Python & GenAI
Roles and Responsibilities:
- Partner with DevOps,Development, Product, andQE teams todesign and executeexploratory, automated,and AI-focusedtests that ensureproduct quality.
- Define quality strategies, test plans, testcases, and testdata for functional, integration, system, regression, API, database, andacceptance testing.
- Design, develop, andmaintain scalable testautomation frameworks usingJava, Python, orGroovy.
- Build automated testsfor AI-enabledfeatures, including promptbehavior, response accuracy, hallucination risks, safety, reliability, latency,and regression coverage.
- Validate AI integrations, APIs, microservices, datapipelines, model interactions, and third-partyservices.
- Establish testing approachesfor both deterministicand non-deterministicAI behavior, includingevaluation datasets, expectedbehavior, quality thresholds, and repeatability.
- Integrate automated testingand AI qualitychecks into CI/CD pipelines inpartnership with DevOpsteams.
- Use tools andframeworks such asSelenium, Cucumber/Gherkin, Playwright, RestAssured, JUnit, TestNG, ReadyAPI, and JMeter.
- Identify, document, prioritize, and track defectsthrough resolution.
- Analyze test coverage, quality trends, AIevaluation results, andproduction feedback toidentify gaps anddrive continuous improvement.
- Collaborate with cross-functional Agile teamsto deliver reliable, secure, and user-focused product features.
- Communicate test results, risks, quality metrics, and release recommendationsclearly to technicaland non-technical stakeholders.
Required Skills:
- 5+ years ofrelevant experience inQuality Engineering, QualityAssurance, test automation, or software development.
- 5+ years ofprogramming experiencewith Java,Python, Groovy, ora comparable language.
- 5+ years ofexperience building andmaintaining test automationframeworks.
- Hands-on experiencewith Selenium, Cucumber/Gherkin, Playwright, RestAssured, or similarautomation technologies.
- Experience testing functionaland non-functionalrequirements, APIs, integrations, databases, microservices, and regression suites.
- Experience with testingframeworks and toolssuch as JUnit, TestNG, qTest, ReadyAPI, and JMeter.
- Understanding of TDD, ATDD, BDD, continuoustesting, and risk-based testing practices.
- Experience testing AIor machine learning-powered applications, APIs, conversational systems, or generative AIfeatures.
- Understanding of AIquality concepts suchas prompt evaluation, grounding, accuracy, relevance, toxicity, bias, hallucination detection, andresponse consistency.
- Ability to createeffective test dataand evaluation datasetsfor AI-drivenfunctionality.
- Strong analytical, problem-solving, communication,and technical documentationskills.
- Abilityto work effectivelyin a fast-paced, cross-functionalAgile environment.
Nice to Have:
- Experience with AIevaluation frameworks,LLM testing, promptengineering, retrieval-augmented generation, embeddings, or model observability.
- Experience with Git, Bitbucket, Jenkins, CI/CD pipelines, Artifactory, SonarQube, Jira, Confluence, AWS, Kubernetes, Maven, orGradle.
- Experience testing cloud-native applications anddistributed microservicearchitectures.
- Experience with security, performance, accessibility, or reliability testing.
- Experience developingautomation scripts andreusable testing utilities.
- Experience with Tableau orother analytics andvisualization platforms.
- Experience working directlywith DevOps teamsto implement automatedquality gates andrelease controls.
Reference: WJ-4483_1646103 · Posted 16 Nov 2026, 05:26 PM