IT & Software

Senior Quality Engineer – Test Automation, AI Testing & Java/Python

Astra-North Infoteck Inc.

Toronto · Ontario · Canada

Senior Quality Engineer – Test Automation, AI Testing & Java/Python

Quality Engineer– Test Automation, Java/Python & GenAI

Roles and Responsibilities:

  • Partner with DevOps,Development, Product, andQE teams todesign and executeexploratory, automated,and AI-focusedtests that ensureproduct quality.
  • Define quality strategies, test plans, testcases, and testdata for functional, integration, system, regression, API, database, andacceptance testing.
  • Design, develop, andmaintain scalable testautomation frameworks usingJava, Python, orGroovy.
  • Build automated testsfor AI-enabledfeatures, including promptbehavior, response accuracy, hallucination risks, safety, reliability, latency,and regression coverage.
  • Validate AI integrations, APIs, microservices, datapipelines, model interactions, and third-partyservices.
  • Establish testing approachesfor both deterministicand non-deterministicAI behavior, includingevaluation datasets, expectedbehavior, quality thresholds, and repeatability.
  • Integrate automated testingand AI qualitychecks into CI/CD pipelines inpartnership with DevOpsteams.
  • Use tools andframeworks such asSelenium, Cucumber/Gherkin, Playwright, RestAssured, JUnit, TestNG, ReadyAPI, and JMeter.
  • Identify, document, prioritize, and track defectsthrough resolution.
  • Analyze test coverage, quality trends, AIevaluation results, andproduction feedback toidentify gaps anddrive continuous improvement.
  • Collaborate with cross-functional Agile teamsto deliver reliable, secure, and user-focused product features.
  • Communicate test results, risks, quality metrics, and release recommendationsclearly to technicaland non-technical stakeholders.

Required Skills:

  • 5+ years ofrelevant experience inQuality Engineering, QualityAssurance, test automation, or software development.
  • 5+ years ofprogramming experiencewith Java,Python, Groovy, ora comparable language.
  • 5+ years ofexperience building andmaintaining test automationframeworks.
  • Hands-on experiencewith Selenium, Cucumber/Gherkin, Playwright, RestAssured, or similarautomation technologies.
  • Experience testing functionaland non-functionalrequirements, APIs, integrations, databases, microservices, and regression suites.
  • Experience with testingframeworks and toolssuch as JUnit, TestNG, qTest, ReadyAPI, and JMeter.
  • Understanding of TDD, ATDD, BDD, continuoustesting, and risk-based testing practices.
  • Experience testing AIor machine learning-powered applications, APIs, conversational systems, or generative AIfeatures.
  • Understanding of AIquality concepts suchas prompt evaluation, grounding, accuracy, relevance, toxicity, bias, hallucination detection, andresponse consistency.
  • Ability to createeffective test dataand evaluation datasetsfor AI-drivenfunctionality.
  • Strong analytical, problem-solving, communication,and technical documentationskills.
  • Abilityto work effectivelyin a fast-paced, cross-functionalAgile environment.

Nice to Have:

  • Experience with AIevaluation frameworks,LLM testing, promptengineering, retrieval-augmented generation, embeddings, or model observability.
  • Experience with Git, Bitbucket, Jenkins, CI/CD pipelines, Artifactory, SonarQube, Jira, Confluence, AWS, Kubernetes, Maven, orGradle.
  • Experience testing cloud-native applications anddistributed microservicearchitectures.
  • Experience with security, performance, accessibility, or reliability testing.
  • Experience developingautomation scripts andreusable testing utilities.
  • Experience with Tableau orother analytics andvisualization platforms.
  • Experience working directlywith DevOps teamsto implement automatedquality gates andrelease controls.
#J-18808-Ljbffr

Reference: WJ-4483_1646103 · Posted 16 Nov 2026, 05:26 PM

Find more on GigBlow

This role is listed on GigBlow for discovery and search. Hiring decisions and applications are handled by the employer or their chosen application system.

Quality Engineer jobs in other states

Browse jobs