AI Engineer

Location: Hybrid / London 3 days

Length: Initially 6 months

Role Overview

We are seeking an experienced AI Engineer to establish and scale quality assurance, validation, and evaluation frameworks for enterprise AI and analytics solutions. The role will focus on ensuring the accuracy, safety, reliability, and performance of semantic models, knowledge graphs, derived data assets, and AI-powered query systems.

Working closely with AI Engineers, Data Engineers, Analytics Engineers, and Product teams, you will build automated testing and evaluation capabilities that enable continuous validation of AI-generated outputs across SQL, Cypher, semantic querying, and hybrid retrieval systems.

Key Responsibilities

Build and maintain golden-question datasets to benchmark AI solution quality and consistency.

Develop automated evaluation frameworks for semantic mappings, measures, derived assets, SQL/Cypher generation, graph retrieval, hybrid search, and multi-intent question answering.

Design and execute validation tests covering data equivalence, join integrity, graph projection accuracy, query safety, access controls, regression, performance, and concurrency.

Create contract, integration, and end-to-end test suites across mocked services, ClickHouse, BigQuery, and Neo4j environments.

Embed testing and evaluation processes into CI/CD pipelines to support continuous delivery and rapid feedback.

Establish repeatable validation across changes to data, semantic models, knowledge graphs, prompts, configurations, and derived assets.

Monitor quality metrics, identify regressions, and work with engineering teams to resolve defects and improve system reliability.

Support AI governance by ensuring outputs are accurate, secure, auditable, and compliant with organisational standards.

Strong experience in software testing, QA engineering, or platform validation.

Experience developing automated testing frameworks using Python.

Understanding of AI, LLM, RAG, semantic layer, or knowledge graph architectures.

Experience validating SQL-based analytical systems and complex data transformations.

Knowledge of graph databases and query languages such as Neo4j and Cypher.

Familiarity with modern data platforms including ClickHouse, BigQuery, Snowflake, Databricks, or similar environments.

Experience creating regression, integration, contract, and end-to-end test suites.

Understanding of CI/CD tooling such as GitHub Actions, Azure DevOps, GitLab CI, or Jenkins.

Strong analytical and problem-solving capabilities.