AI Evaluation Test Automation Engineer
CrowdStrike Inc.
Shape trusted AI evaluation at CrowdStrike using agent frameworks, LLM APIs, and verification-first test automation. Work on cloud-based cybersecurity systems. As a Software Development Engineer in Test, you will help determine how CrowdStrike AI agents are tested, evaluated, and trusted in production. The position combines test automation with eval-driven development across prompts, models, tools, and orchestration. You will support functional, integration, performance, and end-to-end validation. Key Responsibilities:
Maintain golden datasets and scoring pipelines for AI agents Measure correctness, tool use, safety, latency, and cost Feed production failures into evaluation sets Build AI-powered workflows for test data and triage Review agent designs for testability and evaluability Requirements:
Computer Science degree or equivalent experience Knowledge of object-oriented programming and distributed systems Python preferred for evaluation and agent work Experience testing REST, gRPC, and JSON services Verification-first judgment about probabilistic AI output Use AI evaluation, test automation, Python, and distributed-systems skills to improve trusted production testing at CrowdStrike.
#J-18808-Ljbffr
Maintain golden datasets and scoring pipelines for AI agents Measure correctness, tool use, safety, latency, and cost Feed production failures into evaluation sets Build AI-powered workflows for test data and triage Review agent designs for testability and evaluability Requirements:
Computer Science degree or equivalent experience Knowledge of object-oriented programming and distributed systems Python preferred for evaluation and agent work Experience testing REST, gRPC, and JSON services Verification-first judgment about probabilistic AI output Use AI evaluation, test automation, Python, and distributed-systems skills to improve trusted production testing at CrowdStrike.
#J-18808-Ljbffr
Reference: WJ-3875_16269578 · Posted 13 Nov 2026, 06:10 AM