Senior AI Coding Agent Evaluator
OpenTrain AI, Inc.
Evaluate AI-generated code remotely from Canada using software expertise, detailed rubrics, and strong written communication skills. Flexible contract work offers 20+ hours weekly.
As a Senior Software Engineer AI Data contributor, you will assess coding agents through codebases, tickets, source materials, and generated outputs. Your evidence-based reviews help improve AI system behavior. Prior AI training experience is not required.
Key Responsibilities:
- Evaluate model outputs against project guidelines and rubrics
- Review codebases, tickets, source materials, and generated code
- Document evidence-based judgments clearly in writing
- Explain decisions to improve model behavior
- Maintain consistent quality across repeated review tasks
Requirements:
- Relevant professional, academic, or language expertise
- Strong software and written communication skills
- Careful attention to detail during reviews
- Ability to apply detailed instructions and rubrics consistently
- Comfort working independently on remote contractor tasks
Apply software judgment, careful review, and clear writing to evaluate AI coding agents for ongoing remote contract work.
#J-18808-LjbffrReference: WJ-4483_1568429 · Posted 11 Nov 2026, 04:45 AM