AI Systems Engineer RAG Evaluation
Luxoft
Shape reliable AI systems through retrieval, RAG, evaluation, and monitoring for an energy platform supporting complex portfolio decisions. Use Python or Node.js daily.
As a mid-to-senior Applied AI Engineer, you will improve how AI quality, cost, and latency are measured in production. You will build evaluation sets, automated graders, human review processes, and monitoring. The platform uses Ruby on Rails, Node.js/TypeScript, Python, PostgreSQL, Snowflake, and AWS.
Key Responsibilities:
- Build evals tracking quality, cost, and latency
- Improve systems using evaluation results
- Own retrieval and RAG across company data
- Define measurable success for AI applications
- Support customer-facing AI feature refinement
Requirements:
- Production experience delivering LLM features or agents
- Python preferred, or Node.js/TypeScript production experience
- Experience with representative eval sets and automated graders
- Human review and production AI monitoring experience
- Enterprise cloud experience, preferably AWS Bedrock
Bring evaluation, monitoring, retrieval, and RAG expertise to strengthen production AI quality across energy products.
#J-18808-LjbffrReference: WJ-4483_1555847 · Posted 09 Nov 2026, 01:17 AM