Principal AI Data Engineer : Commodity Trading
Eaglecliff Recruitment
As Principal AI Data Engineer, you will lead the design and delivery of scalable data and AI capabilities for a global commodity trader. You will bridge prototyping and production, shaping enterprise AI solutions and data platforms. You’ll work with cross-functional teams to translate requirements into robust implementations and establish best practices. This role offers hands-on engineering, AI experimentation, and technical leadership with a clear impact on the data platform and AI maturity of the business. You will drive architecture and delivery at scale in a dynamic trading environment.
Responsibilities- Design, develop and deploy GenAI, Agentic AI and RAG solutions
- Build scalable data pipelines and architectures for structured and unstructured datasets
- Develop AI agents and workflows using modern LLMs and frameworks
- Implement AI evaluation, testing and benchmarking frameworks
- Leverage Databricks, MLflow, Python and cloud-based AI services
- Establish engineering best practices across CI/CD, testing, version control and deployment
- Contribute to solution architecture, technical roadmaps and delivery backlogs
- Work closely with business stakeholders to translate requirements into production-ready solutions
- 8-12 years of experience
- Strong hands-on experience in AI, GenAI and data engineering
- Advanced knowledge of Databricks, MLflow, Delta Lake and data architecture
- Strong Python and SQL programming skills
- Experience with LLMs, RAG architectures, AI agents, LangChain and LangGraph
- Understanding of data pipelines, ETL/ELT, streaming and cloud technologies, preferably Azure
- Experience with AI evaluation, benchmarking and production deployment
- Strong software engineering practices, including Git, CI/CD and automated testing
- Excellent problem-solving, communication and stakeholder management skills
- Ability to provide technical leadership, drive delivery and work independently
- problem-solving
- communication
- stakeholder management
- Databricks
- MLflow
- Delta Lake
Reference: WJ-747_31297126 · Posted 17 Nov 2026, 05:59 AM