Data Engineering Lead : Data Acquisition
Exciting opportunity for a Data Engineering Lead (Data Acquisition) to join this global Energy company.
This is an exciting time to join this world-leading Energy company as it evolves from a traditional Oil & Gas enterprise into a future-focused Energy leader.
You will lead a data acquisition function within an Energy Trading & Supply organisation, owning ingestion strategy, pipeline delivery on Azure and a team of data engineers serving Trading, Risk, Finance and Operations.
Key Responsibilities
• Own and define the data ingestion strategy; design and build scalable pipelines on Azure (ADLS, ADF, Databricks, Synapse, Event Hubs/Kafka) with robust observability and SLAs.
• Translate requirements from trading, risk, operations and finance into a prioritised engineering backlog, working closely with architects and product owners.
• Serve as the domain's Data Manager, stewarding data definitions, lineage, quality, metadata and access policies.
• Lead, coach and develop a team of data engineers, driving high standards in coding, design and production operations.
• Ensure data platforms are resilient, secure and compliant with relevant trading regulations (REMIT/EMIR).
Experience & Skills
• 10+ years in data engineering, with 3–5 years leading teams in complex, real-time environments.
• Strong hands-on expertise in Azure data services and Python, SQL, Spark/PySpark.
• Solid understanding of streaming and batch ingestion patterns, schema management and API integration.
• Proven ability to lead engineering teams in an Agile/DevOps environment with CI/CD practices.
• Experience with energy trading systems (ETRM/Endur), market data platforms (ICE, Platts, Bloomberg) and SAP is advantageous.
• Degree in Computer Science/Engineering or equivalent; Azure or Databricks certifications desirable.
With a focus within Energy Trading, Oil & Gas, Financial Markets and Commodities, we offer a transparent Recruitment Service that has proven to be reliable and effective for over 40 years. We are ISO accredited and proud of our excellent TrustPilot Reviews.
Eaglecliff Ltd is acting in the capacity of an employment agency for permanent recruitment and an employment business for contractor resourcing.