Data Engineer
Data Engineer
Python | AWS | Databricks
We’re working with a leading organisation operating at the forefront of the energy sector, playing a critical role in supporting the transition to a more sustainable, low-carbon future.
As the business continues to invest in its data capabilities, they are looking for a Data Engineer to join their growing Data Platform team. This is an opportunity to work in a complex, evolving environment where data is central to decision-making across trading, operations, and commercial functions.
The Role
You’ll be responsible for building and maintaining scalable data pipelines that power analytics and reporting across the organisation. Working within a modern Databricks on AWS environment, you’ll help deliver reliable, high-quality data to support business insights.
Key Responsibilities
- Build and maintain scalable ETL/ELT pipelines using Python and PySpark
- Ingest and integrate data from multiple sources including trading, finance, and operational systems
- Work within a Databricks Lakehouse architecture to transform and deliver data
- Optimise Spark jobs for performance and cost efficiency
- Implement and maintain data quality, governance, and reliability standards
- Support the full data lifecycle from ingestion through to reporting and insight delivery
- Collaborate with BI teams and stakeholders to ensure data is accessible and usable
Key Skills & Experience
- Strong experience in Data Engineering (3–6 years)
- Proficiency in Python and PySpark for large-scale data processing
- Hands-on experience with Databricks (Delta Lake, Workflows, Lakehouse architecture)
- Solid knowledge of AWS services (e.g. S3, Kinesis, Lambda, IAM)
- Experience building production-grade data pipelines
- Good understanding of data governance and data quality frameworks
- Strong SQL skills and experience working with complex datasets
Desirable
- Experience within energy, utilities, or similar complex industries
- Exposure to CI/CD for data pipelines or modern data platform tooling