Data Engineer - Azure databricks - python - pyspark - Azure Data Factory - Azure Data Lake

Data Engineer - Azure databricks - python - pyspark - Azure Data Factory - Azure Data Lake - CI/CD

• FTC role - 12 Months
• Salary - Up to £50,000 + Benefits
• Location - Remote

As a Junior Data Engineer, you will play a key role in the development and evolution of our cloud-based data platform. Leveraging technologies including Azure Databricks, Azure Data Factory and PySpark, you will design and deliver scalable data pipelines and lakehouse solutions that enable trusted, high-quality data across the organisation. Working within a modern Azure environment, you will help ensure data is efficiently managed, readily available and governed to support analytics, reporting and business decision-making.

Key Responsibilities

• Design, build and support scalable data platforms, pipelines and lakehouse solutions using Azure Databricks, Azure Data Factory and Python/PySpark.
• Partner with business stakeholders, analysts and technical teams to understand data requirements and deliver robust data solutions that support reporting, analytics and operational needs.
• Develop and maintain data ingestion, transformation and orchestration processes, integrating data from a variety of internal and external sources.
• Ensure data solutions are performant, reliable and maintainable, applying best practices in data engineering, automation and distributed processing.
• Monitor, troubleshoot and optimise data pipelines, proactively addressing issues relating to data quality, availability and performance.

Skills & Experience

• Commercial experience in a Data Engineering role, with strong expertise in Python/PySpark and the Azure data ecosystem.
• Hands-on experience with Azure Databricks, Azure Data Factory, Azure SQL, Azure Data Lake and related cloud technologies.
• Strong understanding of data modelling, ETL/ELT development and data integration principles.
• Knowledge of modern cloud-based data architectures, distributed computing concepts and large-scale data processing.
• Experience using Infrastructure as Code tools such as Terraform or Bicep, alongside CI/CD practices and Azure DevOps.
• Excellent communication and stakeholder management skills, with the ability to engage effectively with both technical and business audiences.