3 of 3 Delta Lake Jobs in the North of England

Apache Spark (PySpark or Scala) and Structured Streaming . Experience working with Kafka (MSK) and handling real-time data . Good knowledge of Delta Lake/Delta Live Tables and the Medallion architecture . Hands-on experience with AWS services such as S3, Glue, Lambda, Batch ...

optimize ETL/ELT workflows that ingest data from various on-prem and cloud-based sources. * Work with Azure services including Azure Data Lake Storage, Azure Data Factory, Azure Synapse Analytics, Azure SQL, and Event Hub. * Implement data quality validation, monitoring, metadata management, and governance processes. * Collaborate closely with … Required Skills and Experience * Proven experience as a Data Engineer working with Azure cloud services. * Strong proficiency in Databricks, including PySpark, Spark SQL, notebooks, Delta Lake, and job orchestration. * Strong SQL and data modeling skills (e.g., dimensional modeling, data vault). * Experience with Azure Data Factory or other ...

stakeholders to understand analytical needs Ensure compliance with best practices around data governance, security, and quality Requirements: Strong experience with Databricks (jobs, notebooks, workflows, Delta Live Tables) Proficiency in PySpark and Python Hands-on experience with Azure Data Lake, Azure Data Factory, or similar cloud services Understanding … Delta Lake, streaming pipelines, and lakehouse architecture Solid knowledge of data engineering principles (ETL/ELT, modelling, optimisation) Benefits: Salary up to £70,000 Annual bonus Structured career development and funded certifications Opportunity to work on high-impact, enterprise-scale data transformation programmes ...