2 of 2 Permanent PySpark Jobs in South Yorkshire

making. Key Responsibilities: Lead the design and development of curated, analytics-ready dimensional and semantic data models. Build, maintain and optimise complex SQL and PySpark transformation pipelines within Databricks. Define and maintain KPI definitions, conformed dimensions and consistent business logic across data domains. Ensure data quality, integrity, testing, observability ...

Strong knowledge of migrating and transforming data from legacy and on-premise systems to cloud platforms. Expertise in automating ETL/ELT pipelines using PySpark and Spark SQL across Azure-based data environments. You and your role As a Data Engineer, you will design build and enhance scalable …/ELT pipelines using Python and PySpark, ensuring data solutions are reliable, efficient and aligned with engineering best practice. You'll work closely with stakeholders to understand complex business requirements, transforming data into trusted, high-quality datasets that enable valuable insights, reporting and decision-making. You will manage ...