3 of 3 Remote/Hybrid PySpark Jobs in the Midlands

complex data led organisation* Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI* Expertise in Streaming Data, Event Driven Design ...

complex data led organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design ...

ecosystem. This role will focus on building enterprise-grade data pipelines, migration frameworks and transformation processes using Azure Synapse Analytics, Azure Data Factory (ADF), PySpark, Python and SQL. You will work closely with technical and business stakeholders to ensure successful delivery of data migration and integration initiatives. Key Responsibilities … Data Factory (ADF). Develop Change Data Capture (CDC) solutions to support incremental and delta data loads. Create data transformation and cleansing processes using PySpark and Spark DataFrames. Develop Python-based utilities and automation scripts to support migration and validation activities. Design and optimise data models, SQL queries, stored ...