3 of 3 PySpark Jobs in Bedfordshire

datalake migration initiatives, including consolidation of legacy data, medallion architecture design, schema evolution, and cutover planningProvide hands-on data engineering expertise, building and optimising PySpark pipelinesEnsure data is secure, high-quality, accessible, and structured for maximum business valueDrive best practice in data governance, cataloguing, lineage, and standards across … operations, analytics, and reporting teamsEnsure compliance with data protection regulations and internal policiesWhat you'll bring Strong hands-on experience in data engineering, including PySpark and production data pipelinesExperience with Microsoft Fabric, Azure Synapse, or similar platforms (e.g. Databricks, AWS)Proven ability to design and manage scalable data pipelines ...

migration initiatives, including consolidation of legacy data, medallion architecture design, schema evolution, and cutover planning Provide hands-on data engineering expertise, building and optimising PySpark pipelines Ensure data is secure, high-quality, accessible, and structured for maximum business value Drive best practice in data governance, cataloguing, lineage, and standards … reporting teams Ensure compliance with data protection regulations and internal policies What you'll bring Strong hands-on experience in data engineering, including PySpark and production data pipelines Experience with Microsoft Fabric, Azure Synapse, or similar platforms (e.g. Databricks, AWS) Proven ability to design and manage scalable data pipelines ...

development. Key Responsibilities Design and develop end-to-end data pipelines using reusable frameworks Build scalable data solutions using Python, SQL, and PySpark Create modular frameworks separating configuration from orchestration Implement robust automated testing (unit, integration, regression) Establish and enforce data quality controls (schema, nulls, row counts, integrity checks … based workflows Troubleshoot and optimise data pipelines for reliability and performance Required Experience Strong Data Engineering background Advanced Python and SQL skills Experience with PySpark or distributed processing tools Proven track record building reusable data frameworks Strong testing experience (pytest, mock data, data validation techniques) Solid understanding of data ...