3 of 3 Remote/Hybrid PySpark Jobs in the North of England

structured and unstructured datasets Validate large-scale data ingestion, transformation, and processing workflows Ensure data integrity, transformation accuracy, reconciliation, and performance under load Oversee PySpark-based data validation and interrogation activities on AWS EMR Support testing and monitoring of workflows orchestrated through Apache Airflow Drive root cause analysis … DynamoDB, AWS CloudWatch, AWS IAM, Python, SQL, Java, Bash, GitLab CI/CD, Jupyter Notebooks, Apache Spark/PySpark, Terraform, Docker Key Skills Strong proficiency in Python, SQL, Java, and Scripting languages such as Bash Hands-on experience testing Big Data platforms and Data Lake solutions Strong experience with ...

maintain ETL/ELT pipelines and data models in Fabric (Data Factory, Notebooks, Spark) Write high-performance Spark SQL, T-SQL, Python/PySpark Manage ingestion, transformation, and loading from multiple sources Translate stakeholder requirements into scalable technical solutions Mentor team members and establish engineering standards, security, and governance … assisted development tools like GitHub Copilot, ChatGPT, and Fabric Copilot Essential Experience: Microsoft Fabric is essential Azure Data ecosystem Lakehouse architectures & Data Factory Python, PySpark, Spark SQL Proven hands-on delivery in this stack If you're an experienced Data Engineer with strong Microsoft Fabric and Azure experience, this ...

pricing models with advanced data science techniques using WTW Radar and Emblem. Query and engineer large datasets (e.g., Python/R/SQL/PySpark) on modern platforms (e.g., Azure Databricks). Create innovative data solutions and present actionable insights to stakeholders. Communicate results clearly to key decision-makers … Street Pricing. What we're looking for: Experience in general insurance pricing with strong modelling skills. Proficiency in programming languages such as R, Python, PySpark, SAS, or SQL. Good knowledge of Azure Databricks and modern data technologies. Experience with WTW Radar software is preferred. A quantitative degree in Mathematics ...