11 of 11 Permanent PySpark Jobs in the North of England

Experience managing multi-engine compute environments CI/CD experience with Azure DevOps, GitHub, and Databricks Asset Bundles Strong programming skills in PySpark, Databricks SQL, and T-SQL Understanding of Medallion Architecture and modern data modelling approaches (Star Schema, Data Vault 2.0) Experience with performance optimisation in Databricks ...

including: CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing frameworks Hands-on experience with Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/ ...

platforms (AWS preferred). Exposure to data warehousing, data lakes and modern data architecture. Understanding of workflow orchestration and automation tools. Python experience (including PySpark) is beneficial but not essential. Exposure to BI platforms such as Power BI, Tableau or QuickSight, with an understanding of how backend data models ...

design decisions. Required Skills & Experience Strong analytical skills with experience working with large-scale and unstructured datasets. Python development experience, including technologies such as PySpark, Pandas and PyArrow. Expertise in distributed data processing frameworks, particularly Apache Spark. Experience building and maintaining ETL pipelines using technologies such as Apache Airflow ...

data modelling, DAX and semantic model designHands-on experience with Microsoft Fabric, particularly lakehouse architecture, notebooks and data pipelinesStrong understanding of Apache Spark/PySpark and large-scale data processingExperience designing and operating modern data platformsExperience developing or enabling AI and machine learning capabilities using business dataStrong understanding ...

semantic model design Hands-on experience with Microsoft Fabric, particularly lakehouse architecture, notebooks and data pipelines Strong understanding of Apache Spark/PySpark and large-scale data processing Experience designing and operating modern data platforms Experience developing or enabling AI and machine learning capabilities using business data Strong understanding ...

hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well-governed data assets … data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics, you will help deliver reliable ...

structured and unstructured datasets Validate large-scale data ingestion, transformation, and processing workflows Ensure data integrity, transformation accuracy, reconciliation, and performance under load Oversee PySpark-based data validation and interrogation activities on AWS EMR Support testing and monitoring of workflows orchestrated through Apache Airflow Drive root cause analysis … DynamoDB, AWS CloudWatch, AWS IAM, Python, SQL, Java, Bash, GitLab CI/CD, Jupyter Notebooks, Apache Spark/PySpark, Terraform, Docker Key Skills Strong proficiency in Python, SQL, Java, and Scripting languages such as Bash Hands-on experience testing Big Data platforms and Data Lake solutions Strong experience with ...

Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client is a global innovator and world leader … with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience is also key, as are excellent communication skills ...

sophisticated pricing models with advanced data science techniques using WTW Radar and Emblem.Query and engineer large datasets (e.g., Python/R/SQL/PySpark) on modern platforms (e.g., Azure Databricks).Create innovative data solutions and present actionable insights to stakeholders.Communicate results clearly to key decision-makers to influence … Optimisation, and Street Pricing.What we're looking for:Experience in general insurance pricing with strong modelling skills.Proficiency in programming languages such as R, Python, PySpark, SAS, or SQL.Good knowledge of Azure Databricks and modern data technologies.Experience with WTW Radar software is preferred.A quantitative degree in Mathematics, Statistics, Engineering, Physics ...

pricing models with advanced data science techniques using WTW Radar and Emblem. Query and engineer large datasets (e.g., Python/R/SQL/PySpark) on modern platforms (e.g., Azure Databricks). Create innovative data solutions and present actionable insights to stakeholders. Communicate results clearly to key decision-makers … Street Pricing. What we're looking for: Experience in general insurance pricing with strong modelling skills. Proficiency in programming languages such as R, Python, PySpark, SAS, or SQL. Good knowledge of Azure Databricks and modern data technologies. Experience with WTW Radar software is preferred. A quantitative degree in Mathematics ...