17 of 17 PySpark Jobs in Central London

analytics, data platforms, or regulatory reporting environments. Strong understanding of insurance data domains , including policy lifecycle, underwriting, claims, and risk data. Strong proficiency in PySpark, Python, and SQL for large-scale data processing. Extensive experience building ETL/ELT pipelines and scalable data architectures . Hands-on experience with ...

Maintain documentation, lineage, governance standards. Required Skills/Experience The ideal candidate will have the following: Strong SQL; Snowflake ELT; dbt experience. Python/PySpark, ETL/ELT design. Data Vault 2.0 or dimensional modeling. AWS services (S3, Glue, Lambda, Redshift) or GCP equivalents. Experience with CI/ ...

TensorFlow Serving, TorchServe). Familiarity with ML lifecycle tools such as MLflow, Kubeflow, or Azure ML Pipelines. Experience working with distributed data processing (e.g., PySpark) and SQL. Understanding of software engineering best practices, including version control (Git). Knowledge of CI/CD principles in ML environments. Experience with ...

basis, clarifying ownership and integration points, and capturing this information in a way that supports risk assessment and decommissioning decisions. Responsibilities: Python and PySpark are required as supporting capabilities, used where needed to analyse data pipelines and confirm how data moves and transforms in practice. The role also requires ...

scalable, secure data solutions using Azure Databricks, Data Factory and Data Lake Storage. Develop and optimise advanced data pipelines with Python, SQL, Spark/PySpark and Delta Lake. Champion strong data quality, governance and observability practices. Modernise legacy systems and support large-scale migrations into Azure. Provide technical leadership … Criteria Extensive experience with Azure services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory. Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing. Proven experience in CI/CD practices using industry-standard tools (e.g., GitHub Actions ...

data modelling for Finance and Enterprise data products, working closely with architecture where required. Implement and maintain data pipelines and ETL workflows in Databricks (PySpark, Delta Lake). Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform) in line with established … influence within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing ...

basis, clarifying ownership and integration points, and capturing this information in a way that supports risk assessment and decommissioning decisions. Responsibilities: Python and PySpark are required as supporting capabilities, used where needed to analyse data pipelines and confirm how data moves and transforms in practice. The role also requires ...

outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

machine learning operations. Excellent communication skills with the ability to influence cross-functional teams and stakeholders. Experience with Python, SQL, Java, HQL, Hive, Pyspark, or similar programming languages preferred Well-versed in design thinking principles Great problem-solving and analytical skills ...

AWS Data Engineer (Innovation Group) Location: London (Hybrid – 3 days per week) Duration: 6 Months Initial Contract Rate: TBD Are you a Data Engineer who thrives on building more than just standard ETL pipelines? Our ...

looking for a technically strong and commercially minded data professional to lead end-to-end model development. What you’ll bring: Strong Python (including PySpark) and SQL skills for data analysis, feature engineering, and modelling Experience building and deploying end-to-end predictive models from scoping through validation ...

their data pipelines, using Python, Spark, and Databricks Workflows to deliver new platform functionality and ensure efficient execution. Key Responsibilities Develop reliable Python and PySpark code to support data ingestion, transformation, and end‐to‐end processing. Deliver new technical features and components aligned to approved solution designs and business ...

PySpark Engineer Lead As the Technical Lead, you will drive the high-stakes migration of legacy SAS analytics to a modern, cloud-native PySpark ecosystem on AWS. This isn't just a lift and shift you will refactor complex procedural logic into scalable, production-ready distributed pipelines … Tier-1 financial services environment. Core Responsibilities Engineering Leadership: Design and develop complex ETL/ELT pipelines and Data Marts using PySpark, EMR, and Glue. Legacy Modernisation: Architect the conversion of SAS Base/Macros into modular, testable Python code using SAS2PY and manual refactoring. Performance Tuning: Optimise Spark ...

Title: Azure Cosmos DB Admin and Developer- with python, pyspark, Cosmos, Azure Location: Hybrid, London UK (3days/Week Onsite) Duration: 6months+ Contract Inside IR35 The Role The DB Admin and Developer will play a critical role in designing and developing scalable, secure, and high-performance applications tailored ...

requiring strong asynchronous communication skills and the ability to understand requirements with limited oversight. The role involves designing and building reusable Python libraries using PySpark/DataFrames, developing Python-based Azure Functions for microservices, and creating shared tooling to accelerate delivery across multiple teams. After completing the initial predefined … libraries that support common business needs in a fast-paced, high-ownership environment. Required Skills and Experience: Strong proficiency in Python. Experience with PySpark and DataFrame-based data processing. Solid understanding of SQL. Hands-on experience with Azure infrastructure. Good understanding of Containers, Microservices, and Functional design patterns. Experience ...

champion best practice methods; upskill the team through coaching and code reviews. Query and engineer large datasets (e.g., Python/R/SQL/PySpark) on modern platforms (e.g., Azure Databricks). Communicate pricing proposals and insights clearly and persuasively to stakeholders at all levels. Collaborate with Modelling, Optimisation … insurance pricing. Strong technical expertise in predictive modelling techniques (e.g. GLMs and machine learning models). Proficiency in programming languages such as R, Python, PySpark, SAS, or SQL. Good knowledge of Azure Databricks and modern data technologies. Experience with WTW Radar software and optimisation tools. A quantitative degree ...

TensorFlow Serving, TorchServe, ONNX, and TensorRT. Build and manage ML pipelines using MLflow, Kubeflow, and Azure ML Pipelines. Work with large-scale data using PySpark and integrate ML solutions into production environments. Monitor and improve model performance to ensure accuracy and efficiency. Contribute to team knowledge by mentoring … serving tools (ONNX, TensorRT, TensorFlow Serving, TorchServe). Experience with ML lifecycle tools (MLflow, Kubeflow, Azure ML Pipelines). Knowledge of distributed data processing (PySpark) and software engineering principles (Git). A collaborative mindset and excellent problem-solving abilities. Experience in data cleansing, exploratory data analysis, and visualisation. ...