19 of 19 Remote/Hybrid PySpark Jobs in London

services such as Synapse, Data Factory, Databricks or Fabric Strong SQL skills with experience building optimised, complex queries Proficiency in Python and/or PySpark for data transformation and automation Strong experience with Power BI and modern data modelling techniques Demonstrable experience leading data engineering initiatives or projects Excellent ...

Designing and building cloud based data and machine learning pipelines that prepare data for analytics, AI, and product use ·Writing clear, well-structured Python, PySpark, and SQL to transform and validate data from multiple upstream sources ·Taking ownership of data quality, consistency, and reliability across the pipeline lifecycle ·Shaping ...

design and ETL logic. DevOps: Hands-on experience with Azure DevOps , CI/CD pipelines, and Key Vault. Bonus: Experience with Azure Databricks or PySpark is highly desirable. Randstad Technologies is acting as an Employment Business in relation to this vacancy. ...

Essential Strong experience with Azure Databricks Infrastructure as Code using Terraform DevOps practices and CI/CD pipelines Git and peer review workflows Python, PySpark, and SQL Cloud operations within Azure To apply for this role please submit your CV or contact Dillon Blackburn on or at . Tenth ...

CDF. High technical proficiency with Cosmos DB SDKs (e.g., .NET, Java, Python, Node.js) Thorough understanding of Spark distributed computing concepts. High technical proficiency in PySpark and Spark Concepts. Experience with concurrency patterns, CLR, and scalable application design. Deep understanding of Azure services including Azure Functions, App Services ...

Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

Lloyd's of London or London Market insurance sector Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns and data modelling. Strong Python, PySpark and Spark SQL capability Solid understanding of data governance Proven track record of ownership, delivery and operating effectively in fast-paced environments. What ...

Experience 5+ years in data engineering, including experience in a senior or lead position Strong expertise across AWS Hands-on experience with Databricks, PySpark, and Lakehouse patterns Strong Python and SQL engineering ability Experience implementing scalable ETL/ELT pipelines Background in data modelling and building analytics-ready datasets ...

Lead Data Engineer - Pyspark/AWS/Python/SAS - Financial Sector As a Lead PySpark Engineer, you will design, develop, and fix complex data processing solutions using PySpark on AWS. You will work hands-on with code, modernising legacy data workflows and supporting large-scale … PySpark migrations. The role requires strong engineering discipline, deep data understanding, and the ability to deliver production-ready data pipelines in a financial services environment. Essential Skills PySpark & Data Engineering Minimum 5+ years of hands-on PySpark experience. SAS to Pyspark migration experience Proven ability ...

Experience Strong hands-on experience with Azure Cosmos DB Query optimisation and throughput management expertise Experience with Cosmos DB Change Data Feed (CDF) Strong PySpark and Spark distributed computing knowledge Experience integrating Spark with Cosmos CDF High proficiency with Cosmos DB SDKs (.NET, Java, Python, Node.js) Deep understanding ...

model development. What you’ll bring: Minimum of two years’ experience in a data science, machine learning, or similar role Strong Python (including PySpark) and SQL skills for data analysis, feature engineering, and modelling Experience building and deploying end-to-end predictive models from scoping through validation and stakeholder ...

different customer contexts and priorities Design and implement AI-enabled workflows that support decision-making and automation Build and maintain data integrations using Python, PySpark, SQL , and distributed systems Partner with customers to define success metrics and ensure solutions deliver real outcomes Feed learnings from the field back into ...

engineering is done across the organisation. Skills & Experience Strong hands-on experience with Azure data platforms Advanced SQL skills Commercial experience using Databricks and PySpark Proven background building and maintaining scalable data pipelines If you're looking for a role where you can combine technical depth, ownership, and influence ...

Your Role: Design, develop, and optimise large-scale cloud data platforms using Azure Data Factory, Data Lake, Key Vault, Azure Functions, Databricks, Delta Lake, PySpark, and Unity Catalog. Collaborate with underwriting, actuarial, Delegated Authority, bordereaux, exposure management, reinsurance, finance, and Solvency II teams to meet Lloyd's market … highly regulated environments. Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns, distributed compute, data modelling, and performance optimisation. Strong programming skills in Python, PySpark, and Spark SQL, with experience implementing CI/CD pipelines (Azure DevOps, branching/merging, automation). Solid knowledge of data governance, lineage, access ...

Defining and enforcing engineering standards across code quality, testing, CI/CD, IaC, and observability Building and optimising ELT/ETL pipelines using Glue (PySpark) , Lambda, Step Functions, and event-driven patterns Championing security-by-design in collaboration with security and assurance teams (IAM, KMS, encryption, auditability) Implementing … across data analytics and data engineering domains Strong hands-on experience with: Amazon S3 (data lake patterns, partitioning, lifecycle management) AWS Glue (Jobs, Crawlers, PySpark, orchestration) Amazon Redshift (data modelling, performance tuning, WLM, Spectrum) Advanced Python (PySpark) and SQL skills Experience implementing CI/CD pipelines and Infrastructure ...

Lead PySpark Engineer (Cloud Migration) Role Type: 5-Month Contract Location: Remote (UK-Based) Experience Level: Lead/Senior (5+ years PySpark) Role Overview We are seeking a Lead PySpark Engineer to drive a large-scale data modernisation project, transitioning legacy data workflows into a high-performance … cloud environment. This is a hands-on technical role focused on converting legacy SAS code into production-ready PySpark pipelines within a complex financial services landscape. Key Responsibilities Code Conversion: Lead the end-to-end migration of SAS code (Base SAS, Macros, DI Studio) to PySpark using automated ...

tagging, monitoring and platform operations. * Architect and design end-to-end data platform solutions focused on scalability, security and reliability. * Lead delivery using Databricks, PySpark, Spark SQL and Azure Data Factory. * Drive DevOps best practices including CI/CD, automated testing and infrastructure as code. * Collaborate closely with data … followed. Your Skills and Experience You will bring: * Strong commercial experience in Azure and Databricks, with deep knowledge of Lakehouse architecture. * Proficiency in PySpark, Python, SQL and Spark SQL. * Strong DevOps engineering skills including CI/CD, automated testing and deployment pipelines. * Experience with monitoring, logging and alerting solutions. ...

Lakehouse solutions across cloud environments. Responsibilities: Design, build, and maintain data pipelines using Databricks (Spark, Delta Lake). Develop batch and streaming workloads using PySpark/Spark SQL. Implement data ingestion, transformation, and orchestration pipelines. Optimise Databricks workloads for performance, reliability, and cost. Integrate Databricks with cloud-native services … across Azure, AWS, or GCP. Skills: Strong hands-on expertise with Databricks in production environments. Advanced PySpark and Spark SQL development experience. Experience building lakehouse architectures using Delta Lake. Strong cloud experience in Azure (preferred), AWS, or GCP. Experience with orchestration tools (ADF, Airflow, dbt, or similar). Certifications ...

champion best practice methods; upskill the team through coaching and code reviews. Query and engineer large datasets (e.g., Python/R/SQL/PySpark) on modern platforms (e.g., Azure Databricks). Communicate pricing proposals and insights clearly and persuasively to stakeholders at all levels. Collaborate with Modelling, Optimisation … insurance pricing. Strong technical expertise in predictive modelling techniques (e.g. GLMs and machine learning models). Proficiency in programming languages such as R, Python, PySpark, SAS, or SQL. Good knowledge of Azure Databricks and modern data technologies. Experience with WTW Radar software and optimisation tools. A quantitative degree ...