1 to 25 of 34 Contract PySpark Jobs in London

Essential Skills for the AWS Data Engineer: Extensive hands-on experience with AWS data services Strong programming skills in Python (including libraries such as PySpark or Pandas) Solid understanding of data modelling, warehousing and architecture design within cloud environments Experience building and managing ETL/ELT workflows and data

data sources into Azure Databricks. Optimize pipelines for performance, reliability, and cost, incorporating data quality checks. Develop complex transformations and processing logic using Spark (PySpark/Scala) for cleaning, enrichment, and aggregation, ensuring accuracy and consistency across the data lifecycle. Work extensively with Unity Catalog, Delta Lake, Spark

Azure Storage. Excellent SQL and data modelling (star/snowflake, dimensional modelling). Knowledge of Power BI dataflows, DAX, and RLS. Experience with Python, PySpark, or T-SQL for transformations. Understanding of CI/CD and DevOps (Git, YAML pipelines). Strong grasp of data governance, security, and performance

Azure Storage. Excellent SQL and data modelling (star/snowflake, dimensional modelling). Knowledge of Power BI dataflows, DAX, and RLS. Experience with Python, PySpark, or T-SQL for transformations. Understanding of CI/CD and DevOps (Git, YAML pipelines). Strong grasp of data governance, security, and performance

experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks . Good proficiency in Python and Spark (PySpark) or Scala. Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns. Extensive experience with Azure data services, including Azure Data Factory

years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks. ·Good proficiency in Python and Spark (PySpark) or Scala. ·Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns. ·Extensive experience with Azure data services, including Azure Data

pipelines to ingest, transform, and load data from diverse sources (APIs, databases, files) into Azure Databricks. Implement data cleaning, validation, and enrichment using Spark (PySpark/Scala) and related tools to ensure quality and consistency. Utilize Unity Catalog, Delta Lake, Spark SQL, and best practices for Databricks development, optimization

Engineer in enterprise environments. Strong hands-on expertise with Azure Data Factory , Databricks , Synapse , and Azure Data Lake . Proficiency in SQL , Python , and PySpark . Experience with data modelling , ETL optimisation , and cloud migration projects . Familiarity with Agile delivery and CI/CD pipelines. Excellent communication skills

Engineer in enterprise environments. Strong hands-on expertise with Azure Data Factory , Databricks , Synapse , and Azure Data Lake . Proficiency in SQL , Python , and PySpark . Experience with data modelling , ETL optimisation , and cloud migration projects . Familiarity with Agile delivery and CI/CD pipelines. Excellent communication skills

patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data quality, data governance processes, Git version control + Agile development environments. Azure

Active Security Clearance (SC) Strong Python skills with modular, test-driven design Experience with Behave for unit and BDD testing (mocking, patching) Proficiency in PySpark and distributed Data processing Solid understanding of Delta Lake (design and maintenance) Hands-on with Docker for development and deployment Familiarity with Azure services

Active Security Clearance (SC) Strong Python skills with modular, test-driven design Experience with Behave for unit and BDD testing (mocking, patching) Proficiency in PySpark and distributed Data processing Solid understanding of Delta Lake (design and maintenance) Hands-on with Docker for development and deployment Familiarity with Azure services

data governance, security, and compliance. Key Skills & Experience* Proven experience as an Azure Data Engineer.* Strong hands-on expertise with Databricks - 5+ years experience (PySpark, notebooks, clusters, Delta Lake).* Solid knowledge of Azure services (Data Lake, Synapse, Data Factory, Event Hub).* Experience working with DevOps teams

enabling advanced analytics and data-driven decision making across the business. Key Responsibilities Design, develop, and maintain ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake. Build and optimise data lakehouse architectures on Azure Data Lake Storage ( ADLS ) . Develop high-performance data solutions using Azure Synapse … Actions, or Azure DevOps Required Skills & Experience 3+ years' experience as a Data Engineer working in Azure environments. Strong hands-on experience with Databricks (PySpark, Delta Lake, cluster optimisation, job scheduling). Solid knowledge of Azure cloud services including: Azure Data Lake Storage Azure Data Factory Azure Synapse/

enabling advanced analytics and data-driven decision making across the business. Key Responsibilities Design, develop, and maintain ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake. Build and optimise data lakehouse architectures on Azure Data Lake Storage ( ADLS ) . Develop high-performance data solutions using Azure Synapse … Actions, or Azure DevOps Required Skills & Experience 3+ years' experience as a Data Engineer working in Azure environments. Strong hands-on experience with Databricks (PySpark, Delta Lake, cluster optimisation, job scheduling). Solid knowledge of Azure cloud services including: Azure Data Lake Storage Azure Data Factory Azure Synapse/

experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks . Good proficiency in Python and Spark (PySpark) or Scala click apply for full job details

engineers and influence data architecture across high-consequence programmes Build and optimise ontologies, pipelines, and semantic models Apply strong data engineering skills (Python, SQL, PySpark) Bring cloud expertise (AWS, Azure, or GCP) and proven team leadership If you have Palantir Foundry experience and want to take ownership of delivery

/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/Scala Experience: Experience working on large data sets, and complex data pipelines click apply for full job details

/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/Scala Experience: Experience working on large data sets, and complex data pipelines. Understanding of Data Architecture and Design, and Data pipeline optimisation. Proven

/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/Scala Experience: Experience working on large data sets, and complex data pipelines. Understanding of Data Architecture and Design, and Data pipeline optimisation. Proven

Senior Data Engineer (Developer) Pyspark We are supporting a leading global financial markets infrastructure and data provider as they modernise and scale their core data engineering capabilities. This role sits at the centre of their transformation programme, delivering high-quality data pipelines, models, and platforms that underpin critical services … repositories Essential Skills Strong experience developing within Microsoft Azure and Microsoft Fabric: Proficiency in Spark programming including DataFrames, RDDs, and Spark SQL Python and PySpark development experience, including notebook-based workflows Hands-on experience with Spark streaming and batch processing Delta table optimisation and Fabric Spark job development Solid

looking for an experienced PySpark Developer with strong Microsoft Fabric and Azure engineering skills to join a major transformation programme within the financial-markets domain. This role is fully hands-on, focused on building and optimising large-scale data pipelines, dataflows, semantic models, and lakehouse components. Key Responsibilities Design … stakeholders to translate requirements into scalable technical solutions Troubleshoot and improve reliability, latency and workload performance Essential Skills Strong hands-on experience with PySpark, Spark SQL, Spark Streaming, DataFrames Microsoft Fabric (Fabric Spark jobs, dataflows, pipelines, semantic models) Azure: ADLS, cloud data engineering, notebooks Python programming; Java exposure beneficial

Python Data Engineer Azure & PySpark - SC Cleared Contract £400-£458pd (Inside IR35) SC Clearance is Essential Summary Were looking for a Python Data Engineer skilled in PySpark, Delta Lake, Azure services, containerized development, and Behave-based testing. Youll design and build scalable data pipelines and maintain high-quality … test-driven code in a cloud environment. What youll do Build and maintain Python/PySpark pipelines for data ingestion, processing, and validation. Write unit and BDD tests using Behave, including mocking and patching. Create and optimize Delta Lake tables for reliable, performant data storage. Use Docker to manage

Description Role/Job Title: Developer (PySpark + Fabric) Work Location: London (Office Based) The Role The role will be integral to realizing the customer's vision and strategy in transforming some of their critical application and data engineering components. As a global financial markets infrastructure and data provider

experienced in building high-quality data pipelines and optimizing large-scale workloads on the Databricks Data Intelligence Platform. The ideal candidate is strong in PySpark, Delta Lake, Azure services, workflow automation, and performance tuning. Key Responsibilities Build and maintain scalable batch and streaming pipelines using Databricks notebooks, Jobs … ensure workloads are cost-efficient and reliable. Apply data quality, lineage, and access controls using Unity Catalog and governance best practices. Develop reusable PySpark ETL and data transformation code. Manage Delta Lake tables with ACID transactions, schema evolution, and time-travel capabilities. Integrate Databricks with Azure ADLS, Key Vault