51 to 75 of 81 PySpark Jobs in London

solutions Technical modelling: Experience building models on both static and time series data, including propensity and forecasting models Coding skills: Advanced SQL, Python and PySpark skills, built on solid software development principles Stakeholder management: Ability to work with both technical and non-technical stakeholders, with strong communication skills ...

data platforms, analytics engineering, and modern Lakehouse solutions. The client is looking for a hands‐on engineering professional with strong Microsoft Fabric, Spark, PySpark and SparkSQL expertise, capable of designing, building and supporting enterprise‐scale data platforms. Start Date: ASAP Duration: 3 Months Initially Pay Rate: Competitive IR35 Status … Outside IR35 Location: London (1 day per week onsite, remainder remote) The Role: The core requirement is strong Lakehouse expertise, particularly Spark, PySpark, and SparkSQL. Hands‐on experience in low‐code is essential. Data & Analytics Expertise: Data architecture (Lakehouse, Medallion, Kimball, 3NF). Data ingestion and processing. Data quality ...

Design, build and maintain scalable and secure data pipelines on the Azure platform. Develop and deploy data ingestion processes using Azure Data Factory, Databricks (PySpark), and Azure Synapse Analytics . Optimise ETL/ELT processes to improve performance, reliability and efficiency. Integrate multiple data sources including Azure Data Lake … Required Skills & Experience - Azure Data Engineer: Proven commercial experience as a Data Engineer delivering enterprise-scale solutions in Azure Azure Data Factory Azure Databricks (PySpark) SQL & Python Understanding of CI/CD in a data environment , ideally with tools like Azure DevOps. Experience working within consultancy or client-facing ...

London, United Kingdom | Posted on 23/07/2026 Contract Role: Senior AI/ML Research Engineer Contract Location: London Contract Duration: 6 months initially (with potential extension) Contract Start Date: Immediate Contract Classification ...

London-based programme focused on data platforms, analytics engineering and Lakehouse solutions. You will design, build and support enterprise-scale data platforms with Spark, PySpark, SparkSQL and Microsoft Fabric. The role requires hands-on coding, data governance, ingestion and API integration capabilities, plus a passion for modern analytics ...

team. This role offers a genuine blend of leadership and engineering, with around 30-50% hands‐on development. Key Skills: Python & Databricks Spark/PySpark Technical leadership Data architecture & Lakehouse Agile delivery Stakeholder management #J-18808-Ljbffr ...

London is hiring a Data Technical Lead to own the Azure Databricks data platform, with hands-on experience in Structured Streaming, Delta Live Tables, PySpark, Python and SQL. The role is onsite three days per week on an initial contract basis through the end of the year. You will ...

together with reporting tools such as Tableau Desirable Experience Machine Learning and predictive modelling experience Experience working with large datasets and technologies such as PySpark or Hive Familiarity with version control systems such as Git Understanding of MLOps principles and production deployment practices Exposure to containerisation technologies such ...

global organization through resourcing/location strategy. Experience and Qualifications Proven track record delivering solutions in complex organizations. Experience with AWS, Databricks, DBT, PySpark, React, JavaScript, Terraform, etc. Fast learner, thrives in ambiguity, translates vision into product scope. Passion for automation and user experience innovation. Builds high‐performing teams ...

location strategy.**“I am the person Capital Group is looking for.”*** Proven track record delivering solutions in complex organizations.* Experience with AWS, Databricks, DBT, PySpark, React, JavaScript Terraform, etc.* Fast learner, thrives in ambiguity, translates vision into product scope.* Passion for automation and user experience innovation.* Builds high-performing ...

Senior Databricks Architect Technology: Databricks (Spark, PySpark/SQL) and data lake architectures. Location: London, UK. Compensation: Competitive (including bonus). Job Description We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling a modern data mesh architecture. The role … integrating legacy systems with cloud-native platforms to support analytics, reporting, and domain-owned data products. Primary Skills Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. Hands‐on experience with streaming technologies, especially Kafka/Confluent Kafka. Experience integrating legacy data sources such ...

Level Required in Years: 10 - 15 years Mandatory Required Skills Hands-on experience with Quantexa platform/Decision Intelligence Strong in Apache Spark/PySpark Proficiency in Python/Scala Experience in ETL, Data Engineering, Data Warehousing Knowledge of entity resolution, graph analytics, link analysis Experience working with large …/HDFS) Detailed Job Description Design and develop solutions using Quantexa Decision Intelligence Platform Build data ingestion, transformation, and ETL pipelines using Spark/PySpark Work on entity resolution, network analytics, and graph-based data models Integrate multiple structured/unstructured data sources for analytics use cases Ensure data ...

existing pipelines. Working within a growing data team, you'll design scalable data models, build robust ELT pipelines in Databricks using SQL, Python and PySpark, and integrate data from multiple internal and third-party systems. You'll collaborate closely with Analytics Engineers to ensure data is reliable, well-governed … architecture and help define long-term data standards, rather than simply delivering against predefined requirements. You'll need experience with: Databricks, SQL, Python and PySpark Building and optimising ELT/data pipelines Data modelling, lakehouse architecture and modern data platforms Azure data services API and third-party system integration ...

Senior Databricks Architect - London (Hybrid) - Databricks - Spark - PySpark - AWS - Up to £115,000 + Bonus Our client, a large-scale organisation undergoing significant investment in data and cloud technologies, is seeking an experienced Senior Databricks Architect to join its expanding Data & AI practice. This is an exciting opportunity … time data pipelines from legacy and modern source systems Implementing real-time and event-driven data processing solutions Developing data transformation frameworks using Spark, PySpark and SQL Designing Lakehouse architectures using Databricks and Medallion Architecture principles Creating scalable, reusable data products to support modern data operating models Integrating data ...

New Permanent Opportunity for an eDV Cleared Data Engineer with AWS Cloud Experience for a leading National Security Consultancy in London. Up to £85,000 DoE plus bonuses Active eDV required London Location – full-time ...

salary of up to £120,000. The successful candidate will engage with clients, focusing on architecture and data solutions while leveraging expertise in Spark, PySpark, and Delta Lake. You'll lead workshops, translate complex requirements, and contribute to modernizing enterprise data platforms. #J-18808-Ljbffr ...

Role –Senior Databricks Architect Technology – Databricks (Spark, PySpark/SQL) and data lake architectures Location – London, UK Duration-Permanent Job Description We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling a modern data mesh architecture. The role focuses … integrating legacy systems with cloud-native platforms to support analytics, reporting, and domain-owned data products. Primary Skills • Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. • Hands-on experience with streaming technologies, especially Kafka/Confluent Kafka. • Experience integrating legacy data sources such ...

adoption of modern data mesh principles across complex enterprise environments. What You'll Be Doing Architect and deliver modern data solutions using Databricks, Spark, PySpark and SQL Design scalable batch and real-time data pipelines that ingest data from legacy and modern platforms Lead the development of cloud-based … making Provide technical leadership and architectural direction on large-scale transformation programmes Essential Experience Strong hands-on expertise with Databricks Extensive experience using Spark, PySpark and SQL Proven track record designing and implementing data lake architectures Experience building both batch and streaming data pipelines Strong knowledge of Kafka Good ...

**Take the lead on meaningful, high-impact projects as a Senior Data Engineer.** We are seeking a Senior Data Engineer to join a growing organisation based in London, offering a hybrid work model. This role ...

Principal Consultant - Lead Data Analyst/Data Modeller Company: Transform Together Contract: 6-month Fixed Term Contract, with option to extend or become a permanent member of the team Compensation: OTE £90,000 Location: Hybrid ...

data processing solutions Work with modern data platforms such as Microsoft Fabric, Databricks, or similar technologies Build and optimise data solutions using Python, Spark, PySpark, and/or SQL Support data integration, transformation, and performance optimisation Collaborate with stakeholders to ensure high-quality, reliable data architecture Essential Skills & Experience … Hands-on experience with Microsoft Fabric, Databricks, or equivalent modern data platforms Strong coding experience in Python and/or Spark/PySpark Solid SQL skills and experience working with structured datasets Desirable Experience Knowledge of relational database design , normalisation , and data modelling Experience designing efficient, scalable data architectures ...

within the Lloyd’s marketplace Good understanding of tools and platforms used across the Lloyd’s insurance ecosystem Architect and optimize data solutions using PySpark and Datamart (GOS – Oracle as a Service). Develop and maintain data platforms leveraging Snowflake and Azure services (Blob Storage, Data Lake, Synapse Analytics … standards and best practices. Requirements: Proven experience in data engineering and architecture with strong expertise in ETL/ELT processes. Hands-on experience with PySpark, Snowflake, and Oracle-based data solutions. Proficiency in Azure ecosystem: Blob Storage, Data Lake, Synapse Analytics, Databricks, Functions, and Monitor. Strong understanding of data ...

Staff Data Engineer | AWS | PySpark | Data Modernisation | London | Contract | Inside IR35 We're supporting a global, technology‐led organisation undertaking a large‐scale cloud data transformation programme. This is not a traditional data engineering role. The organisation is modernising hundreds of complex data pipelines, establishing reusable engineering patterns … culture. Responsibilities Defining technical standards and engineering patterns for a large‐scale data modernisation programme Re‐engineering complex legacy data pipelines into production‐grade PySpark Building scalable, reusable data movement frameworks across multiple engineering teams Driving best practice around AWS Glue, Apache Spark and Apache Airflow Designing high‐quality ...

Staff Data Engineer | AWS | PySpark | Data Modernisation | London | Contract | Inside IR35 We're supporting a global, technology-led organisation undertaking a large-scale cloud data transformation programme. This isn't a traditional Data Engineering role. The organisation is modernising hundreds of complex data pipelines, establishing reusable engineering patterns … responsible for Defining technical standards and engineering patterns for a large-scale Data Modernisation programme Re-engineering complex legacy data pipelines into production-grade PySpark Building scalable, reusable data movement frameworks across multiple engineering teams Driving best practice around AWS Glue, Apache Spark and Apache Airflow Designing high-quality ...

engineering standards, and help drive best practices across the wider data community. You'll build scalable data pipelines and frameworks using Azure, Databricks and Pyspark, while mentoring other engineers through code reviews, technical guidance and pair programming. Working closely with Product and Delivery teams, you'll play … delivering solutions that create real business value. Skills & Experience Strong experience building cloud-native data platforms, ideally on Azure Commercial experience with Databricks, Pyspark and Python Advanced SQL and data engineering expertise Experience designing and delivering scalable ETL/data pipeline solutions A passion for technical leadership, mentoring engineers ...