14 of 14 PySpark Jobs in Central London

data engineering solutions using Python . Extensive hands-on experience with Databricks , including Workflows, Notebooks and Delta Lake. Strong experience with Apache Spark/PySpark . Proven experience building scalable ETL/ELT pipelines. Strong SQL and data modelling skills. Experience designing modern Lakehouse architectures. Experience working with cloud ...

performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules within data pipelines. Data Transformation & Processing: Implement complex data transformations using Spark (PySpark or Scala) and other relevant technologies. Develop and maintain data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy throughout … years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks. Strong proficiency in Python and Spark (PySpark) or Scala. Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns. Extensive experience with Azure data services, including Azure Data ...

Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure Data Engineering certifications Databricks certifications What's in it for you: Location ...

design and delivery of enterprise-scale Azure Data Platform and Databricks solutions. Strong hands-on expertise with Azure Databricks, Structured Streaming, Delta Live Tables, PySpark, Python and SQL, with a focus on performance optimisation. Strong understanding of cloud data architecture, data governance, data quality and engineering best practices. Excellent ...

outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

solutions from concept to implementation. Qualifications Extensive experience with Azure Data Platform and Databricks. Hands-on expertise with Structured Streaming, Delta Live Tables (DLT), PySpark, Python, and SQL. Proven track record delivering large-scale enterprise data platforms. Strong leadership and mentoring capabilities, with the ability to guide engineering teams ...

considering: We are looking for an experienced AWS Databricks Engineer with strong hands-on expertise in Databricks, AWS cloud services, PySpark, Spark SQL, Delta Lake, Python, SQL, and data engineering. The candidate will be responsible for designing, developing, optimizing, and supporting scalable data pipelines and lakehouse solutions … time. Your Role: Design, develop, and maintain scalable data pipelines using Databricks on AWS. Build and optimize data processing frameworks using PySpark, Spark SQL, Python, Delta Lake, and Databricks notebooks. Develop end-to-end ETL/ELT pipelines for ingestion, transformation, validation, and consumption. Work with AWS services such ...

solutions Technical modelling: Experience building models on both static and time series data, including propensity and forecasting models Coding skills: Advanced SQL, Python and PySpark skills, built on solid software development principles Stakeholder management: Ability to work with both technical and non-technical stakeholders, with strong communication skills ...

Design, build and maintain scalable and secure data pipelines on the Azure platform. Develop and deploy data ingestion processes using Azure Data Factory, Databricks (PySpark), and Azure Synapse Analytics . Optimise ETL/ELT processes to improve performance, reliability and efficiency. Integrate multiple data sources including Azure Data Lake … Required Skills & Experience - Azure Data Engineer: Proven commercial experience as a Data Engineer delivering enterprise-scale solutions in Azure Azure Data Factory Azure Databricks (PySpark) SQL & Python Understanding of CI/CD in a data environment , ideally with tools like Azure DevOps. Experience working within consultancy or client-facing ...

team. This role offers a genuine blend of leadership and engineering, with around 30-50% hands‐on development. Key Skills: Python & Databricks Spark/PySpark Technical leadership Data architecture & Lakehouse Agile delivery Stakeholder management #J-18808-Ljbffr ...

Level Required in Years: 10 - 15 years Mandatory Required Skills Hands-on experience with Quantexa platform/Decision Intelligence Strong in Apache Spark/PySpark Proficiency in Python/Scala Experience in ETL, Data Engineering, Data Warehousing Knowledge of entity resolution, graph analytics, link analysis Experience working with large …/HDFS) Detailed Job Description Design and develop solutions using Quantexa Decision Intelligence Platform Build data ingestion, transformation, and ETL pipelines using Spark/PySpark Work on entity resolution, network analytics, and graph-based data models Integrate multiple structured/unstructured data sources for analytics use cases Ensure data ...

New Permanent Opportunity for an eDV Cleared Data Engineer with AWS Cloud Experience for a leading National Security Consultancy in London. Up to £85,000 DoE plus bonuses Active eDV required London Location – full-time ...

Role –Senior Databricks Architect Technology – Databricks (Spark, PySpark/SQL) and data lake architectures Location – London, UK Duration-Permanent Job Description We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling a modern data mesh architecture. The role focuses … integrating legacy systems with cloud-native platforms to support analytics, reporting, and domain-owned data products. Primary Skills • Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. • Hands-on experience with streaming technologies, especially Kafka/Confluent Kafka. • Experience integrating legacy data sources such ...

champion best practice methods; upskill the team through coaching and code reviews. Query and engineer large datasets (e.g., Python/R/SQL/PySpark) on modern platforms (e.g., Azure Databricks). Communicate pricing proposals and insights clearly and persuasively to stakeholders at all levels. Collaborate with Modelling, Optimisation … insurance pricing. Strong technical expertise in predictive modelling techniques (e.g. GLMs and machine learning models). Proficiency in programming languages such as R, Python, PySpark, SAS, or SQL. Good knowledge of Azure Databricks and modern data technologies. Experience with WTW Radar software and optimisation tools. A quantitative degree ...