1 to 25 of 340 PySpark Jobs in the UK

Data Architect (Azure Databricks, Specialty Insurance)

Location
Greater London, England, United Kingdom
with Lloyd’s, bordereaux, reinsurance, or regulatory and actuarial reporting. Hands‐on experience designing solutions on Azure Databricks, including Delta Lake, Unity Catalog, Spark (PySpark and Spark SQL) and Databricks SQL. Experience with core Azure data services and their integration with Databricks, such as ADLS Gen2, Azure Data Factory ...

Data and AI – Data Architect or Engineer

Location
Greater London, England, United Kingdom
with one or more cloud data ecosystems: Azure (Data Factory, Synapse, Fabric, ADLS, Databricks) AWS (Glue, Redshift, EMR) GCP (BigQuery, Dataflow) Databricks (Spark/PySpark) Snowflake (SQL, Snowpark, performance tuning) Strong SQL and/or Python capability, and familiarity with tools such as dbt, Git, CI/ ...

Senior Fabric Engineer

Location
United Kingdom
with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark or Spark SQL to manage and process large datasets. Expertise in Building ETL and ELT Pipelines: You are skilled in building robust ...

Data Engineer

Location
Greater London, England, United Kingdom
models in data warehouses, lakehouse environments or analytics tools such as Power BI. Familiarity with modern data engineering frameworks and tools (e.g. Spark/PySpark, Pandas, orchestration tools). Experience working with cloud data platforms on AWS or Azure (e.g. Databricks, Redshift, Microsoft Fabric). Understanding of software engineering ...

Senior Data Engineer

Location
Greater London, England, United Kingdom
Lake Storage, Azure Data Factory, , MS Fabric ETL/ELT processes and designing, building and testing data pipelines Building data transformations using python/PySpark Azure Cloud Version control and CI/CD tools, specifically Azure DevOps Service Analytics and MI products including MS Power BI Data catalogue & governance ...

Senior Data Engineer

Location
City of Edinburgh, Scotland, United Kingdom
Lake Storage, Azure Data Factory, , MS Fabric ETL/ELT processes and designing, building and testing data pipelines Building data transformations using python/PySpark Azure Cloud Version control and CI/CD tools, specifically Azure DevOps Service Analytics and MI products including MS Power BI Data catalogue & governance ...

Analytics Engineer

Hiring Organisation
Wesleyan
Location
Birmingham, West Midlands, United Kingdom
Employment Type
Temporary Contract, Hybrid
Data Protection Act 2018). Preferred Experience in financial services, insurance or a similarly regulated sector. Experience with Python (or other scripting languages, e.g. PySpark) for data engineering and automation tasks. Familiarity with DevOps/CI-CD practices and version control (e.g. GitHub, Azure DevOps) for data ...

Senior Data Engineer

Hiring Organisation
Sellick Partnership
Location
Greater Manchester, United Kingdom
Employment Type
Permanent
Salary
£47000 - £57000/annum
practical data engineering solutions * Excellent analytical and problem-solving skills Desirable experience/qualifications will include * Azure Data Factory * Microsoft Purview * Python, Spark or PySpark * DevOps, CI/CD pipelines and Infrastructure as Code * DataOps practices * AI, machine learning or advanced analytics platforms * Dynamics 365, CRM or ERP integrations ...

Senior Data & AI Platform Engineer

Location
Manchester, England, United Kingdom
deep Azure Synapse/Databricks experience with clear ability to specialise quickly in Fabric. Expert SQL/T-SQL plus strong Python or PySpark, with a track record of building maintainable ELT/ETL pipelines and analytical data models. Strong Power BI and DAX experience, including semantic modelling, incremental ...

Senior Data Engineer

Location
Chilworth Old Village, England, United Kingdom
need is strong, proven experience delivering solutions on Microsoft Fabric (Data Factory, Lakehouse, Warehouse and Notebooks) or Databricks, strong proficiency in Python, PySpark and SQL, a solid understanding of data modelling, warehousing and ETL/ELT principles, and experience of delivering a platform or data migration. It would ...

Senior Data Engineer

Location
Southampton, England, United Kingdom
need is strong, proven experience delivering solutions on Microsoft Fabric (Data Factory, Lakehouse, Warehouse and Notebooks) or Databricks, strong proficiency in Python, PySpark and SQL, a solid understanding of data modelling, warehousing and ETL/ELT principles, and experience of delivering a platform or data migration. It would ...

Senior Data Engineer – Azure Databricks

Location
Greater London, England, United Kingdom
Good understanding of modern software platform architecture and the role of the data platform within the broader platform ecosystem Good Scala skills (or strong PySpark plus a genuine willingness to work primarily in Scala) Strong SQL skills for transformations, data modelling, and debugging data issues Experience building and supporting ...

Application Developers | Data engineers

Location
United Kingdom
REST, etc.)2 + years Hands-on experience with Redis-backed state management and event-driven processing.2 + years of experience in Apache Spark (PySpark/Spark SQL)2+ years building and deploying Cloud based solutions using - Azure Databricks with UC, Snowflake, Functions, Service Bus2+ years of experience ...

Data Engineer

Location
Southampton, England, United Kingdom
direct dependence on operational databases or replicas. Fabric Data Factory and pipelines with stronger orchestration, monitoring and automated deployment. Python, notebooks and Spark/PySpark where they are the right tool for modern analytical workloads. Direct Lake and other Fabric-native patterns where they provide a genuine benefit. Better ...

Data Engineer

Location
Nottingham, England, United Kingdom
with Microsoft Fabric technologies including OneLake, Lakehouse, Warehouse, Data Factory, Spark and Power BI. Strong Engineering Skills - You're proficient in SQL, Python and PySpark, with experience developing scalable and maintainable data solutions. Modern Data Platform Experience - You’ve successfully built, enhanced or operated cloud‐based data platforms, ideally ...

Senior Data Engineer

Location
City Of London, England, United Kingdom
deployment automation and infrastructure-as-code practices. What We're Looking For Essential Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred). Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good ...

Databricks Data Engineer

Hiring Organisation
EXL Service
Location
Leeds, Yorkshire, United Kingdom
Employment Type
Permanent Contract, Hybrid
ideally in enterprise-scale environments. Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse and Azure Data Lake. Hands-on experience with Spark (PySpark/SparkSQL), ETL/ELT design and data pipeline orchestration. Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. ...

Data Operations Engineer

Location
Manchester, England, United Kingdom
communication and collaboration skills. Desirable Experience Experience across some of the following would be beneficial: AWS Redshift, Athena, EMR or Kinesis. Apache Spark or PySpark for distributed data processing. AWS Step Functions or other workflow orchestration technologies. Terraform, CloudFormation or other Infrastructure as Code tooling. GitHub Actions, GitLab ...

Data Operations Engineer

Location
Greater London, England, United Kingdom
communication and collaboration skills. Desirable Experience Experience across some of the following would be beneficial: AWS Redshift, Athena, EMR or Kinesis. Apache Spark or PySpark for distributed data processing. AWS Step Functions or other workflow orchestration technologies. Terraform, CloudFormation or other Infrastructure as Code tooling. GitHub Actions, GitLab ...

Data Engineer

Location
Greater London, England, United Kingdom
deployment automation and infrastructure-as-code practices. What We're Looking For Essential Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred). Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good ...

Data Engineer

Location
Greater London, England, United Kingdom
Data Lake, Lakehouse, Data Warehouse, Data Vault ETL/ELT processes and designing, building and testing data pipelines Building data transformations using Python/PySpark Azure Cloud Version control and CI/CD tools, specifically Azure DevOps Service Analytics and MI products including MS Power BI Data catalogue & governance ...

Databricks Data Engineer

Hiring Organisation
GIS Recruiter Ltd
Location
London, United Kingdom
Employment Type
Permanent
Salary
£75000 - £95000/annum Hybrid Working
secure cloud-based data architectures. Essential Technical Skills 5+ years of hands-on experience with Databricks. Expert-level knowledge of Azure Databricks. Advanced PySpark, Spark SQL and Python programming. Strong experience developing complex ETL/ELT pipelines. Deep understanding of Delta Lake, Delta Tables and Delta Live Tables/ ...

Data Engineer

Location
Wigan, England, United Kingdom
looking for experienced Data Engineers with proven expertise in building pipelines in Databricks and a strong grasp of Apache Spark, whether in PySpark, including Structured Streaming. You should have experience with Kafka (MSK) and real-time data ingestion, as well as a deep understanding of Delta Lake, Declarative Pipelines ...

Data Engineer

Hiring Organisation
UK Tote Group
Location
London, England, United Kingdom
looking for experienced Data Engineers with proven expertise in building pipelines in Databricks and a strong grasp of Apache Spark, whether in PySpark, including Structured Streaming.You should have experience with Kafka (MSK) and real-time data ingestion, as well as a deep understanding of Delta Lake, Declarative Pipelines ...

Data Engineer

Location
West of England, England, United Kingdom
Azure Data Lake Storage. Advanced SQL development and performance tuning skills, including experience with Azure SQL Database and SQL Server. Proficiency in Python, PySpark, Scala and Spark for data processing and distributed computing. Understanding of data warehousing concepts, dimensional modelling and ETL/ELT frameworks. Knowledge of data quality ...