1 to 25 of 77 Remote/Hybrid PySpark Jobs in the UK

Engineer Hands-on experience with Microsoft Fabric (essential) Experience working within Azure data environments Strong SQL development skills Experience with Python and/or PySpark Knowledge of Azure Synapse Analytics and Azure Data Factory Experience building and maintaining data pipelines and data integration solutions Understanding of modern data warehousing ...

Engineer Hands-on experience with Microsoft Fabric (essential) Experience working within Azure data environments Strong SQL development skills Experience with Python and/or PySpark Knowledge of Azure Synapse Analytics and Azure Data Factory Experience building and maintaining data pipelines and data integration solutions Understanding of modern data warehousing ...

Practical Microsoft Fabric experience, including Lakehouse, Data Factory, Warehouse and Spark In addition to the above, strong hands-on experience with Azure Databricks, including PySpark, Delta Live Tables and Databricks Workflows and orchestration Knowledge of Azure Data Platform services, including ADF, ADLS Gen2, Azure Synapse Analytics A strong background ...

data warehouses or similar analytical platforms. Experience developing reliable ETL or ELT processes in the Medallion architecture. Experience with Microsoft Fabric data engineering, including Pyspark and dbt. Practical knowledge of data quality, testing, monitoring and operational support. Strong problem-solving skills and attention to detail. Clear communication ...

enterprise-scale environments. Deep expertise in Azure Databricks , Azure Data Factory , Azure Synapse , and Azure Data Lake . Hands-on experience with Spark (PySpark/SparkSQL) , ETL/ELT design, and data pipeline orchestration . Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. ...

ideally in enterprise-scale environments. Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse and Azure Data Lake. Hands-on experience with Spark (PySpark/SparkSQL), ETL/ELT design and data pipeline orchestration. Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. ...

scale data platforms,lakehousesand data warehouses. Data modelling and dimensional modelling techniques such as Kimball. ETL/ELT development and large-scale data integration. PySpark, Spark SQL and modern transformation techniques. Advanced SQL and relational databases including SQL Server, PostgreSQL and Oracle. Integrating structured, semi-structured and unstructured data ...

vs. centralized models, capacity (F-SKU) sizing and licensingDesign end-to-end Microsoft Fabric solutions: OneLake, Lakehouse, Warehouse, Data Factory pipelines, Dataflows Gen2, Notebooks (PySpark/Spark SQL), Real-Time Intelligence (Eventstreams, Eventhouse/KQL) and Power BI + Power BI CoPilotDesign governance, security and operating frameworks: data governance ...

such as Claude Code or equivalent tools, to improve engineering productivity while maintaining high standards of quality, governance and security. Desirable experience Regulated environment. PySpark for large-scale data transformations. Migrating legacy data estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven systems. Data-quality ...

Microsoft Azure environments Hands-on, current engineering skills: you still write the code, not just the diagrams Deep expertise with Azure Databricks, Spark (PySpark/Scala), Delta Lake and Lakehouse architecture Strong Snowflake architecture and optimisation experience Experience with Azure Data Factory, ADLS Gen2, Synapse Analytics and/ ...

Experience of core MS Fabric features including: OneLake, Lakehouses, Warehouses, Data Pipelines, Dataflows, Gen2, Notebooks, Eventstreams + Eventhouses. Using SQL, T-SQL, Spark SQL, PySpark + Python for data ingestion, data transformation, data optimisation, data modelling, and converting structured/unstructured source data into validatd Delta tables. Using Power ...

reusable data foundations. Essential Experience developing data engineering solutions on cloud platforms Strong SQL and Python skills. Experience with Databricks, Delta Lake and Spark (PySpark or Scala). Knowledge of modern data warehousing or lakehouse architectures. Hands-on experience developing production-grade data pipelines. Understanding of data modelling ...

accelerate development across code, infrastructure, testing and documentation while maintaining appropriate quality and security controls. Desirable experience Financial services or another regulated environment. PySpark for large-scale batch or streaming transformations. Migrating SQL Server and SSIS estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven ...

challenge Technical lead or lead engineer experience on enterprise scale Databricks platforms on Azure Strong hands on data engineering background with Python or PySpark and advanced SQL Delta Lake, Unity Catalog, medallion architecture, Databricks Workflows and Auto Loader Declarative pipelines (Delta Live Tables or Lakeflow) and Structured Streaming ...

following technical expertise: Platform Experience: Strong, hands-on experience with Databricks or Snowflake on AWS. Programming & Orchestration: Proven Python skills, including deep expertise in PySpark (or Spark) and Airflow. Data Modelling: Advanced SQL skills and extensive experience with end-to-end data modelling. Data Integration: Proven experience building both ...

architecture Experience with cloud data platforms (AWS preferred - Redshift, Athena, S3, Glue. Azure accepted) Understanding of workflow orchestration and automation tools Python experience (including PySpark) is beneficial but not essential Exposure to BI platforms such as Power BI, Tableau or QuickSight, with an understanding of how backend data models ...

Data Stack, critically ADF and Synapse (experience with Microsoft Fabric is a plus) Highly developed python and data pipeline development knowledge, must include substantial PySpark experience Demonstrable DevOps and DataOps experience with an understanding of best practices for engineering, test and ongoing service delivery An understanding of Infrastructure ...

modern data architectureExperience with cloud data platforms (AWS preferred – Redshift, Athena, S3, Glue. Azure accepted)Understanding of workflow orchestration and automation toolsPython experience (including PySpark) is beneficial but not essentialExposure to BI platforms such as Power BI, Tableau or QuickSight, with an understanding of how backend data models support ...

Lambda, ECS, RDS, S3, SQS, API Gateway, and CloudWatch.Design, develop, and maintain data pipelines and ETL workflows using technologies such as Glue, Spark/PySpark, Kafka, or similar platforms.Implement and maintain Infrastructure as Code (IaC) using Terraform or equivalent tools.Drive observability, monitoring, logging, and operational readiness across software platforms ...

API Gateway, and CloudWatch. Design, develop, and maintain data pipelines and ETL workflows using technologies such as Glue, Spark/PySpark, Kafka, or similar platforms. Implement and maintain Infrastructure as Code (IaC) using Terraform or equivalent tools. Drive observability, monitoring, logging, and operational readiness across software platforms and services. ...

within financial services environments You have expertise with GCP including BigQuery, Pub/Sub, Cloud Composer and IAM You have strong Python, SQL and PySpark skills You have experience with real-time data streaming using Kafka or Spark You have a good knowledge of Data Lakes, Data Warehousing, Data ...

patterns, batch and streaming ingestion, event driven design Conceptual, logical and physical data modelling. Dimensional and data vault both useful Advanced SQL, Python or PySpark Declarative pipelines (Delta Live Tables or equivalent), Databricks Workflows, Auto Loader and Structured Streaming Microsoft Purview for data governance, cataloguing and lineage Azure Data ...

patterns, batch and streaming ingestion, event driven design Conceptual, logical and physical data modelling. Dimensional and data vault both useful Advanced SQL, Python or PySpark Declarative pipelines (Delta Live Tables or equivalent), Databricks Workflows, Auto Loader and Structured Streaming Microsoft Purview for data governance, cataloguing and lineage Azure Data ...

data models and analytics; exposure to predictive modelling and machine learning is a plus. Proficient in SQL and Python, with bonus points for PySpark, SparkSQL, and Git. Skilled in data visualisation with tools such as Tableau or Power BI. Confident writing efficient code and troubleshooting sophisticated queries. Clear ...

communication, and stakeholder management skills. Requirements Candidates should have: 4-6 years of experience in data engineering with a strong focus on Databricks (SQL & PySpark). Strong hands-on experience with Azure data services, including ADF, ADLS, and Azure Functions. Proven experience designing and implementing metadata-driven data pipelines. ...