1 to 25 of 66 DemandTrendPermanent PySpark Jobs in the UK excluding London

data retention and archival strategies in cloud environments. Strong understanding and practical implementation of Medallion Architecture (Bronze, Silver, Gold layers) for structured data processing. Advanced programming skills in Python, PySpark, and SQL, with the ability to build modular, efficient, and scalable data pipelines. Deep expertise in data modeling for both relational databases and data warehouses, including Star and Snowflake More ❯

data retention and archival strategies in cloud environments. Strong understanding and practical implementation of Medallion Architecture (Bronze, Silver, Gold layers) for structured data processing. Advanced programming skills in Python, PySpark, and SQL, with the ability to build modular, efficient, and scalable data pipelines. Deep expertise in data modeling for both relational databases and data warehouses, including Star and Snowflake More ❯

data retention and archival strategies in cloud environments. Strong understanding and practical implementation of Medallion Architecture (Bronze, Silver, Gold layers) for structured data processing. Advanced programming skills in Python, PySpark, and SQL, with the ability to build modular, efficient, and scalable data pipelines. Deep expertise in data modeling for both relational databases and data warehouses, including Star and Snowflake More ❯

on data strategy, risk mitigation, and platform evolution. Mentor and develop data engineering talent, fostering a high-performance culture. Requirements: Extensive hands-on experience with AWS data tools: Glue, PySpark, Athena, Iceberg, Lake Formation. Strong proficiency in Python and SQL. Proven leadership in data engineering strategy and execution. Deep understanding of market data and its business applications. Experience with More ❯

record in data integration, ETL processes, and optimising large-scale data systems Expertise in cloud-based data platforms (AWS, Azure, Google Cloud) and distributed storage solutions Proficiency in Python, PySpark, SQL, NoSQL, and data processing frameworks (Spark, Databricks) Expertise in ETL/ELT design and orchestration in Azure, as well as pipeline performance tuning & optimisation Competent in integrating relational More ❯

an impact, get in touch ASAP as interviews are already taking place. Don't miss out! Key Skills: AWS, Data, Architecture, Data Engineering, Data Warehousing, Data Lakes, Databricks, Glue, Pyspark, Athena, Python, SQL, Machine Learning, London More ❯

and DevOps and collaborate with the Software Delivery Manager and data engineering leadership. What you'll need Hands-on Databricks experience Strong Azure Cloud knowledge Proficient in SQL, Python, PySpark ETL & pipeline design (Matillion preferred, alternatives acceptable) Practical data modelling & pipeline architecture Terraform or Bicep for IaC About the company The company is one of the longest-established financial More ❯

and DevOps and collaborate with the Software Delivery Manager and data engineering leadership. What you'll need Hands-on Databricks experience Strong Azure Cloud knowledge Proficient in SQL, Python, PySpark ETL & pipeline design (Matillion preferred, alternatives acceptable) Practical data modelling & pipeline architecture Terraform or Bicep for IaC About the company The company is one of the longest-established financial More ❯

including Azure DevOps or GitHub Considerable experience designing and building operationally efficient pipelines, utilising core Cloud components, such as Azure Data Factory, Big Query, AirFlow, Google Cloud Composer and Pyspark etc Proven experience in modelling data through a medallion-based architecture, with curated dimensional models in the gold layer built for analytical use Strong understanding and or use of More ❯

regular company events What you'll need: Solid experience in data engineering, management and analysis Strong experience with Azure Data Warehouse solutions and AWS Databricks platforms Exceptional Python/PySpark + additional languages for data processing Strong SQL with experience across both relational databases (SQL Server, MySQL) and NoSQL solutions (MongoDB, Cassandra) Hands-on knowledge of AWS S3 and More ❯

data tooling such as Synapse Analytics, Microsoft Fabric, Azure Data Lake Storage/One Lake, and Azure Data Factory. Understanding of data extraction from vendor REST APIs. Spark/Pyspark or Python skills a bonus or a willingness to develop these skills. Experience with monitoring and failure recovery in data pipelines. Excellent problem-solving skills and attention to detail. More ❯

It is entering a phase of hardening and re-engineering to enhance the speed, ease, and quality of ingestion across the portfolio. Key Requirements: Strong proficiency in Python and PySpark Successful track history in a Data Engineering role including Database Management (ideally SQL), Data Orchestration (ideally Apache Airflow or Dagster), Containerisation (ideally Kubernetes and Docker) .Data Pipelines (big data More ❯

Microsoft Azure data services (Azure Data Factory, Azure Data Fabric, Azure Synapse Analytics, Azure SQL Database). Experience building ELT/ETL pipelines and managing data workflows. Proficiency in PySpark, Python, SQL, or Scala. Strong data modelling and relational database knowledge. Solid understanding of GDPR and UK data protection. Preferred Power BI experience. Familiarity with legal industry platforms. Awareness More ❯

Microsoft Azure data services (Azure Data Factory, Azure Data Fabric, Azure Synapse Analytics, Azure SQL Database). Experience building ELT/ETL pipelines and managing data workflows. Proficiency in PySpark, Python, SQL, or Scala. Strong data modelling and relational database knowledge. Solid understanding of GDPR and UK data protection. Preferred Power BI experience. Familiarity with legal industry platforms. Awareness More ❯

financial datasets Python experience, particularly for data processing and ETL workflows Hands-on experience with cloud platforms- Azure Experience designing and maintaining data pipelines using tools like Databricks and PySpark Knowledge of data warehousing solutions - Snowflake experience would be brilliant Understanding of CI/CD processes for deploying data solutions Some exposure to big data technologies and distributed processing More ❯

Engineer/Data Engineering role Large and complex datasets Azure, Azure Databricks Microsoft SQL Server Lakehouse, Delta Lake Data Warehousing ETL CDC Stream Processing Database Design ML Python/PySpark Azure Blob Storage Parquet Azure Data Factory Desirable: Any exposure working in a software house, consultancy, retail or retail automotive sector would be beneficial but not essential. More ❯

Essential Skills Include: Proven leadership and mentoring experience in senior data engineering roles Expertise in Azure Data Factory, Azure Databricks, and lakehouse architecture Strong programming skills (Python, T-SQL, PySpark) and test-driven development Deep understanding of data security, compliance, and tools like Microsoft Purview Excellent communication and stakeholder management skills Experience with containerisation and orchestration (e.g., Kubernetes, Azure More ❯

Troubleshoot issues and continuously improve data infrastructure Explore AI-driven enhancements to boost data accuracy and productivity Requirements: Strong experience with: Azure Databricks, Data Factory, Blob Storage Python/PySpark SQL Server, Parquet, Delta Lake Deep understanding of: ETL/ELT, CDC, stream processing Lakehouse architecture and data warehousing Scalable pipeline design and database optimisation A proactive mindset, strong More ❯

engineering and Azure cloud data technologies. You must be confident working across: Azure Data Services, including: Azure Data Factory Azure Synapse Analytics Azure Databricks Microsoft Fabric (desirable) Python and PySpark for data engineering, transformation, and automation ETL/ELT pipelines across diverse structured and unstructured data sources Data lakehouse and data warehouse architecture design Power BI for enterprise-grade More ❯

end-to-end ownership of demand delivery Provide technical guidance for team members Providing 2nd or 3rd level technical support About You Experience using SQL, SQL Server DB, Python & PySpark Experience using Azure Data Factory Experience using Data Bricks and Cloudsmith Data Warehousing Experience Project Management Experience The ability to interact with the operational business and other departments, translating More ❯

and validate data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What’s in it for you? Base salary of More ❯

and validate data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What’s in it for you? Base salary of More ❯

and validate data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What’s in it for you? Base salary of More ❯

Large and complex datasets Azure, Azure Databricks ML/Machine Learning/AI/Artificial Intelligence Microsoft SQL Server Lakehouse, Delta Lake Data Warehousing ETL Database Design Python/PySpark Azure Blob Storage Azure Data Factory More ❯

we would like to discuss with you. Please note this role requires onsite attendance once a week and has been deemed inside IR35. Requirement: Experience in Azure Synapse, ETL, Pyspark, SQL , data Modelling and Data Bricks. Design and develop Azure Pipelines including data transformation and data cleansing Document source-to-target mappings Re-engineer manual data flows to enable More ❯