51 to 75 of 84 PySpark Jobs in the UK excluding London

commercial or Agile environment. To be successful in this role its essential that you will: Have experience of AWS , Python, SQL, Gitand PySpark Desirable experience needed will be: SISS or SAS experience Quality Assurance and Test Automation experience Experience of Database technologies Experience in Financial Services organisation About ...

Databricks-based data warehousing solution Designing and developing data models, ETL pipelines, and data integration processes Large scale data processing using PySpark Monitoring, tuning, and optimising data platforms for reliability and performance Upskilling the wider team in Databricks best practices, including modern architecture patterns Location: Atherstone (Hybrid – 3 days ...

with ambiguity and will want to dive deep and fix problems at scale. The environment looks a bit like this Python (pandas, numpy, scipy, PySpark) SQL AWS or GCP (BigQuery/RedShift/Snowflake) Spark/Databricks Airflow/Dagster ...

quality and structure of data Translate business needs into clear technical plans Experience required: Strong skills in data modelling SQL with either Python or PySpark Databricks or similar cloud data platform Experience communicating with both technical and non-technical users Leadership or mentoring experience Insurance or regulated sector experience ...

background in data analysis, data processing, data modeling, with excellent mathematical and problem-solving skills. A background in commercial optimisation analytics. Proficiency in Python, PySpark and SQL for data extraction, transformation, and analysis. Expertise in data-driven storage solutions and handling large, complex datasets. Good understanding of classical statistics ...

complex operational challenges. You'll design and implement scalable generative AI workflows, often using technologies like Palantir AIP, while building robust data pipelines with PySpark, Python, and SQL. A key responsibility is executing sophisticated data integrations across distributed systems and enterprise environments, including ERPs and CRMs. You'll collaborate ...

data for critical client projects. How to develop and enhance your knowledge of agile ways of working and working in open source stack (PySpark/PySql). Quality engineering professionals utilise Accenture delivery assets to plan and implement quality initiatives to ensure solution quality throughout delivery. As a Data ...

Sales, Customer Care and Logistics. What You'll Do Lakehouse Engineering (Azure + Databricks) Build and maintain scalable ELT pipelines using Lakeflow Declarative Pipelines, PySpark and Spark SQL. Work within a Medallion architecture (Bronze ? Silver ? Gold) to deliver reliable, high-quality datasets. Ingest data from multiple sources including ChargeBee … Databricks: Lakeflow Declarative Pipelines, Lakeflow Workflows, Unity Catalog, Delta Lake Azure: ADLS Gen2, Data Factory, Event Hubs (optional), Key Vault, private endpoints Languages: PySpark, Spark SQL, Python, Git DevOps: Azure DevOps Repos & Pipelines, CI/CD Analytics: Power BI, Fabric What We're Looking For Experience Commercial and proven ...

Services : S3, Redshift, Glue, Athena, Lake Formation, IAM Data Engineering/ETL: Building and optimizing ETL pipelines Data ingestion, transformation & orchestration using AWS Glue (PySpark/Python) Working wi... ...

independently and collaborate within cross-functional teams Some knowledge of programming languages such as Python is preferred Experience with data processing technologies such as PySpark is preferred Familiarity with AWS* services and cloud-based data workflows is preferred ...

work independently and collaborate within cross-functional teams Some knowledge of programming languages such as Python (preferred) Experience with data processing technologies such as PySpark (preferred) Familiarity with AWS services and cloud-based data workflows (preferred ...

What We're Looking For Strong data modelling expertise and experience with Data Lineage Maps. Hands-on skills with SQL/Python, Synapse Notebooks, PySpark, and Delta Lake. Familiarity with MS Fabric and medallion architecture. Excellent communication and stakeholder management skills. Agile mindset and ability to thrive in cross ...

from scratch-this is the one. What You'll Be Doing Lakehouse Engineering (Azure + Databricks) Engineer scalable ELT pipelines using Lakeflow Declarative Pipelines , PySpark , and Spark SQL across a full Medallion Architecture (Bronze Silver Gold) . Implement ingestion patterns for files, APIs, SaaS platforms (e.g. subscription billing … Work With Databricks : Lakeflow Declarative Pipelines, Workflows, Unity Catalog, SQL Warehouses Azure : ADLS Gen2, Data Factory, Key Vault, vNets & Private Endpoints Languages : PySpark, Spark SQL, Python, Git DevOps : Azure DevOps Repos, Pipelines, CI/CD Analytics : Power BI, Fabric What We're Looking For Experience 5-8+ years ...

Design, build and maintain scalable and secure data pipelines on the Azure platform. Develop and deploy data ingestion processes using Azure Data Factory, Databricks (PySpark), and Azure Synapse Analytics. Optimise ETL/ELT processes to improve performance, reliability and efficiency. Integrate multiple data sources including Azure Data Lake (Gen2 … Required Skills & Experience - Senior Data Engineer: Proven commercial experience as a Data Engineer delivering enterprise-scale solutions in Azure Azure Data Factory Azure Databricks (PySpark) SQL & Python Understanding of CI/CD in a data environment, ideally with tools like Azure DevOps. Experience working within consultancy or client-facing ...

Senior Data Engineer Salary: Up to £70,000 I am working with a forward-thinking organisation that is modernising its data platform to support scalable analytics and business intelligence across the Group. With a strong ...

With a focus on intuitive front-end creation, robust backendFrameworks and strong data design Lead, a globally dispersed team Experience with AWS, Databricks, DBT, PySpark, React, JavaScript Terraform. Experience with AI/ML/Gen AI/LLM What you need to do now If you're interested ...

expertise in data processing within a cloud-based data platform. Junior Data Engineer duties: Develop, test, and maintain scalable data pipelines using Python and PySpark within an Azure Databricks development environment Clean, aggregate, and transform complex datasets to meet business requirements. Assist with data quality checks and support data … Junior Data Engineer, you will have: Strong proficiency in Python, particularly for data manipulation and automation scripts; experience in libraries such as pandas, PySpark, numpy, pyodbc or associated would be beneficial Hands-on experience (or academic project experience) using PySpark. An understanding of ETL/ELT processes and/ ...

Services : S3, Redshift, Glue, Athena, Lake Formation, IAM Data Engineering/ETL: Building and optimizing ETL pipelines Data ingestion, transformation & orchestration using AWS Glue (PySpark/Python) Working with structured/semi-structured telecom datasets (CDRs, network logs, subscriber data) Data Lake Technologies: Expertise in Apache Iceberg table format … Candidate Profile 3 10+ years of experience in AWS Data Engineering/Support Engineering. Strong telecom domain understanding. Hands-on with Iceberg, Athena, Glue (PySpark), Python, Redshift, S3, ETL frameworks. Strong troubleshooting mindset and ability to work in 24 7 or rotational support environments (if required). ...

Design end-to-end data architectures using Medallion (Bronze/Silver/Gold) patterns Build metadata-driven ingestion pipelines and transformation frameworks Develop advanced PySpark/Spark SQL notebooks for data cleansing and modelling Create production-ready semantic models and support BI teams Implement governance, security, and CI/… clients to translate business requirements into technical solutions Essential Skills Strong experience with Microsoft Fabric workloads (Lakehouse, Data Factory, Pipelines, Notebooks, Semantic Models) Advanced PySpark/Spark SQL for large-scale transformations Integration with Dynamics 365, Dataverse, and/or Business Central CI/CD implementation using Azure DevOps ...

across modern BI platforms, while supporting decision-making in Financial Advice and Wealth Management. Key Responsibilities Build and deploy data models and reports using PySpark, Microsoft Fabric or Databricks, and Power BI Apply relational data modelling and data warehousing techniques (Kimball methodology or similar) Manage version control and collaboration … business, particularly in Financial Advice and Wealth Management Requirements 3-5 years' experience in relational data modelling and data warehousing Strong experience with PySpark, Power BI, Microsoft Fabric/Databricks Knowledge of Git and CI/CD pipelines Understanding of the UK Financial Advice & Wealth Management markets highly advantageous ...

Location: UK (Hybrid/Remote within UK) Type: Contract/Full-TIme Role Overview: Seeking an experienced developer with strong Java, PySpark, and Python skills to design, build, and optimize data-driven applications and batch/streaming pipelines. Key Responsibilities: • Design, develop, and maintain backend services and applications using … Java and Python. • Build and optimize scalable data processing pipelines using PySpark (batch and/or streaming). • Implement robust ETL/ELT workflows to ingest, transform, and validate large datasets. • Write clean, testable, and efficient code; perform unit/integration testing and support CI/CD practices. • Monitor ...

Practice Lead – Defence Data Transformation Location: London/Hybrid Salary: £90,000 – £110,000 + Bonus & Benefits Palantir Foundry | Python | PySpark | Ontology | Data Engineering*Eligibility for clearance is required Join a £20m Programme Driving the Future of Defence Data Our client has just secured a £20 million programme … Stakeholders —translate technical concepts into operational impact. What We’re Looking For Essential: Palantir Foundry experience, ontology modelling, Agile leadership. Tech Stack: Python, SQL, PySpark, strong data engineering principles. Bonus Skills: Graph technologies (RDF, OWL, SPARQL), cloud platforms (AWS/Azure/GCP), integration tools (Airflow, Kafka, Elasticsearch). ...

Pyspark Engineer- (Data Engineering, AWS GLUE) SC Cleared OR EligibleStevenage (Hybrid) 2-3 days onsiteUp to £80,000High-impact programme - Revolutionary platformI am looking for a Pyspark expert to take the reins on a range of highly ambitious Data Migration projects supporting a range of truly high-impact … organisational and regulatory standards.Relevant exposure; Strong hands-on experience with ETL processes and tools (Talend, Informatica, Matillion, Pentaho, MuleSoft, Boomi) or scripting using Python, PySpark, and SQL. Proficient-level SQL skills for complex query development, performance tuning, indexing, and data transformation across on-premise databases and AWS cloud environments. ...

days per week onsite What you'll be doing: Design, build, and optimise scalable data pipelines and architecture Work hands-on with SQL, PySpark, Databricks, Azure, and Data Lake to deliver high-performance solutions Translate business needs into technical requirements and data-driven solutions Ensure best practice in data … environment What you'll need: Proven experience as a Data Engineer or Senior Data Engineer with some team leadership responsibilities Strong technical expertise with PySpark, Databricks, Azure, and Data Lake Deep understanding of data architecture and ETL processes in cloud environments The ability to balance coding and technical delivery ...

days per week onsite What you’ll be doing: Design, build, and optimise scalable data pipelines and architecture Work hands-on with SQL, PySpark, Databricks, Azure, and Data Lake to deliver high-performance solutions Translate business needs into technical requirements and data-driven solutions Ensure best practice in data … environment What you’ll need: Proven experience as a Data Engineer or Senior Data Engineer with some team leadership responsibilities Strong technical expertise with PySpark, Databricks, Azure, and Data Lake Deep understanding of data architecture and ETL processes in cloud environments The ability to balance coding and technical delivery ...