1 to 25 of 69 Permanent PySpark Jobs in the UK

with multiple data analytics tools (e.g. Power BI) Deep understanding of data wareho using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts to both More ❯

Storage . Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for More ❯

and prompt engineering. Mandatory Skills: Cloud Platforms:Deep experience with AWS (S3, Lambda, Glue, Redshift) and/or Azure (Data Lake, Synapse). Programming & Scripting:Proficiency in Python, SQL, PySpark etc. ETL/ELT & Streaming:Expertise in technologies like Apache Airflow, Glue, Kafka, Informatica, EventBridge etc. Industrial Data Integration:Familiarity with OT data schema originating from OSIsoft PI, SCADA More ❯

with various businesses and gaining an overview of many different sectors. What We’re Looking For 10 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience in implementing best practices for More ❯

of working Champion DevOps and CI/CD methodologies to ensure agile collaboration and robust data solutions Engineer and orchestrate data models and pipelines Lead development activities using Python, PySpark and other technologies Write high-quality code that contributes to a scalable and maintainable data platform To be successful in this role, you will need to have the following More ❯

data governance, security, and access control within Databricks. • Provide technical mentorship and guidance to junior engineers. Must-Have Skills: • Strong hands-on experience with Databricks and Apache Spark (preferably PySpark). • Proven track record of building and optimizing data pipelines in cloud environments. • Experience with AWS services such as S3, Glue, Lambda, Step Functions, Athena, IAM, and VPC. • Proficiency More ❯

hands-on approach to data science, analytics, and ML solutions. Continuously optimise data workflows for performance, reliability, and scalability. What youll need: Proven hands-on experience with Databricks, Python, PySpark, and SQL. Machine learning experience in a cloud environment (AWS, Azure, or GCP). Strong understanding of ML libraries such as scikit-learn, TensorFlow, or MLflow. Solid background in More ❯

on approach to data science, analytics, and ML solutions. • Continuously optimise data workflows for performance, reliability, and scalability. What you’ll need: • Proven hands-on experience with Databricks, Python, PySpark, and SQL. • Machine learning experience in a cloud environment (AWS, Azure, or GCP). • Strong understanding of ML libraries such as scikit-learn, TensorFlow, or MLflow. • Solid background in More ❯

and influence decisions. Strong understanding of data models and analytics; exposure to predictive modelling and machine learning is a plus. Proficient in SQL and Python, with bonus points for PySpark, SparkSQL, and Git. Skilled in data visualisation with tools such as Tableau or Power BI. Confident writing efficient code and troubleshooting sophisticated queries. Clear and adaptable communicator, able to More ❯

and monitoring. Data modelling expertise to develop low-level designs and implement models against business requirements, using design patterns such as Inmon, Kimball and Data Vault. Excellent Databricks, Python, PySpark and Spark SQL knowledge, including writing, testing and quality assuring code and knowledge of Unity Catalog best practice to govern data assets as well as Azure and/or More ❯

Essential Skills Include: Proven leadership and mentoring experience in senior data engineering roles Expertise in Azure Data Factory, Azure Databricks, and lakehouse architecture Strong programming skills (Python, T-SQL, PySpark) and test-driven development Deep understanding of data security, compliance, and tools like Microsoft Purview Excellent communication and stakeholder management skills Experience with containerisation and orchestration (e.g., Kubernetes, Azure More ❯

warehousing techniques, including the Kimball Methodology or other similar dimensional modelling standards, is essential to the role. Technical experience building and deploying models and reports utilizing the following tools: PySpark Microsoft Fabric or Databricks Power BI Git CI/CD pipelines (Azure DevOps experience preferred) An understanding of the structure and purpose of the Financial Advice and Wealth Management More ❯

Data Management services Knowledge & experience of working in Cloud environment Experience and knowledge Essential Good knowledge of Microsoft Office Advanced knowledge of a programming language such as SQL, Python, PySpark, R or similar and able to demonstrate the ability to learn new programming languages as required Office packages - in particular, Excel Knowledge of Data Management or ETL (Extract, Transform More ❯

Experience in a Data Engineer/Data Engineering role Large and complex datasets Azure, Azure Databricks Microsoft SQL Server Lakehouse, Delta Lake Data Warehousing ETL Database Design Python/PySpark Azure Blob Storage Azure Data Factory Desirable: Exposure ML/Machine Learning/AI/Artificial Intelligence More ❯

data models and transformation pipelines using Databricks, Azure, and Power BI to turn complex datasets into reliable, insight-ready assets. You'll apply strong skills in SQL, Python, and PySpark to build efficient ELT workflows and ensure data quality, performance, and governance. Collaboration will be key as you partner with analysts and business teams to align data models with More ❯

have: Hands-on experience creating data pipelines using Azure services such as Synapse, Data Factory or Databricks Commercial experience with Microsoft Fabric Strong understanding of SQL and Python/PySpark Experience with Power BI and data modelling Some of the package/role details include: Salary up to £85,000 Flexible hybrid working model (normally once/twice per More ❯

have: Hands-on experience creating data pipelines using Azure services such as Synapse, Data Factory or Databricks Commercial experience with Microsoft Fabric Strong understanding of SQL and Python/PySpark Experience with Power BI and data modelling Some of the package/role details include: Salary up to £85,000 Flexible hybrid working model (normally once/twice per More ❯

Collaborate with brilliant behavioural scientists and product teams who'll challenge them in the best ways The ideal candidate will: Know their way around AWS data tools (Glue/PySpark, Athena) and Microsoft Fabric Be able to write clean Python and SQL in their sleep Have battle scars from integrating CRMs (HubSpot, Salesforce) via APIs Actually care about data More ❯

explain commercial impact. Understanding of ML Ops vs DevOps and broader software engineering standards. Cloud experience (any platform). Previous mentoring experience. Nice to have: Snowflake or Databricks Spark, PySpark, Hadoop or similar big data tooling BI exposure (PowerBI, Tableau, etc.) Interview Process Video call - high-level overview and initial discussion In-person technical presentation - based on a provided More ❯

involve: Supporting the BI reporting team by creating and maintaining data solutions for KPI reporting. Developing scalable, performance-optimised ELT/ETL pipelines using T-SQL, Python, ADO, C#, PySpark and Jupyter Notebooks. Working with the Gazetteer and GIS teams to maintain stable, consolidated database platforms for mapping and GIS systems. Contributing to the development and maintenance of a More ❯

based modelling workflows and PR reviews via Tabular Editor. Excellent design intuition-clean layouts, drill paths, and KPI logic. Nice to Have Python for automation or ad-hoc prep; PySpark familiarity. Understanding of Lakehouse patterns, Delta Lake, metadata-driven pipelines. Unity Catalog/Purview experience for lineage and governance. RLS/OLS implementation experience. More ❯

based modelling workflows and PR reviews via Tabular Editor. Excellent design intuition-clean layouts, drill paths, and KPI logic. Nice to Have Python for automation or ad-hoc prep; PySpark familiarity. Understanding of Lakehouse patterns, Delta Lake, metadata-driven pipelines. Unity Catalog/Purview experience for lineage and governance. RLS/OLS implementation experience. More ❯

the perfect match? Proven experience in a data analytics or credit risk role, ideally within utilities, financial services or other regulated industry Strong coding skills in SQL, Python and PySpark for data extraction, transformation, modeling and forecasting Solid understanding of forecasting techniques, scenario modelling, and regression-based analytics Strong commercial acumen, with the ability to translate complex analytical findings More ❯

to transform raw data into trusted, actionable insights that power critical business decisions. Key Responsibilities Design and implement scalable data pipelines and ETL/ELT workflows in Databricks using PySpark, SQL, and Delta Lake. Architect and manage the Medallion (Bronze, Silver, Gold) data architecture for optimal data organization, transformation, and consumption. Develop and maintain data models, schemas, and data … data platforms, Lakehouse architecture, and data engineering frameworks. Required Qualifications 6+ years of experience in data engineering 3+ years of hands-on experience with Databricks, Delta Lake, and Spark (PySpark preferred). Proven track record implementing Medallion Architecture (Bronze, Silver, Gold layers) in production environments. Strong knowledge of data modeling, ETL/ELT design, and data lakehouse concepts. Proficiency More ❯

met. The role involves structuring analytical solutions that address business objectives and problem solving. We are looking for hands-on experience in writing code for AWS Glue in Python, PySpark, and Spark SQL. The successful candidate will translate stated or implied client needs into researchable hypotheses, facilitate client working sessions, and be involved in recurring project status meetings. You More ❯