76 to 100 of 136 PySpark Jobs in the UK

with Azure Data Platform technologies, including both Databricks and Data Factory Proven experience delivering complex ETL solutions Solid programming experience using Python, SQL and PySpark Strong stakeholder engagement skills with a diplomatic and collaborative working style Experience working within professional services or regulated environments is desirable If this role ...

machine learning operations. Excellent communication skills with the ability to influence cross-functional teams and stakeholders. Experience with Python, SQL, Java, HQL, Hive, Pyspark, or similar programming languages preferred Well-versed in design thinking principles Great problem-solving and analytical skills ...

platforms on Microsoft Azure. The role involves leading the development of secure, reliable, and efficient data pipelines using Azure Synapse Analytics, Azure Data Factory, PySpark, and Azu click apply for full job details ...

data for critical client projects. How to develop and enhance your knowledge of agile ways of working and working in open source stack (PySpark/PySql). Quality engineering professionals utilise Accenture delivery assets to plan and implement quality initiatives to ensure solution quality throughout delivery. As a Data ...

data for critical client projects. How to develop and enhance your knowledge of agile ways of working and working in open source stack (PySpark/PySql). Quality engineering professionals utilise Accenture delivery assets to plan and implement quality initiatives to ensure solution quality throughout delivery. As a Data ...

/GCP (based on organization need) integrated with Databricks. Data Engineering & Platform Build Build and optimize large-scale ETL/ELT pipelines using PySpark, SQL, and Delta. Oversee data quality frameworks, metadata management, lineage, and monitoring. Leadership & Stakeholder Engagement Work closely with business leaders and product teams to translate … years overall experience in data engineering or data architecture. 5+ years of hands-on architecture experience specifically in Databricks . Expert in Apache Spark , PySpark , Databricks SQL , Delta Lake, and distributed systems. Strong understanding of Lakehouse architecture , BI ecosystems, and modern data platforms. Cloud expertise in Azure/ ...

Data Architect - Databricks - Hybrid A leading Microsoft partnered consultancy is seeking an experienced Data Architect to design and deliver modern data solutions built on Databricks Lakehouse Architecture. You'll work with a variety of enterprise ...

Data Architect - Databricks - Hybrid - Up to £125,000 A leading Microsoft partnered consultancy is seeking an experienced Data Architect to design and deliver modern data solutions built on Databricks Lakehouse Architecture. You'll work with ...

Data Architect - Databricks - Hybrid - Up to £125,000 A leading Microsoft partnered consultancy is seeking an experienced Data Architect to design and deliver modern data solutions built on Databricks Lakehouse Architecture. You'll work with ...

using Octopus Deploy or Azure functions/DevOpS is ideal. Experience working with big datasets. Develop and maintain data processing workloads using Python and PySpark, including notebook-based development where appropriate. Support the migration and modernisation of legacy databases and ETL processes to cloud-based architectures. Strong analytical ...

analysis, model development, and data interrogation. Strong knowledge of PyTorch or similar ML frameworks. Experience handling large datasets and distributed processing with PySpark or similar is desirable . Experience of experimentation tools such as MLflow is a bonus. Comfort with deploying ML models into production environments. Familiarity with ...

analysis, model development, and data interrogation. Strong knowledge of PyTorch or similar ML frameworks. Experience handling large datasets and distributed processing with PySpark or similar is desirable . Experience of experimentation tools such as MLflow is a bonus. Comfort with deploying ML models into production environments. Familiarity with ...

engineering is done across the organisation. Skills & Experience Strong hands-on experience with Azure data platforms Advanced SQL skills Commercial experience using Databricks and PySpark Proven background building and maintaining scalable data pipelines If you're looking for a role where you can combine technical depth, ownership, and influence ...

support reporting, analytics, and business operations. This includes developing ETL/ELT workflows using both low-code tools and code-based approaches (SQL, Python, PySpark), ensuring performance, scalability, and reliability across the data platform. You will take ownership of data modelling and architecture , designing scalable data warehouse and lakehouse … understanding of data modelling (star schema, dimensional modelling, lakehouse design) Experience building and optimising ETL/ELT pipelines using SQL, Python, and/or PySpark Knowledge of data governance, quality, and security best practices Experience implementing CI/CD pipelines and DevOps practices Ability to work independently and manage ...

AWS Data Engineer (Innovation Group) Location: London (Hybrid – 3 days per week) Duration: 6 Months Initial Contract Rate: TBD Are you a Data Engineer who thrives on building more than just standard ETL pipelines? Our ...

Data Quality Reporting • Strong capabilities to manage Data Catalogues and Data Lineage • Experience within Data Warehouses, Data Lake and Data Mesh • Experience with Python, Pyspark and ETL is desirable. This is an exciting opportunity to lead and influence data governance in a well-respected Media organisation during an exciting ...

reporting, and downstream data services across the organisation. Key Responsibilities Design, build, and maintain data pipelines using Databricks Develop ETL/ELT processes with PySpark and Spark SQL Transform and model structured and semi-structured datasets Improve performance, reliability, and cost efficiency of data workloads Ensure compliance with data … Skills & Experience Active SC Clearance - mandatory and non-negotiable Strong hands-on experience with Databricks in production environments Expert knowledge of Apache Spark , including PySpark and Spark SQL Proficiency with Python for data engineering Experience delivering solutions on cloud data platforms ( AWS preferred , Azure acceptable) Understanding of data lake ...

timely, and built to last. What You'll Be Responsible For Developing and deploying scalable data engineering solutions on the Databricks Lakehouse platform using PySpark, Spark SQL, and Python Building batch pipelines that feed our Discovery Analytics platform, powered by Power BI, with accurate and reliable data Contributing … aware mindset — someone who understands how good data engineering creates real value, not just technical output Solid hands-on experience with Databricks, Delta Lake, PySpark, and Spark SQL Strong Python and SQL skills with a focus on clean, maintainable, production-ready code Working knowledge of CI/CD practices ...

datasets optimized for Power BI. Secure: Implement rigorous data security, GDPR compliance, and client ring-fencing (Row-Level Security). Innovate: Create SQL and PySpark notebooks to prepare feature engineering datasets for AI/ML models. Collaborate: Work independently via Jira/GitHub, maintaining high CI/… What You Bring 5+ years of Data Engineering experience. Expertise: Deep knowledge of Snowflake and Microsoft Fabric . Tech Stack: Advanced SQL , Python , and PySpark (Notebooks). Modelling: Strong relational data modelling and cloud cost optimization skills. DevOps: Experience with GitHub, CI/CD, and workflow orchestration. Randstad Technologies ...

maintain scalable ETL/ELT data pipelines within Microsoft Fabric Lakehouse environments Develop and optimise data ingestion/transformation workflows Develop Python and PySpark processes for data transformation and large-scale processing Support the development and optimisation of Power BI datasets, reports and dashboards Build and maintain scalable semantic … data engineering solutions Skills and Experience Hands-on expertise with Microsoft Fabric, Lakehouse architecture and Data Pipelines Experience developing Python or PySpark solutions for large-scale data processing Understanding of ETL/ELT design patterns, data modelling and modern data platform architecture Experience developing Power BI datasets, semantic models ...

PySpark Developer (SAS & AWS) 12+ Month Contract | UK Fully Remote | Inside IR35 We're supporting a large-scale data and AI transformation programme. We are looking for an experienced Data & AI - LLM Model Developer with advanced PySpark and AWS expertise to help modernise complex Legacy data platforms. This … hands-on contract role where you'll play a key part in SAS-to-PySpark migration , building scalable, cloud-native data pipelines and delivering production-ready solutions in a regulated environment. What you'll be doing Designing, developing, and optimising PySpark-based data pipelines on AWS Converting Legacy ...

their data pipelines, using Python, Spark, and Databricks Workflows to deliver new platform functionality and ensure efficient execution. Key Responsibilities Develop reliable Python and PySpark code to support data ingestion, transformation, and end‐to‐end processing. Deliver new technical features and components aligned to approved solution designs and business ...

demonstrable commitment to developing your knowledge and expertise Significant technical data science knowledge Desirable Criteria: Understanding of supply chain data sources Experience with Pyspark/Spark SQL Experience delivering interactive visualisations in Python and SQL Graph data experience, including with graph query languages and knowledge graphs Experience working ...

experience to Design & Develop a ETL pipelines in aws Glue for ingesting data from sql server or other sources Hands on experience on PySpark & Python Experience of implementing change data capture pipelines using aws dms for near-real time or batch ingestion Experience of developing click apply for full ...

high-quality outcomes. Your Responsibilities: Lead the design and implementation of scalable, secure, and cost-efficient ETL/ELT pipelines using AWS Glue, Python (PySpark), and MWAA (Airflow). Define solution architectures, data models, orchestration patterns, and CI/CD for data workflows. Own the technical roadmap, decomposition …/data engineering, with 5+ years leading delivery of production solutions in an AWS data engineering environment. Advanced hands-on expertise with: Python (including PySpark & data engineering patterns) AWS Glue (Jobs, Crawlers, Glue Studio, Glue Catalog, PySpark, Job bookmarks) MWAA (Apache Airflow) (DAG design, scheduling, sensors, retries, XComs ...