601 to 625 of 1,236 Data Pipeline Jobs in the UK

Location: London, Hybrid 💰 Rate: Up to £600 per day 📆 Contract: 6 months (Outside IR35, potential to go perm) 🛠 Tech Stack: Azure Data Factory, Synapse, Databricks, Delta Lake, PySpark, Python, SQL, Event Hub, Azure ML, MLflow We’ve partnered with a new AI-first professional services consultancy that’s … a founding team of contractors with the view to scale into permanent leadership as the business grows. They’re now hiring a Senior Data Engineer to build robust, scalable data solutions powering AI, analytics, and future client delivery. If you love creating high-performance data … clean, governed data layers for ML, analytics, and application use 🔹 Supporting LLM/AI data workflows through structured and unstructured pipeline design 🔹 Contributing to reusable data engineering components across client and internal use cases 🔹 Collaborating with data scientists, architects, and product More ❯

Location: London, Hybrid 💰 Rate: Up to £600 per day 📆 Contract: 6 months (Outside IR35, potential to go perm) 🛠 Tech Stack: Azure Data Factory, Synapse, Databricks, Delta Lake, PySpark, Python, SQL, Event Hub, Azure ML, MLflow We’ve partnered with a new AI-first professional services consultancy that’s … a founding team of contractors with the view to scale into permanent leadership as the business grows. They’re now hiring a Senior Data Engineer to build robust, scalable data solutions powering AI, analytics, and future client delivery. If you love creating high-performance data … clean, governed data layers for ML, analytics, and application use 🔹 Supporting LLM/AI data workflows through structured and unstructured pipeline design 🔹 Contributing to reusable data engineering components across client and internal use cases 🔹 Collaborating with data scientists, architects, and product More ❯

Senior Data Engineer – Financial Services Sector Location: London (4 days per week in-office) Salary: up to £80,000 We are working with a financial services company currently undergoing a significant data transformation. This is an exciting opportunity for a Senior Data Engineer to … join the team and help build a modern, cloud-first data environment. About the Role: In this hands-on engineering role, you will work with a range of tools and technologies, including SQL, Azure, and cloud-based solutions, to develop and optimise scalable data pipelines. The … role will also involve establishing data governance and ensuring the seamless flow of data throughout the organisation. You’ll have the opportunity to work with the latest technologies in the industry, including Azure, Microsoft Fabric, and modern data architecture practices, while collaborating with cross More ❯

Senior Data Engineer – Financial Services Sector Location: London (4 days per week in-office) Salary: up to £80,000 We are working with a financial services company currently undergoing a significant data transformation. This is an exciting opportunity for a Senior Data Engineer to … join the team and help build a modern, cloud-first data environment. About the Role: In this hands-on engineering role, you will work with a range of tools and technologies, including SQL, Azure, and cloud-based solutions, to develop and optimise scalable data pipelines. The … role will also involve establishing data governance and ensuring the seamless flow of data throughout the organisation. You’ll have the opportunity to work with the latest technologies in the industry, including Azure, Microsoft Fabric, and modern data architecture practices, while collaborating with cross More ❯

A fantastic opportunity has arisen for a Data Analyst to join a business in Pocklington. In this key role, you will be responsible for capturing, analysing, and reporting data to optimise supply chain efficiency, biological asset management, and commercial decision-making. This position offers the chance … to provide valuable insights that drive business performance and strategic growth. The Role Reporting to the Head of Optimisation, the Data Analyst will generate data-driven insights to enhance performance across the business. This is an excellent opportunity for a data professional to contribute … and exciting opportunities for career development within a dynamic, forward-thinking environment. You will be involved in cutting-edge data projects that are shaping the future of food production. Key Responsibilities: Ensure accurate and timely data capture across the entire supply chain. Develop and maintain automated More ❯

Senior Data Engineer | Music Industry | Up to £85K | AWS, Python, SQL Are you passionate about music and data ? We're looking for a Senior Data Engineer to build and scale a cutting-edge data platform in the music industry. What You’ll … Do: 🎵 Design & develop robust data pipelines (ETL/ELT) ensuring accuracy & reliability. ☁️ Own & evolve our AWS-powered data platform , integrating multiple sources (databases, APIs, external datasets). 🚀 Optimise performance —tune queries, enhance scalability & streamline infrastructure. 🔍 Work with data modelling & integration , collaborating closely with analysts … stakeholders. 🛠 Tech stack: AWS, Python, SQL, Data Lakes/Warehouses, APIs. What We’re Looking For: ✅ Strong expertise in Python & SQL for data processing. ✅ Experience with AWS services (Lambda, S3, Glue, Redshift, etc.). ✅ Background in building & optimising scalable data platforms . ✅ Knowledge of More ❯

performance, implementing improvements based on analysis. Document and standardize ML processes, pipelines, and model specifications. Collaborate with cross-functional teams spanning research, engineering, data science, product managers and other stakeholders to understand business needs and identify opportunities for ML applications. Work closely with engineering teams to integrate ML … frameworks (e.g., TensorFlow or PyTorch). Solid understanding of various machine learning algorithms (e.g., classification, regression, clustering) and their practical applications. Proficient in data manipulation and analysis using tools like SQL and Pandas. Broad ML skillset and are happy to work … on all aspects of ML problems. Not only modeling, but also feature work in data pipelines, some implementation in data pipeline workflows, experimentation setup and analysis. Experience with model evaluation metrics and techniques for ensuring model quality and generalization. Experience with cloud platforms (e.g., GCP More ❯

performance, implementing improvements based on analysis. Document and standardize ML processes, pipelines, and model specifications. Collaborate with cross-functional teams spanning research, engineering, data science, product managers and other stakeholders to understand business needs and identify opportunities for ML applications. Work closely with engineering teams to integrate ML … frameworks (e.g., TensorFlow or PyTorch). Solid understanding of various machine learning algorithms (e.g., classification, regression, clustering) and their practical applications. Proficient in data manipulation and analysis using tools like SQL and Pandas. Broad ML skillset and are happy to work … on all aspects of ML problems. Not only modeling, but also feature work in data pipelines, some implementation in data pipeline workflows, experimentation setup and analysis. Experience with model evaluation metrics and techniques for ensuring model quality and generalization. Experience with cloud platforms (e.g., GCP More ❯

interact and collaborate in person. In Technology, we're asking that you attend the office a minimum of 1 day per week. Our Data Environment Our Data teams are aligned to divisions covering the following Banking Services & Products, Customer Identity & Financial Crime and Data & ML Engineering. Our Data teams are excited about delivering meaningful and impactful insights to both the business and more importantly our customers. Hear from the team in our latest blogs or our case studies with Women in Tech . We are looking for talented data … problems and challenges, who can work across teams do great things here at Starling, to continue changing banking for good. Responsibilities: Analyse source data from source databases (PostgreSQL) to understand structures, relationships and semantics. Design and implement streaming data pipelines using AWS EMR and PySpark to More ❯

edge ideas, best-in-class products and solutions, and unparalleled access to capital and liquidity. What We do/The Team The FX Data Analytics & AI Technology team, within Citi's FX Technology organization, seeks a highly motivated Senior Java Data Engineer. The FX Data Analytics & Gen AI Technology team provides data, analytics, and tools to Citi FX sales and trading globally and is responsible for defining and executing the overall data strategy for FX. The successful candidate will be responsible for developing and implementing data-driven … models, and engineering robust data and analytics pipelines, to unlock actionable insights from our vast amount of global FX data. The role will be instrumental in executing the overall data strategy for FX and will benefit from close interaction with a wide range of stakeholders across More ❯

Prestigious opportunity with a Market Leading Global Retail organisation for a Senior Data Engineer to join our success story in Blackburn. Following a period of significant growth, we are expanding our team to further drive the business forward. You will be responsible for:- Designing, building and maintaining scalable … data pipelines using Python, MS SQL server and robust data pipelines to transform and store data. Managing the process of developing and implementing data models, schemas and architecture solutions. Working with stakeholders, data scientists and analysts to understand data requirements. … Providing insights into data trends and opportunities. Work on processes to improve efficiency, reduce latency and ensure high data availability. Monitoring, logging and alerting mechanisms to identify and resolve data-related issues. If you possess a combination of some of the following skills, then More ❯

Prestigious opportunity with a Market Leading Global Retail organisation for a Senior Data Engineer to join our success story in Blackburn. Following a period of significant growth, we are expanding our team to further drive the business forward. You will be responsible for:- Designing, building and maintaining scalable … data pipelines using Python, MS SQL server and robust data pipelines to transform and store data. Managing the process of developing and implementing data models, schemas and architecture solutions. Working with stakeholders, data scientists and analysts to understand data requirements. … Providing insights into data trends and opportunities. Work on processes to improve efficiency, reduce latency and ensure high data availability. Monitoring, logging and alerting mechanisms to identify and resolve data-related issues. If you possess a combination of some of the following skills, then More ❯

Prestigious opportunity with a Market Leading Global Retail organisation for a Senior Data Engineer to join our success story in Blackburn. Following a period of significant growth, we are expanding our team to further drive the business forward. You will be responsible for:- Designing, building and maintaining scalable … data pipelines using Python, MS SQL server and robust data pipelines to transform and store data. Managing the process of developing and implementing data models, schemas and architecture solutions. Working with stakeholders, data scientists and analysts to understand data requirements. … Providing insights into data trends and opportunities. Work on processes to improve efficiency, reduce latency and ensure high data availability. Monitoring, logging and alerting mechanisms to identify and resolve data-related issues. If you possess a combination of some of the following skills, then More ❯

Select how often (in days) to receive an alert: VIE Data Scientist Databricks About Crown Crown Holdings, Inc. , is a world leader in the metal packaging production process. We design and manufacture a wide range of innovative and sustainable metal packaging solutions and products. Our clients are some … to our long-term future. About this opportunity Your work environment : Our European Information Systems and Services (EISS) department is looking for a Data Scientist Databricks Engineer on a VIE contract for 12-24 months based at Wantage (UK). With a good background in data engineering, you will support our IT Teams in the development and maintenance of data dashboards using Microsoft technologies (including Power BI, Azure Data Factory) and Databricks. The ideal candidate will be capable to learn or have a first experience in designing scalable data More ❯

A new role for a Senior Data Engineer has arisen with an international Tech company working on a major UK Government Programme designing and delivering a range of data solutions. This is an opportunity to play a key role in a collaborative function architecting data pipelines and infrastructure. The Role The Senior Data Engineer will design and implement complex data flows to connect operational systems, build and maintain data environments, evaluate and develop tools to improve data analytics platforms and recognise opportunities to reuse existing … data flows between teams. Requirements This role requires experience in data engineering with a proven track record in configuring and using ETL platforms as well as setting up and administering AWS or Azure Data Platforms. Given the collaborative nature of the environment, excellent communication More ❯

quality. With a strong focus on operational efficiency and digital transformation, we're looking for a Business Intelligence (BI) Analyst to help drive data-informed decisions across our shop floor and wider business operations. Role Overview As a BI Analyst, you will be responsible for developing, maintaining, and … optimizing data pipelines and reporting systems that deliver actionable insights to stakeholders across the business. A key part of your role will involve understanding the manufacturing process and shop floor activities in real-time, which requires full-time on-site presence and collaboration with production teams. Key Responsibilities … Design, build, and maintain data pipelines that integrate production, sales, and operational data. Develop and manage Power BI dashboards and reports tailored to shop floor and business needs. Translate raw data from systems like Navision/Business Central (Nav to BC) into meaningful insights. Support the More ❯

quality. With a strong focus on operational efficiency and digital transformation, we're looking for a Business Intelligence (BI) Analyst to help drive data-informed decisions across our shop floor and wider business operations. Role Overview As a BI Analyst, you will be responsible for developing, maintaining, and … optimizing data pipelines and reporting systems that deliver actionable insights to stakeholders across the business. A key part of your role will involve understanding the manufacturing process and shop floor activities in real-time, which requires full-time on-site presence and collaboration with production teams. Key Responsibilities … Design, build, and maintain data pipelines that integrate production, sales, and operational data. Develop and manage Power BI dashboards and reports tailored to shop floor and business needs. Translate raw data from systems like Navision/Business Central (Nav to BC) into meaningful insights. Support the More ❯

exciting point in our journey, leveraging Generative AI (GenAI) , Large Language Models (LLMs) , and advanced Retrieval-Augmented Generation (RAG) techniques to build intelligent, data-driven systems that deliver powerful PR insights. You'll also work on developing agentic workflows that autonomously orchestrate tasks, enabling scalable and dynamic solutions. … Our data stack is modern and efficient, designed to process large-scale information, automate analysis pipelines, and integrate seamlessly with AI-driven workflows. This is an excellent opportunity to make a significant impact on projects that push the boundaries of AI-powered insights and automation. If you're … managed AI/ML services. Hands-on experience with Docker, Kubernetes, and container orchestration. Expertise with Databricks, including ML workflows and data pipeline management. Familiarity with tools like MLflow, DVC, Prometheus, and Grafana for versioning and monitoring. Experience implementing security and compliance standards for AI systems. Strong More ❯

Senior Data Engineer | Bristol | Permanent Full-Time | Hybrid Working Peaple Talent are pleased to be working with a returning client in Bristol looking to recruit for a Senior Data Engineer. This is a full-time permanent role, and would require hybrid working in Bristol. This Senior … growth by sharing knowledge and improving collective engineering practices. In-depth knowledge of data modelling best practices and common patterns used in pipeline architecture. Confident in version control with Git and familiar with setting up CI/CD workflows using platforms like Azure DevOps or similar tools. … Airflow for managing complex data workflows. Practical familiarity with low-code or no-code platforms such as Talend and SnapLogic for streamlined pipeline development. Strong experience working with cloud-based data services, especially within AWS environments, utilizing tools like Lambda, S3, Redshift, Glue, Athena and More ❯

is central to that mission-reinventing how users find and connect with content in new and relevant ways. We're looking for a Data Engineer to join a squad passionate about delivering pioneering search experiences powered by Machine Learning (ML) and Large Language Models (LLMs). In this … role, you'll help craft the team's data strategy and build the systems that power smarter, more intuitive discovery on Spotify. You'll work alongside a dedicated mix of ML and Backend Engineers to tackle some of Spotify's most exciting challenges in retrieval, ranking, and personalization. … If you're excited by data systems, ML-powered products, and the opportunity to impact how millions of people explore audio content-this is the role for you. What You'll Do Design, build, and maintain scalable, reliable data pipelines that support advanced search capabilities. Partner More ❯

client-side incubation support with ready-made out-of-the box capability supported by smart, execution-focussed practitioners. The Role Are you a data enthusiast with a passion for transforming raw information into meaningful insights? We’re looking for a talented Fabric/Data Consultant with … and Power BI to join our dynamic team. This role is perfect for someone who enjoys working with cutting-edge technology, developing robust data architectures, and driving strategic business intelligence initiatives. If you're excited about helping organizations unlock the full potential of their data, we … d love to hear from you. Essential Expertise: Proven experience in data engineering with exposure to Azure Synapse or Azure Data Factory or Databricks. Familiarity with Microsoft Fabric or a strong willingness to learn and adapt to its ecosystem. Proficiency in Python programming with a focus More ❯

Company: My client are a fast-growing fintech transforming how people manage and move money. As a Data Engineer, you’ll play a pivotal role in building and scaling the data infrastructure that powers everything from real-time fraud detection to personalized financial insights. Key Responsibilities … Design, build, and maintain scalable data pipelines (batch & streaming) Work with Data Scientists and Analysts to enable fast, reliable access to data Optimize our data architecture for performance, reliability, and cost Collaborate with engineers to integrate data best practices across … teams Champion data quality, governance, and documentation Key Requirements: Strong experience with Python, SQL, and modern ETL tools (e.g., Airflow, dbt) Solid grasp of cloud platforms (AWS/GCP/Azure) and data warehouses (e.g., BigQuery, Snowflake) Familiarity with streaming technologies (Kafka, Kinesis, etc.) Passion for More ❯

Company: My client are a fast-growing fintech transforming how people manage and move money. As a Data Engineer, you’ll play a pivotal role in building and scaling the data infrastructure that powers everything from real-time fraud detection to personalized financial insights. Key Responsibilities … Design, build, and maintain scalable data pipelines (batch & streaming) Work with Data Scientists and Analysts to enable fast, reliable access to data Optimize our data architecture for performance, reliability, and cost Collaborate with engineers to integrate data best practices across … teams Champion data quality, governance, and documentation Key Requirements: Strong experience with Python, SQL, and modern ETL tools (e.g., Airflow, dbt) Solid grasp of cloud platforms (AWS/GCP/Azure) and data warehouses (e.g., BigQuery, Snowflake) Familiarity with streaming technologies (Kafka, Kinesis, etc.) Passion for More ❯

Prestigious opportunity with a Market Leading Global Retail organisation for a Data Engineer to join our success story in Blackburn. Following a period of significant growth, we are expanding our team to further drive the business forward. You will be responsible for:- Designing, building and maintaining scalable data pipelines using Python, MS SQL server and robust data pipelines to transform and store data. Managing the process of developing and implementing data models, schemas and architecture solutions. Working with stakeholders, data scientists and analysts to understand data requirements. Providing … insights into data trends and opportunities. Work on processes to improve efficiency, reduce latency and ensure high data availability. Monitoring, logging and alerting mechanisms to identify and resolve data-related issues. If you possess a combination of some of the following skills, then LETS More ❯

power the training of the world's most capable AI frontier models, pushing the boundaries of scale, performance, and product deployment. The AI Data team at Microsoft AI is responsible for all aspects of data preparation to support our model pre-training operations, including collecting data from the source, extracting and transforming the most useful data, and understanding the impact of changes to data by training and evaluating new models. We are an interdisciplinary team of engineers and scientists, learning from each other, and collaborating to create the best models … next generation of systems that will transform the field. In particular, we are looking for candidates who: Are passionate about the role of data in large-scale AI model training Will thrive in a highly collaborative, fast-paced environment Have a high degree of craftsmanship and pay close More ❯