776 to 800 of 1,236 Data Pipeline Jobs in the UK

PR review and architecture discussions Desirable (but not essential): Event-driven architecture (e.g., SNS, SQS, EventBridge) AI/LLM integration (Python experience welcome) Data pipelines and ETL workflow knowledge Experience scaling cloud services and optimising for performance and cost in serverless environments Location & Flexibility: This is a London More ❯

Lead Data Engineer - London/Hybrid (3days per week onsite) - Up £120K + 10% Bonus TogetherWeTech is hiring a Lead Data Engineer for our exciting iGaming scale-up. As a Lead Data Engineer, you'll be an integral hire in the company's transition … to bringing all its tech in-house. This means, becoming their first-ever Lead Data Engineer, designing and building robust, scalable, and high-performance real-time data. Key Benefits: 10% Bonus Private Health Insurance Personal development training budget Quarterly rewards Annual trips for company conferences Key Requirements & Responsibilities … Design, develop, and optimise real-time data pipelines and infrastructure. Build efficient data models for real-time analytics. Proven experience in managing real-time data pipelines across multiple initiatives. Expertise in distributed streaming platforms (Kafka, Spark Streaming, Flink). Experience with GCP (preferred), AWS More ❯

Lead Data Engineer - London/Hybrid (3days per week onsite) - Up £120K + 10% Bonus TogetherWeTech is hiring a Lead Data Engineer for our exciting iGaming scale-up. As a Lead Data Engineer, you'll be an integral hire in the company's transition … to bringing all its tech in-house. This means, becoming their first-ever Lead Data Engineer, designing and building robust, scalable, and high-performance real-time data. Key Benefits: 10% Bonus Private Health Insurance Personal development training budget Quarterly rewards Annual trips for company conferences Key Requirements & Responsibilities … Design, develop, and optimise real-time data pipelines and infrastructure. Build efficient data models for real-time analytics. Proven experience in managing real-time data pipelines across multiple initiatives. Expertise in distributed streaming platforms (Kafka, Spark Streaming, Flink). Experience with GCP (preferred), AWS More ❯

Geospatial Data Engineer 6 month contract (initially) Remote-Working Outside IR35 Market Rates The Opportunity: ISR are proud to be supporting a high-profile digital transformation programme and we’re looking for an experienced Geospatial Data Engineer to join the team on a contract basis. In … this role, you'll work at the intersection of geospatial data and cloud engineering – designing, building and maintaining robust pipelines that enable the transformation and analysis of large-scale geospatial datasets. Skills and Experience: Strong programming skills in Python, with experience using libraries such as GeoPandas, Shapely and … GDAL Experience with cloud platforms (AWS, Azure or GCP) and infrastructure-as-code tools Proficiency in spatial databases (e.g., PostGIS) and data pipeline tools (e.g., Airflow) Familiarity with geospatial APIs, formats (GeoJSON, WKT) and data services Security Clearance: Candidates must be eligible to pass Baseline More ❯

We are looking for a Data Engineer to help us build structured, high-quality data solutions for our Content Understanding teams. Our teams process petabytes of data using tools such as BigQuery and Dataflow, and where needed we develop our own data … tooling to support and evolve our music catalog. Location London Job type Permanent What You'll Do Take ownership over, maintain and improve data pipelines that are integral to our success Evaluate and optimize our offerings for efficiency and reliability at scale Identify areas of architectural improvement to … remove the potential for human error Collaborate with product and engineering teams to understand data requirements and create innovative solutions to challenging problems Understand the analytics data we collect from our systems and how it relates to the business value being produced Who You Are Proven More ❯

A new role for a Lead Data Engineer has arisen with an international Tech company working on a major UK Government Programme designing and delivering a range of data solutions. This is an opportunity to play a key role in a collaborative function architecting data … pipelines and infrastructure, leading in both a technical sense but also through mentorship and establishing best practices and standards. The Role The Lead Data Engineer will design and implement complex data flows to connect operational systems, build and maintain data environments, evaluate and develop … tools to improve data analytics platforms and recognise opportunities to reuse existing data flows between teams. Requirements This is a leadership role and requires not only demonstrable experience of leading on data engineering projects but requires someone with a proactive, creative approach, able to More ❯

seamless human-AI interaction. Build and deploy open-source models in cloud environments, optimizing inference and serving costs. Improve and maintain data pipeline reliability and participate in on-call rotations. Debug and fix issues in ML pipelines, even when the cause is obscure. Collaborate with cross-functional … Optimize prompts, refine model outputs, and experiment with novel prompting strategies. Implement backend endpoints to bridge AI capabilities into our production stack. Label data and refine model training workflows. Hire and manage part-time annotators to improve data quality. Create quick prototypes using Dash/Streamlit … GCP is a big plus. DevOps experience is a big plus. Prompt engineering expertise and creative problem-solving mindset. Experience with processing multimodal data (text, images, audio) is a plus. Perks and Benefits: Fully Remote: Work from anywhere with flexible hours. In-person Meetups and regular team-building More ❯

of our colleagues. Range of knowledge and experience of data and information storage, processing, governance, scientific solutions and services. Data pipeline, transfer, storage, transformation and archiving solutions, with a focus on scientific data, across a high level of diversity. Hybrid cloud and on … Skills UML/SysML/MBSE. MS Azure architecture frameworks and design patterns. TCP/IP networking, HTTP/REST, APIs and other data transfer mechanisms such as SSH, SFTP, VPN links. Project management (APM/PMI/DSDM Agile/Scrum). CSV/LOD model or More ❯

You will need to login before you can apply for a job. View more categories View less categories Sector Data Science Role Senior Executive Contract Type Permanent Hours Full Time Job Description: Are you passionate about Data and Analytics (D&A) and excited about how it … can transform enterprise operations? Do you possess strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Are you eager to work in a dynamic, fast-growing category? If yes, you might be the ideal candidate for the Data and Analytics role in Global … agenda requires scalable digitization, thought leadership, decision-making, technical expertise, and the ability to navigate complex challenges while leading a team of top data and analytics professionals. With Digital First, PN is adopting a product-based model to develop business-facing digital capabilities. The role involves developing and More ❯

Octopus Energy, a leading force in the energy sector, is seeking a Data Engineer to join their Energy Markets team. This role sits at the intersection of energy trading, forecasting, and technological innovation, supporting the company's mission towards Net Zero transition. The position offers a unique opportunity … operational and reporting pipelines across regions Maintain critical data pipelines supporting trading, forecasting, risk and PPA processes Develop automations and alerts for pipeline monitoring Set up and maintain processes for capturing, preparing and loading data into the data lake Design and build operational … Engineer Strong aptitude with SQL, Python and Airflow Experience in Kubernetes, Docker, Django, Spark and related monitoring tools Experience with dbt for pipeline modelling Ability to shape needs into requirements and design scalable solutions Quick understanding of new domain areas and data visualization Team player More ❯

pipelines. This person will possess a strong background in DevOps practices, machine learning principles, and cloud computing platforms. You will work closely with data scientists and software engineers to streamline the deployment and monitoring of machine learning models, ensuring efficiency and reliability in ML operations. We hire based … with version control systems (e.g., Git). • Familiarity with CI/CD tools and practices. • Strong problem-solving and analytical skills. • Understanding of data structures and algorithms. • Ability to design and develop scalable, efficient, and maintainable software systems. • Experience with microservice architecture, API development. Machine Learning (ML): • Deep … machine learning principles, algorithms, and techniques. • Experience with popular ML frameworks and libraries like TensorFlow, PyTorch, scikit-learn, or Apache Spark. • Proficiency in data preprocessing, feature engineering, and model evaluation. • Knowledge of ML model deployment and serving strategies, including containerization and microservices. • Familiarity with ML lifecycle management, including More ❯

DATA ENGINEER LONDON BASED £75,000-95,000 PER ANNUM This growing e-commerce trading company are searching for a new Data Engineer. You will be responsible for building and maintaining a new AWS serverless platform. You will also be responsible for building and maintaining data pipelines, using Python. THE COMPANY This E-commerce trading company have been growing since their formation. They are expanding their Data and Analytics team as the CEO has realised how important Data and Technology are to the business and is investing further in Machine … Learning variables. THE ROLE You will be joining an expanding Data and Analytics team, which currently has more than 20 employees. You will be responsible for building and maintaining a new AWS serverless data platform. Improve best coding practices within the Business. Stream real-time Data More ❯

together individuals with Mathematics, Physics and Computer Science backgrounds who are passionate about applying rigorous scientific methods to financial challenges. As a fundamentally data-driven business, our success is heavily linked to the acquisition, processing, and analysis of vast datasets. High-quality, well-managed data forms … the critical foundation for our quantitative research, strategy development, and automated trading systems. As a Data Engineer within our Quantitative Platform team, you will play a pivotal role in building and maintaining the data infrastructure that fuels our research and trading strategies. You will be responsible … diverse datasets - including market, fundamental, and alternative sources - ensuring their timely acquisition, rigorous cleaning and validation, efficient storage, and reliable delivery through robust data pipelines. Working closely with quantitative researchers and technologists, you will tackle complex challenges in data quality, normalization, and accessibility, ultimately providing the More ❯

pipelines. This person will possess a strong background in DevOps practices, machine learning principles, and cloud computing platforms. You will work closely with data scientists and software engineers to streamline the deployment and monitoring of machine learning models, ensuring efficiency and reliability in ML operations. We hire based … with version control systems (e.g., Git). Familiarity with CI/CD tools and practices. Strong problem-solving and analytical skills. Understanding of data structures and algorithms. Ability to design and develop scalable, efficient, and maintainable software systems. Experience with microservice architecture, API development. Machine Learning (ML): Deep … machine learning principles, algorithms, and techniques. Experience with popular ML frameworks and libraries like TensorFlow, PyTorch, scikit-learn, or Apache Spark. Proficiency in data preprocessing, feature engineering, and model evaluation. Knowledge of ML model deployment and serving strategies, including containerization and microservices. Familiarity with ML lifecycle management, including More ❯

Data Analyst 📍 West London (3 days a week on-site) 💸 £55,000 - £60,000 About the Company A leading name in the UK property sector is on the hunt for a mid-level Data Analyst to join their growing data team. With a strong … foundation in Business Intelligence and a forward-thinking approach to tech, this company has spent the last five years investing heavily in their data infrastructure and are now gearing up to take things even further - including AI implementation over the next 6-12 months. 🚀 They're working on … internal data projects with real commercial impact and are looking for someone who can not only crunch the numbers, but also tell the story behind them. The Role You'll take the lead on key analytics initiatives, working end-to-end across data pipelines, visualisation, and More ❯

Contract Opportunity: Senior AWS Machine Learning Data Engineer Location: London | Duration: 6 Months (Extension Possible) | Start Date: ASAP Rate: Up to £600 a day (Inisde IR35) Are you a cloud-native data engineer with a passion for building intelligent systems? I'm looking for a Senior … AWS Machine Learning Data Engineer to a cutting-edge team on a contract basis, working on high-impact ML solutions that scale. About the Role: In this role, you'll lead the design and implementation of data pipelines and ML infrastructure on AWS. You'll collaborate … with data scientists, architects, and product teams to deliver production-grade ML systems across a variety of domains — from predictive analytics to real-time personalization. ? What You'll Do: Architect and build scalable, secure, and maintainable data pipelines on AWS (Glue, Lambda, Step Functions, S3, Redshift More ❯

Wood Mackenzie is the global data and analytics business for the renewables, energy, and natural resources industries. Enhanced by technology. Enriched by human intelligence. In an ever-changing world, companies and governments need reliable and actionable insight to lead the transition to a sustainable future. That's why … the heart of our decisions Future Focused - we accelerate change Curious - we turn knowledge into action Job Description We are looking for a Data Analyst to join Wood Mackenzie's Power and Renewables Data team. This role will focus on the analysis, programmatic acquisition, and preparation … of data for delivery to our customers and research teams. Wood Mackenzie produces research and data that the energy industry relies on to understand and navigate the energy transition. Examples of products that you will contribute to include: Power supply and demand long-term outlooks Power More ❯

Data Engineering Manager £70,000 - £80,000 Hybrid - 2x per week in Portsmouth/London We are representing a large company operating across Scotland and the south of England, that concentrates on the maintenance and upgrade of critical infrastructure to ensure reliable energy supply. They are looking for … an Data Engineering Manager to head up their successful, dynamic team. As the Data Engineering Manager, you will own the data engineering strategy and roadmap, design and architect data pipelines, and manage and optimise the data platform. You will lead … alongside individuals from a 3rd party. We are looking for: Proven experience in managing teams, providing technical guidance and coaching Experience developing the Data strategy, vision, and control budgets Experience with AWS, SQL, and Python If this sounds of interest, please apply today More ❯

Data Engineering Manager £70,000 - £80,000 Hybrid - 2x per week in Portsmouth/London We are representing a large company operating across Scotland and the south of England, that concentrates on the maintenance and upgrade of critical infrastructure to ensure reliable energy supply. They are looking for … an Data Engineering Manager to head up their successful, dynamic team. As the Data Engineering Manager, you will own the data engineering strategy and roadmap, design and architect data pipelines, and manage and optimise the data platform. You will lead … alongside individuals from a 3rd party. We are looking for: Proven experience in managing teams, providing technical guidance and coaching Experience developing the Data strategy, vision, and control budgets Experience with AWS, SQL, and Python If this sounds of interest, please apply today More ❯

Data Engineer Term: Full-time, Permanent Location: High Wycombe with hybrid remote working available Salary: £50000 - £65000 pa depending on experience + excellent benefits Focusrite are looking for an experienced data engineer to join our business intelligence team. You'll help us build and maintain a … robust yet flexible data stack, delivering high quality information where it's needed (and keeping it secure where it's not!). We make extensive use of SQL, dbt, Snowflake and Power BI, as well as various other tools. We work closely together using pair programming, peer review … to learn new things, independently and from each other. We are looking for someone who: Is an SQL expert, able to quickly identify data issues with a pithy query. Has opinions about permissions, schemas, APIs and models, and can share them collaboratively. Knows how to structure, profile and More ❯

products including e-liquids, vape kits, accessories, and CBD items. As the business continues to grow, they’re now looking to add a Data Analyst to their expanding team in North London. As a Data Analyst, the main goal is to turn data into … actionable insights that support strategic, data-driven decisions. This involves gathering, cleaning, analysing, and interpreting data to uncover trends and patterns that drive improved business performance and revenue growth. In this role: You will… Collect, clean, and analyse data from multiple sources to support … strategic decision-making Automate workflows and build efficient data pipelines to improve data collection and processing Identify trends and insights through data analysis, surveys, and competitor research Collaborate with stakeholders to define KPIs and track performance through dashboards and visual reports Ensure data More ❯

tier quants, engineers, and analysts, contributing to systems that power critical investment decisions globally. 🧠 What You’ll Be Doing 📊 Design and develop scalable data pipelines for trading and quantitative analysis 🔧 Automate complex workflows to enhance performance and reduce friction 🤝 Work closely with cross-functional teams to build efficient More ❯

tier quants, engineers, and analysts, contributing to systems that power critical investment decisions globally. 🧠 What You’ll Be Doing 📊 Design and develop scalable data pipelines for trading and quantitative analysis 🔧 Automate complex workflows to enhance performance and reduce friction 🤝 Work closely with cross-functional teams to build efficient More ❯

Conduct independent research to identify opportunities and inefficiencies in financial markets. Create portfolio optimisation tools and risk management models. Maintain and improve codebases, data pipelines, and database integrations. Enhance automation of processes and integrate AI-driven solutions where applicable. Collaborate with IT and investment teams to optimise internal More ❯

Conduct independent research to identify opportunities and inefficiencies in financial markets. Create portfolio optimisation tools and risk management models. Maintain and improve codebases, data pipelines, and database integrations. Enhance automation of processes and integrate AI-driven solutions where applicable. Collaborate with IT and investment teams to optimise internal More ❯